Image Feature Extraction: Methods, Techniques, and Applications

Key takeaways

Image feature extraction identifies information in raw image data that is useful for image processing, data analysis, and computer vision tasks [1].

• Image feature extraction is useful for image classification, object detection, object segmentation, image matching, image stitching, and object tracking.

• Traditional image feature extraction methods include edge, corner, texture, shape, and color features, while deep learning methods include convolutional neural networks and vision transformers.

• You can use Python, JavaScript, and MATLAB for extracting features from images.

Explore image feature extraction, including traditional and deep learning methods. Then, if you're ready to gain cutting-edge AI skills, consider enrolling in DeepLearning.AI's Deep Learning Specialization. This five-course series is designed to help you master the fundamentals of deep learning by building and training deep neural networks, identifying key architecture parameters, implementing vectorized neural networks, applying deep learning to applications, and more.

What is image feature extraction?

Image feature extraction identifies information in raw image data that is useful for image processing, data analysis, and computer vision tasks [1].

A feature refers to an aspect of a digital image or of something in an image, such as a shape, object, or structure. Corners, edges, shapes, colors, textures, and regions are some commonly extracted types of image features. Deep learning approaches can extract more complex patterns as features.

But why do you need features for vision tasks? Unlike raw image data, features are more informative and compact.

What is an example of feature extraction?

An example of feature extraction is the process of creating a wide panorama from a series of photos. You can extract key features from each photo, then, using these features, identify overlapping patches and align them seamlessly to create the panorama.

Why is image feature extraction useful?

Image feature extraction is a key step in several image processing and computer vision tasks. Some common tasks and their relevant features include image classification, object detection, object segmentation, image matching, image stitching, and object tracking.

Image classification

Image classification is the process of assigning a label or category to an image based on its entire content. For example, a photo of a car is labeled with its make or categorized as a car or vehicle. 

Salient image features are extracted from all over the image and supplied to classification algorithms to predict a label or category.

Object detection

Object detection identifies and locates objects of one or more relevant classes, such as persons or vehicles, in an image. It is useful for a variety of applications, such as robotics and medical image analysis. 

For example, the Viola-Jones algorithm detects all the faces in a photo by extracting a type of regional features called Haar-like features.

Image segmentation

Object segmentation is a specialized type of object detection where, instead of identifying just the bounding rectangles, the true outlines of each object are identified. Algorithms like GrabCut, Watershed, and U-Net are some popular choices.

Since these segment outlines involve complex contours, texture, and color feature extraction are frequently used. Modern deep learning techniques identify complex hidden patterns to use as features [2].

Image matching

Image matching answers questions like:

• Is a set of photos depicting the same scene from different camera angles?

• Is an object in one photo present in another photo, possibly with a different size and orientation?

Image matching uses a set of key interest points and their descriptors that don't change with rotation, scaling, or illumination.

Image stitching

Image stitching creates seamless collages or panoramas from multiple photos by identifying their overlapping regions. One of the uses of image matching using keypoint features is in stitching. Overlapping areas are matched using keypoint features [3].

Object tracking

Object tracking, also called motion tracking, follows the location of an object across multiple images of the same scene. For example, you can use it to track a person or vehicle in a video.

Tracking algorithms like mean shift use color features, while others like Lucas-Kanade optical flow rely on corner features [4].

What’s the difference between traditional and deep learning image feature extraction?

You can extract features using either traditional techniques or modern deep learning approaches.

Traditional techniques are manually designed for extracting specific features. Most of them work on any image without specific knowledge or training on a data set, but are also sometimes combined with traditional machine learning training. Many of these traditional techniques are also quite sensitive to image changes and occlusions.

Deep learning approaches automatically learn features from the images they're trained on. They also outperform traditional techniques on most vision tasks. However, they require more computing power and training time.

Should you still learn traditional image feature extraction methods?

You may find traditional techniques useful even as a deep learning practitioner. By learning them, you can not only employ them for common tasks but also develop better intuition about how and why deep learning approaches work.

Traditional techniques may help you in these common tasks:

• Process images on low-resource devices: Many traditional algorithms run efficiently with acceptable accuracy on devices with weak processors or limited memory.

• Prepare training data sets: You may sometimes have to fine-tune or train your deep learning models. For that, you need to prepare and augment suitable image data sets. You may find legacy image feature extraction quicker, easier, and good enough for such data set preparation.

Read more: The Best Way to Learn Deep Learning

What are the main traditional image feature extraction methods?

Traditional techniques are handcrafted algorithms to extract features, such as edges, corners, textures, blobs, colors, and keypoints.

Edge features

Edges are regions of sharp changes in image intensity. The Canny edge detection algorithm is a popular gradient-based technique to extract edge features.

Corner feature extraction

A corner is a distinctive, rapid change in curve direction that's easier to match than lines. The Harris corner detector is a gradient-based approach that extracts the same corner features even if the orientation or illumination changes. The Shi-Tomasi detector is an improved version of it.

Texture feature extraction

Texture features are the surface characteristics of an image region. Some techniques include local binary patterns, gray level co-occurrence matrix, and Gabor filters.

Shape and geometry features

Shape features capture geometric characteristics of a shape, such as its area and centroid. The Hough transform can identify shapes, such as lines, circles, and rectangles. The generalized Hough transform can identify any arbitrary shape.

Blob features

A blob feature is a local region of similarity that is distinct from its surrounding regions. You can extract them using algorithms such as the Laplacian of Gaussian or the determinant of the Hessian.

Color-based features

Color attributes are widely used as informative and intuitive features.

A color histogram is a statistical feature that indicates the frequency distribution of different color intensities in an image.

Color moments are also statistical features about color distribution, such as its mean, variance, and skewness.

Keypoints and feature descriptors

A keypoint is a distinctive feature in an image, such as a corner, and its feature descriptor is a vector that describes the region around it.

The scale-invariant feature transform (SIFT) is a technique to obtain a feature descriptor that doesn't change even if the scale, rotation, or illumination of the image changes. The speeded-up robust features (SURF) technique is a more efficient version of SIFT [5].

How does deep learning extract image features?

Two modern deep learning methods for computer vision are the convolutional neural network (CNN) and the vision transformer (ViT).

Convolutional neural networks

A CNN is a deep learning architecture whose building block is the convolution operator. It consists of convolutional, fully-connected, and additional layers. The early layers extract simple features, such as edges and corners. Each subsequent layer extracts increasingly complex features, such as shapes and object parts.

Vision transformers

A ViT is a deep learning architecture with the attention algorithm as its building block. It consists of many self-attention layers and feedforward layers. While a CNN's convolutional filters can identify features only locally, a ViT's self-attention layers can identify global context and long-range dependencies in each layer.

Which tools can you use for image feature extraction?

Look into these languages and libraries for extracting features from images:

• Python: For traditional techniques, consider using scikit-image or OpenCV-Python. For extracting features with ViTs, look into the Hugging Face transformers package. If you want to extract using CNNs, use PyTorch's torchvision.models.feature_extraction package. 

• JavaScript: For traditional feature extraction in web applications, use OpenCV.js. For extraction using deep learning, use TensorFlow.js.

• MATLAB: Use the computer vision toolbox.

How do you choose the right image feature extraction method?

If you need computational simplicity or interpretability of results, try the traditional approaches first. They're also suitable for tasks like image stitching, where specific class knowledge is not required.

For higher accuracy, use deep learning approaches or combine them with traditional approaches.

Explore more with our free deep learning and AI resources

Get insights into in-demand skills and career trends by subscribing to our LinkedIn newsletter, Career Chat. Then, discover more about deep learning and AI:

• Find a path: AI Career Paths: Explore Roles & Specializations

• Watch on YouTube: Linear Regression Explained | Python Machine Learning Tutorial with Real Data

• Bookmark for later: JavaScript Cheat Sheet

With Coursera Plus, you can learn and earn credentials at your own pace from over 350 leading companies and universities. With a monthly or annual subscription, you’ll gain access to over 10,000 programs—just check the course page to confirm your selection is included.  

订阅 Coursera Plus，培养就业技能

开始为期 7 天的免费试用

• 
• 
• 
• 

开始为期 7 天的免费试用