Skip to content
Search

Feature detection

lastmod 2025-02-11created 2016-01-2640 edits

In computer vision and image processing the concept of feature detection refers to methods that aim at computing abstractions of image information and making local decisions at every image point whether there is an image feature of a given type at that point or not. The resulting features will be subsets of the image domain, often in the form of isolated points, continuous curves or connected regions.

Categories

Features in brief

Problems with the Harris, GFTT, and FAST corner detection methods

The Harris, GFTT, and FAST corner detection methods are robust to translation and rotation, but have the drawback of being vulnerable to scale changes.

Edge detection

Corner detection

Blob detection

  • Laplacian of Gaussian (LoG)
  • Difference of Gaussians (DoG)
  • Determinant of Hessian (DoH)
  • Maximally stable extremal regions
  • PCBR
  • SimpleBlobDetector

Ridge detection

Hough transform

Structure tensor

  • Structure tensor
  • Generalized structure tensor

Affine invariant feature detection

  • Affine shape adaptation
  • Harris affine
  • Hessian affine

Scale space

  • Scale-space axioms
  • Implementation details
  • Pyramids

Feature description

To perform Feature Detection on an image, it is important to find distinctive patterns or structures within the image. For this, algorithms such as SIFT, SURF, and ORB are used; these algorithms detect keypoints in the image, extract local patterns around them, and generate feature descriptors.

Scale-invariant feature transform (SIFT)
A keypoint detection algorithm that is invariant to image scale and rotation
Its features are robust to image scale and rotation, so it shows stable performance in most situations
However, SIFT consumes a lot of computing resources
Speeded up robust features (SURF)
An algorithm that improves the speed of keypoint detection and descriptor computation over SIFT
Offers performance similar to SIFT while being faster
Oriented FAST and Rotated BRIEF (ORB)
Combines the FAST and BRIEF algorithms; an algorithm that is much faster than SIFT and SURF while still performing well
Its keypoint detection results do not change when the image is rotated or rescaled
Widely used, especially as an algorithm suited to real-time computation on embedded devices and mobile apps
Features from Accelerated Segment Test (FAST)
An algorithm that detects keypoints in an image quickly
Because it is simple and fast, it is often used together with other algorithms: initial keypoint detection is followed by a more accurate detection algorithm (e.g. BRIEF, SIFT, SURF)
Binary Robust Invariant Scalable Keypoints (BRISK)
An algorithm published in a 2011 paper; to detect keypoints in an image, it creates patches of specific sizes in scale space and detects keypoints with the FAST method using the intensity difference within each patch
To describe keypoints, it uses a binary descriptor built from the distribution of intensity difference values
Accelerated-KAZE (AKAZE)
An algorithm published in 2013 that uses the KAZE method to detect keypoints in an image
KAZE blurs the image in a multiscale manner in the image's scale space and uses the Difference of Gaussian (DoG) to find extrema within the scale-space. It then detects keypoints at the extrema using the Hessian matrix
AKAZE is a variant of KAZE designed to process this faster
After detecting keypoints in the scale-space, AKAZE extracts the direction of each keypoint and generates the keypoint's descriptor according to that direction
Gradient Location and Orientation Histogram (GLOH)
GLOH is a method for extracting strong features for object recognition and image classification. The feature vector is generated by combining gradient orientation and location information within a specific region of the image.
Histogram of oriented gradients (HOG)
HOG is a feature extraction method mainly used for person recognition, object detection, and image classification. It extracts shape features from a given image to generate a vector that reflects the object's shape well.

Comparison of the algorithms

Main OpenCV keypoint algorithms and descriptor characteristics

Feature_detection_-_compare_1.png

Computation speed comparison

Feature_detection_-_compare_2.png

ORB is the fastest because it implements the FAST algorithm internally.

Repeatability comparison

Feature_detection_-_compare_3.png

Keypoint detection algorithm performance comparison

Feature_detection_-_compare_4.png

ORB has poor matching quality and detection results, but it is used often because it is fast.

For matching performance, AKAZE is better.

SHIFT has the best matching performance but the slowest computation time.

Feature matching

The core of Feature Matching is comparing the feature descriptors extracted from two images to find pairs of corresponding keypoints. Matching methods mainly include Nearest Neighbor matching and matching using RANSAC (Random Sample Consensus).

OpenCV APIs

cv2.BFMatcher()
Performs feature matching using the brute-force matching algorithm
Shows high accuracy especially on small data sets; in most cases it is slower than cv2.FlannBasedMatcher() but gives better results
cv2.FlannBasedMatcher()
One of the feature matching algorithms for fast Nearest Neighbor search
Uses the FLANN (Fast Library for Approximate Nearest Neighbors) algorithm to find the Nearest Neighbor between feature descriptors
Matches similar features in two images by running a k-NN (k-Nearest Neighbor) search over the set of feature descriptors. By default, cv2.FlannBasedMatcher is set to k=2
  • index_params: parameter that configures the index for Flann-based matching. Passed as a dictionary, with two sub-parameters, algorithm and trees
    • algorithm : the algorithm to use for matching; usually one of the constants of the cv2.flann.IndexParams class
    • trees : the number of trees used for matching
  • search_params: parameter that sets the number of neighbors to search and the maximum search distance when matching. Passed as a dictionary, with two sub-parameters, checks and eps
    • checks : the number of neighbors to search
    • eps : the maximum search distance

Deep learning based

  • LoFTR - Detector-free local feature matching with transformers

Feature point

To obtain the corresponding points of certain points, the points need to be reliable and distinctive (since only corresponding points should be matched with each other).

Such feature points can be used for

  • Image alignment (homography, fundamental matrix)
  • 3D reconstruction
  • Motion tracking
  • Object recognition
  • Indexing and database retrieval
  • Robot navigation

and more.

See also

Favorite site

References

  1. UpdateOnFeatures_StefanHaller.pdf ↩

  2. Datascienceschool.net_-_7eb4b2a440824bb0a8c2c7ce3da7a4e2.ipynb.zip ↩

author Zer0 sitemap