Digital image processing

lastmod 2026-04-27created 2014-02-11175 edits

Digital image processing (or digital picture processing) is the use of computer algorithms to perform image processing on digital images. As a subfield of digital signal processing, digital image processing has many advantages over analog image processing. It allows a much wider range of algorithms to be applied to the input data and can avoid problems such as noise and signal distortion arising during processing.

Basic mathematics

• 2D Transformation
  • Transformation matrix
• Hungarian algorithm - when you need to solve an assignment problem
• Homogeneous Coordinates - When linking the coordinate system of 3D projected onto a single image with the image, all points on the resulting projection line are expressed in Homogenous coordinates.

Statistics

• Robust Parameter Estimation - If processing still works even when the data contains outliers, it is called robust.
  • Random sample consensus (RANSAC) - A method to keep the least square method model from being skewed when outliers are mixed into the sample.
  • Median absolute deviation

Geometry

• Region of interest (ROI)
• Bounding box (Minimum bounding box)

Image Calibration

• Shake correction (Degrees of freedom)
• Image stabilization - (IS; image stabilization)
• Unwrap labels - e.g., a method of flattening the text on a wine bottle label onto a plane
• Camera Calibration

Image Transformation

• Liquify filter
• Distortion Mirror
• Remapping
  • cv2.remap

Distortion

Distortion (optics) - about "distortion of the image"

• Concave/convex lens distortion (Lens Distortion)
• Radial Distortion

Geometric transformations

• Scaling up/scaling down/rotation
• Homography - geometric transformation and projection of images, such as extracting a 2D image from a camera
• Affine space
• Perspective Transform
  • cv2.getPerspectiveTransform
  • cv2.warpPerspective
• E.T.C
  • cv2.warpPolar - Remaps an image to polar or semilog-polar coordinate space.
• Thin-Plate Spline Shape Transform
  • cv2.createThinPlateSplineShapeTransformer - circular hemisphere shape (picture a dome-shaped CCTV camera) ¹²

Color grading (color correction)

Color correction such as brightness/contrast, color mapping, color harmonization, quantization, or color conversion to another color space

• Gamma correction
  • Python:colorsys
• Image Smoothing/Image Blurring - techniques for blurring an image
  • Gaussian blur - image blurring (smudging)
  • Guided filter - An edge-preserving smoothing light filter.
• Deblurring/Deblur
  • OpenCV: Out-of-focus Deblur Filter
  • OpenCV: Motion Deblur Filter
• Histogram
  • Histogram specification/Histogram matching
  • cv2.calcHist, np.histogram
• k-means - used in Color Quantization, which reduces the number of colors

Image Filtering

• Linear Filter
• Box blur (Box Filter)
  • cv2.boxFilter
• Convolution
  • cv2.filter2D
  • cv2.sepFilter2D

Digital compositing/optical compositing/image compositing

Combining two or more images. E.g. the matte bind used in film

• Seamless Cloning

Pixelation (Pixelized) - censorship

• Bayer filter - (what I used to call a Mosaic filter)
• Demosaicing

Noise reduction (Denoising)

• Dehazing (Single Image Dehazing; a method for removing the blurriness, smoke, fog, etc. called haze (hazy))
• OpenCV: Periodic Noise Removing Filter - periodic noise removal filter

Interpolation, recovering Raw image formats using a Bayer filter

Feature Detection and Description

• Features from accelerated segment test (FAST)
• Speeded up robust features (SURF)
• Harris Corner Detector
• Oriented FAST and rotated BRIEF (ORB)
• Descriptor Matchers (Feature matching)
  • Fast Library for Approximate Nearest Neighbors (FLANN)
  • Brute-Force Matching
  • Accelerated-KAZE Features (AKAZE)

Image registration

Aligning two or more images

• Jaccard index (Jaccard distance; IoU) - region-based similarity calculation
• Template matching - Searches for images similar to the template image.
• Image stitching - Panorama Image

Morphological filtering (edge and corner detection)

• Morphological Transformations - (Mathematical morphology) - image erosion and dilation
  • cv2.erode, cv2.dilate, cv2.morphologyEx
• Laplacian - detects a single lump (Blob) in an image that clearly stands out from the surrounding pixels
• Image Contours
  • Convex hull - outermost contour detection
  • Hough transform - Extracts the most overlapping parts among the straight lines formed by the points in an image. Mainly used to detect lines or circles.
  • Canny edge detector - outline extraction
  • Sobel operator
  • Scharr
  • Laplacian
  • Contours - extracting image boundaries.
    • cv2.findContours
  • Connected Components - Finds similar components in an image. Can be used for object extraction.
• Optical flow - computes the optical flow of an image
• Image Pyramid; Pyramid (image processing); Pyramid representation

Object Recognition

Same as Object detection.

• Cascading classifiers - recognizes objects using filters.

Image segmentation

• Watershed - image segmentation
  • cv2.watershed
• GrabCut (Graph cuts) - foreground extraction
  • cv2.grabCut
• Background subtraction/Foreground detection
  • Background differencing
    • cv2.accumulateWeighted - weighted accumulation function for computing a moving average
    • cv2.absdiff - static background differencing
• Mean shift filter
  • cv2.pyrMeanShiftFiltering
• Distance transform (Distatnce Transformation)
  • cv2.distanceTransform(src, distanceType, maskSize)
• Connected-component labeling (labeling; Labeling)
  • cv2.connectedComponents
  • cv2.connectedComponentsWithStats
• Flood fill - filling an adjacent region with color (like the fill tool in Paint)
  • cv2.floodFill
• OpenCV: Anisotropic image segmentation by a gradient structure tensor - anisotropic image segmentation by a gradient structure tensor
  • In mathematics, the gradient structure tensor (also called the second-moment matrix, second-moment tensor, inertia tensor, etc.) is a matrix derived from the gradient of a function.
  • It summarizes the predominant directions of the gradient near a given point and the degree to which those directions are consistent (coherence).
  • The gradient structure tensor is widely used in image processing and computer vision for 2D/3D image segmentation, motion detection, adaptive filtering, local image feature detection, and more.

Wide dynamic range (WDR)

Creating an HDR (High Dynamic Range) image by combining multiple images

• OpenCF:HDR

2D object recognition using geometric hashing

Centroid tracking and correlation tracking

• Motion estimation (Visual tracking; Video Segmentation)
  • Blob tracking with (Blob detection)
  • Kalman filter - predicts the next state of an object from its previous state (e.g. mouse pointer tracking)
  • OpenCV APIs
    • cv2.TrackerBoosting_create
    • cv2.TrackerMIL_create
    • cv2.TrackerKCF_create
    • cv2.TrackerTLD_create
    • cv2.TrackerMedianFlow_create
    • cv2.TrackerGOTURN_create
    • cv2.TrackerCSRT_create
    • cv2.TrackerMOSSE_create
• Image moment - used to find the center of an image.
  • cv2.moments

Fragmentation grouping

Groups multiple fragmented objects and the like into a single object.

• Non-Maximum Suppression - A method that keeps only the highest-scoring IoU among overlapping IoUs and removes the rest. Used in yolo

Properties of Light.

• Glare-free/Anti-Glare/Anti-Reflection - removing reflected light/glare in image processing (burning effect?)

3D Reconstruction

Methods for reconstructing 3D models without depending on 3D annotation data

• SLAM
• Point Cloud

Machine Learning OpenCV

• ANN_MLP
• DTrees
• Boost
• RTrees
• EM - Expectation Maximizaion
• KNearest
• LogisticRegression
• NormalBayesClassifier
• SVM
• SVMSDG - Stochastic Gradient Descent SVM.

Application areas

• Computer vision (Vision Inspection System; Vision Inspection; Machine vision)
• Ordering pictures (sorting)
• Augmented reality
• Facial recognition system
• Feature detection
• Lane departure detection
• Non-optical imaging
• Medical image processing
• Microscope image processing
• Morphological image processing
• Remote sensing
• Fire Detection
• Image restoration
• Concrete Cracks Detection
• Pedestrian detection (Human detection)
• Traffic signal (sign) recognition (Traffic Signs Recognition)
• Fingerprint Recognition (Fingerprint)
• License Plate Recognition
• Car Counting (Vehicle Counting)
• Lane Detection

Libraries

• OpenCV
• Dlib
• OpenMP
• Threading Building Blocks (Intel Threading Building Blocks) (TBB)

Terms

Glitch

A term for a temporary error in a system. Picture a TV screen turning into a smeared mess when the signal is not received.

Documentation

Regular paper 10-15-2-10: Object Tracking Algorithm Using Weighted Color Centroids Shifting

Object_Tracking_Algorithm_Using_Weighted_Color_Centroids_Shifting.pdf ³

http://ksbe-jbe.org/bbs/download.php?bo_table=m21&wr_id=367&no=0&ei=9MzRU-OwMIa6uATCiIGACA

NTHU Rain Removal Project - removing rain from images

http://www.ee.nthu.edu.tw/cwlin/Rain_Removal/Rain_Removal.htm

Adaptive Airlight Detection for Dark Channel Prior-based Haze Removal

https://www.kics.or.kr/storage/paper/event/summer/publish/9D-9.pdf

Retinex algorithm (image brightness adjustment)

https://flowergom.wordpress.com/2011/11/01/retinex-algorithm/

http://kipl.tistory.com/33

Enhancement of Haze Removal using Transmission Rate Compensation

http://society.kisti.re.kr/sv/SV_svpsbs03V.do?method=download&cn1=JAKO201311637858548

See also

• OpenCV:Category - OpenCV categories organized with screenshots.
• OpenCV
• Digital image (image)
• libvips - a very fast image processing library with low memory usage

Favorite site

• Wikipedia (en) description of digital image processing

References