Skip to content
Search

Image segmentation

lastmod 2026-04-03created 2020-06-1525 edits

(For how memory regions are divided in a computer system, see Memory segmentation.)

In computer vision, segmentation is the process of partitioning a digital image into multiple sets of pixels. The goal of segmentation is to simplify or change the representation of an image into something more meaningful and easier to analyze. Image segmentation is used in particular to locate objects and boundaries (lines, curves) in images.

The result of image segmentation is a set of regions that collectively cover the entire image, or a set of contours extracted from the image (see edge detection). Each pixel in a region is similar with respect to some feature or computed property, such as color, intensity, or texture. Adjacent regions are significantly different with respect to the same features.

Applications

Practical applications of image segmentation include:

  • Medical imaging
    • Detection of tumors and diseases
    • Measurement of blood vessel volume
    • Computer-assisted surgery
    • Diagnosis
    • Treatment planning
    • Study of anatomical structure
  • Detection of objects such as roads and forests in satellite images
  • Face recognition
  • Fingerprint recognition
  • Automatic traffic control systems
  • Machine vision

Several general-purpose algorithms and techniques have been developed for image segmentation. Since there is no general solution to the image segmentation problem, these techniques often have to be combined with domain knowledge to solve the segmentation problem of a given domain effectively.

Types

Segmentations-example.png

Summary:

  • Semantic segmentation: classifies pixels by class
  • Instance segmentation: distinguishes individual objects within the same class
  • Panoptic segmentation: combines the two methods to provide a complete segmentation of every pixel in the image

Semantic Segmentation

A technique that assigns a label to every pixel in an image, marking things that belong to the same class with the same label. Individual objects are not distinguished.

Instance Segmentation

A technique that separates and identifies each object in an image, down to its shape. It is an extended form of semantic segmentation.

Features:

  • For example, if there are several cars in an image, each car is recognized as an independent instance and marked separately.
  • That is, even if they belong to the same 'car' category, each car gets a different label.
    • Arbitrary IDs are assigned
    • Overlapping objects are detected individually
    • Objects without a fixed shape, such as sky or road, are not given class labels,

Panoptic Segmentation

A combination of semantic segmentation and instance segmentation: a technique that assigns a label to every pixel in an image while also distinguishing individual objects. This method classifies categories into 'stuff' and 'things'.

Stuff
Semantic segmentation is performed. Refers to objects without a fixed shape. (e.g. 'sky', 'road')
Things
Instance segmentation is performed. Refers to objects that have independent instances (e.g. 'person', 'car')

Anomaly Detection vs Semantic Segmentation

See Anomaly detection#Anomaly Detection vs Semantic Segmentation for details.

Image processing

Algorithms:

Some techniques commonly used in OpenCV image processing

K-Means Algorithm

Finding boundary regions using cv2.kmeans

Contour Detection

Finding boundary regions using cv2.findContours

If the edges are blurry, an enhancement algorithm is used to strengthen them (Image Edge Enhancement)

Morphological Transformations

When boundaries are hard to find, morphological operations are sometimes used to force edges to form.

Thresholding

Mask processing using cv2.threshold

Color Detection

Color range processing using cv2.inRange

Deep learning

Video segmentation

Challenge

See also

Favorite site

author Zer0 sitemap