Data annotation

lastmod 2026-09-30created 2024-01-118 edits

Data annotation is labeling or tagging the relevant information/metadata in a dataset so that machines can understand the information. A dataset can be in any format, such as images, audio files, video footage, or text.

Data is one of the three key elements of AI development. Developing computer recognition engines requires large-scale training data that must be annotated. Such data includes images, sounds, text, and more. Annotations for these types of data include classification, frames, annotations, markings, and so on.

Categories

• Annotation Tool - Collection of tools for Data annotation work used in machine learning and more.
• AI-Hub
• Label backpropagation (Label Propagation)

Labeling caveats

Classification

• Auto-labeling
  • Automates labeling using AI and machine learning algorithms.
  • A model labels data automatically, and a person reviews and corrects it.
• Semi-automatic Labeling
  • A labeling tool generates initial labels, and a person reviews and corrects them.
  • A way to improve efficiency while maintaining accuracy.
• Cloud-based Labeling
  • Labeling work is performed through a cloud platform.
  • Easy data storage, collaboration, and accessibility.
• Collaborative Labeling
  • Provides collaboration features so that several people can label at the same time.
  • Easy project management and division of work.
• Data Augmentation
  • Transforms labeled data to expand the dataset.
  • A way to improve model performance.

3D point cloud labeling methods

Assigning a class id and an instance id to each point as scalar values (per-point labeling) is the most common representation in 3D point cloud semantic / instance / panoptic segmentation labeling.

Almost all public datasets and training frameworks ultimately converge on the form of two integer arrays (semantic, instance) the same length as the number of points.

However, no official standard (ISO, etc.) exists, and the storage format differs by dataset.

De facto standard formats

Even when the storage structure looks different, as with ScanNet and S3DIS, everything is converted into a per-point semantic array + instance array at the loader stage. Therefore the per-point scalar field approach is the most general form as a representation of the source data.

Labeling tools

Segmentation tools

• CloudCompare — Cut out a region (Segment) → fill a scalar field with a constant → merge. A very common workflow in labs and small industrial projects
• point_labeler — The tool used to create SemanticKITTI
• Semantic Segmentation Editor — Made by Hitachi
• Commercial tools — Segments.ai, Supervisely, BasicAI, etc.

The output of all the tools above is ultimately a per-point label array.

Detection tools (for reference)

• CVAT, labelCloud, etc. are mainly for 3D bounding box (cuboid) labeling
• For detection tasks, a different family from segmentation

Practical recommendations

1. Manage the class mapping file separately

• Write a yaml/json file that specifies the id → name mapping and the ignore label value (usually -1 or 255)
• Version-controlling it together with the data is the de facto convention

2. Define instance id rules

• Assign uniquely within a scene
• Set background/stuff classes to -1 (or 0)

3. Caveats for CloudCompare scalar fields

• Scalar fields are stored as float by default
• When exporting to PLY, the scalar_ prefix is added to the property name (e.g., scalar_class)
• Match the property name in the loader and cast to integer:

# Round float scalar field and cast to int / float 스칼라 필드를 반올림 후 정수로 변환
segment = np.rint(ply["vertex"]["scalar_class"]).astype(np.int32)

4. Separate source and training formats

• Keep the source as PLY (scalar fields)
• For training, keep a separate script that converts to the format the framework requires (e.g., Pointcept's segment.npy, instance.npy)
• Separating the labeling tool from the training code makes management easier

See also

• Artificial intelligence (AI)

Favorite site

• Data annotation - Simple English Wikipedia, the free encyclopedia
• Annotation - Wikipedia
• What is data labeling? - Definition, types, and comparison of Korean vendors - Datahunt
• Story 9 - Data labeling, data annotation
• What is Data Labeling? - Data Labeling Explained - AWS
• (Deep learning) What is data annotation, annotation techniques
• Cutting 2.16 million won of labeling down to 90,000 won | GeekNews
  • [Original] We cut 2.16 million won of labeling down to 90,000 won | NARUBROWN's tech blog
  • Cutting image labeling costs by 96%: "Leave the human work to the system" covers a real-world engineering case where, even in an environment short on budget and time, repetitive human labor was replaced with a software pipeline to implement a core feature
  • Key summary
    • Problem: no existing model was suitable for automatically recognizing and registering famous character dolls, and human labeling had clear limits in cost, speed, and scalability.
    • Approach: not "should we add more people," but decomposing the human decision process into a system and turning it into a pipeline.
  • Four-stage automation pipeline design
    1. CLIP filtering – removes large numbers of meaningless images to cut LLM costs
    2. YOLO detection – crops only the main object to narrow the scope of analysis
    3. LVM labeling – applies a high-performance VLM only to refined data
    4. LVM verification – conditional verification based on confidence further reduces the number of calls