3D Point Cloud Segmentation

lastmod 2026-06-23created 2026-06-233 edits

A summary of implementation methods, SoTA models, frameworks, and labeling tools and methods for segmenting 3D point cloud data acquired with LVS scanners and similar devices using deep learning.

3D point cloud segmentation (3D Point Cloud Segmentation) is a deep learning task that assigns a class to each point of point cloud data acquired with LVS laser scanners and similar devices. This page is organized around small-scale, high-precision, domain-specific scenarios (e.g., weld bead inspection), not large-scale outdoor scenes for autonomous driving.

Overall implementation

Typical pipeline: data preparation → preprocessing → labeling → training → inference/post-processing

Preprocessing: LVS scans are dense point clouds built from accumulated line profiles, so noise/outlier removal (SOR, ROR), downsampling (voxel grid), and coordinate normalization are important. The PTv3 family is based on voxel serialization, so the voxel size directly affects performance.
Semantic vs Instance: if you only need to distinguish types, use semantic segmentation; if you need to distinguish individual objects (bead 1/bead 2), use instance segmentation.
Handling class imbalance: the key difficulty with industrial data. (a) weighted CrossEntropy + Lovász-Softmax, (b) point-level oversampling/copy-paste augmentation, (c) self-supervised pretraining to stabilize initial representations.
Small-data strategy: self-supervised pretraining (Sonata) → fine-tune with a small number of labels, or generate large numbers of pseudo-labels with CAD-based automatic labeling.

SoTA deep learning models

Model	Type	Features	Recommended use
PTv3 (Point Transformer V3)	Transformer	Best balance of speed/memory/accuracy, current standard backbone	First-choice backbone, fine shape detection
Sonata (2025)	Self-supervised pretraining	Self-distillation on a PTv3 backbone	Few labels; fine-tune after pretraining
OA-CNNs / MinkowskiNet	Sparse Voxel CNN	Lightweight, cost-effective for simple tasks	Simple separation such as defect presence
KPConv / PointNeXt	Point-based	Proven baseline	Comparison group

¹Sonata (2025) set a new SoTA on a range of indoor and outdoor 3D perception tasks with self-distillation-based self-supervised learning.
If fine shapes (weld defects) matter, a transformer (PTv3) is better; for simple region separation, a sparse CNN is better.

Open-source frameworks

Pointcept — first choice. Integrates PTv3, Sonata, PTv2, OA-CNNs, and PointGroup. You only need to write a custom dataset adapter.
MMDetection3D (OpenMMLab) — strong in model variety and config management, somewhat heavy.
Open3D-ML — integrates preprocessing, visualization, and simple models. Use alongside as an auxiliary tool.
Torch-Points3D — maintenance has stalled; not recommended for new projects.

INFORMATION
On DGX Spark (GB10/aarch64), check the PyTorch ARM64 build and aarch64 compilation issues with spconv/Minkowski in advance.

Labeling tools

Tool	License	Features
CloudCompare	Open source	Standard for precise editing of industrial point clouds, plugins/scripting. First choice
CVAT	Open source	SAM-based AI-assisted labeling, web/collaboration
Semantic Segmentation Editor (Hitachi)	Open source	Flexible point cloud segmentation labeling
Segments.ai	Commercial	Multi-sensor labeling platform, for outsourcing/large scale

Labeling methods

Core principle: painting segments based on over-segmentation

Labeling of benchmark datasets is standardly done with an over-segmentation-based approach, neither selecting individual points nor using cubes (boxes). Backbone models such as PTv3 do not define a labeling method and use the dataset labels as-is, so the actual method has to be looked at per dataset.

ScanNet approach (de facto standard)

Automatic over-segmentation first: ²apply normal-based graph-cut to the mesh to split it in advance into small geometric segments.
Assign labels per segment: in a web interface, click over-segmented segments to assign classes (per piece, not per point).
Segments are small enough that most contain only a single object → painting per piece still yields point-level labels.

Cost: ³ScanNet labeling took 500 workers an average of 22.3 minutes per scene. Impossible if each point had to be clicked individually.
ScanNet++: ⁴free-text instance labels assigned on top of decimated mesh over-segmentation, about 1 hour per scene on average.

Other datasets

Matterport3D: ⁵floor plan → extract room-level regions → paint object instances with the ScanNet painting interface (hierarchical).
S3DIS: split the space into rooms, then select point regions per object within each room.
SemanticKITTI: LiDAR (no mesh), so multiple scans are accumulated into one coordinate system and then regions are painted.

What "cubes" are for

Cubes (3D bounding box / cuboid) are the labeling method for object detection datasets (KITTI/nuScenes/Waymo detection) and are unsuitable for segmentation, which needs a label on every point. However, in practice a hybrid that uses them as a rough first-pass selection tool (select a region with a box → assign in bulk → refine boundaries) is common.

Applying to weld inspection: summary

Recommended workflow:

Generate large numbers of pseudo-labels via CAD registration (synergy with digital twins)
Precisely correct only a small number with CloudCompare
Sonata self-supervised pretraining on unlabeled scans
PTv3 fine-tune (handle imbalance with Lovász + class weighting)
Gradual improvement with human-in-the-loop

Key lesson: "Split first with over-segmentation, then paint piece by piece." In industrial settings, replacing ScanNet's "automatic over-segmentation → paint pieces" with "CAD registration → transfer surface regions" gives the same efficiency with higher precision.

Favorite site

References

¹ ² ³ ⁴ ⁵

</references>

author Zer0 sitemap