A Framework for Interactive 3D Segmentation with Adaptive Masking and Fine-Grained Version Control

Fine-tuning remains an important step in adapting 3D foundation models to downstream applications, but it is laborious and time-consuming to annotate training datasets in 3D. Existing semi-automatic annotation methods for segmentation masks fail to generalize beyond RGB point cloud data and are difficult to control. While a number of 3D annotation interfaces provide undo/redo functionality to help correct mistakes, they lack a comprehensive data versioning strategy, limiting the potential for user analysis and quality control. To tackle these issues, we present DynamicSAM Annotator, an AI-assisted point segmentation framework with the following contributions: 1) a novel interactive 3D point cloud segmentation model that exclusively uses XYZ coordinates with an adaptive thresholding mechanism in segmentation masks; 2) a 3D annotation platform to enable efficient 3D point cloud annotation for semantic and instance segmentation tasks; and 3) a fine-grained version control system that efficiently persists each annotation operation to disk using Git-like operations. Rigorous evaluations across seven benchmark datasets demonstrate DynamicSAM's effectiveness in both indoor and outdoor environments, achieving an average Intersection over Union (IoU) of 59.6% for a single click, highlighting the model's outstanding generalization capabilities. Meanwhile, our simulated benchmark shows that our version control system maintains sub-second latency even with tens of thousands of snapshots stored.