We present Convpaint, a universal computational framework for interactive pixel classification. Convpaint uses pretrained convolutional neural networks (CNNs), vision transformers (ViTs), or classical filter banks for feature extraction in combination with fast-to-train machine learning (ML) classifiers to enable easy segmentation across a wide variety of tasks. By integrating ViT-based features, Convpaint extends traditional pixel classification to image domains that require rich semantic understanding. Convpaint's modular design allows users to rapidly switch between feature extractors, balancing speed, spatial accuracy, and semantic depth based on the specific dataset. Available within the Python-based napari software ecosystem, Convpaint integrates seamlessly with other plugins into image processing pipelines, which we demonstrate with example workflows across different data modalities, from subcellular to cellular to animal scale.
Hinderling et al. (Thu,) studied this question.