Reproduce “Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sebastian Cavada, Francesco Pelosin, Lapo Faggi
A training-free framework enables fine-grained semantic segmentation in low-data environments by combining class-agnostic segmentation from SAM3 with prototype-based classification using PCA-whitened DINOv3 embeddings. This approach yields a mean class accuracy of up to 0.55 and a mean Intersection over Union of 0.31 on the FungiTastic dataset.
Key insights
- PCA whitening of DINOv3 embeddings significantly improves prototype-based classification performance by over 0.20 mAcc, enhancing the discriminative power of features by mitigating nuisance variability.
- Decoupling class-agnostic segmentation from fine-grained classification offers a scalable solution, maintaining constant segmentation cost regardless of the number of fine-grained classes.
- The geometry of the embedding space is crucial for effective prototype-based classification, with specific transformations like PCA whitening improving feature utility in low-data regimes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.