Reproduce “Advancing Fine-Grained Classification by Structure and Subject Preserving Augmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eyal Michaeli, Ohad Fried
SaSPA is a generative data augmentation method designed for fine-grained visual classification, which generates diverse yet class-fidelity-preserving synthetic images by combining Canny edge maps, BLIP-diffusion subject representations, and LLM-generated prompts. This approach consistently outperforms existing methods across various FGVC benchmarks and few-shot learning scenarios, while also effectively mitigating contextual bias.
Key insights
- Combining abstract structural (Canny edges) and subject-level (BLIP-diffusion reference) conditioning allows for high fidelity and diversity in generated images without requiring computationally intensive generative model fine-tuning.
- Using separate source images for edge extraction and subject reference is key to maximizing beneficial diversity while preserving fine-grained class characteristics.
- Metrics like FID may not accurately reflect the utility of synthetic data for downstream classification tasks; instead, higher LPIPS scores indicating greater perceptual diversity correlated better with improved model performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.