Reproduce “CLIPSym: Delving into Symmetry Detection with CLIP” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tinghan Yang, Md Ashiqur Rahman, Raymond A. Yeh
The CLIPSym framework combines pre-trained CLIP models with a rotation-equivariant decoder and Semantic-Aware Prompt Grouping to detect both reflection and rotation symmetries in images. It establishes new state-of-the-art F1-scores on the DENDI dataset, achieving 66.5% for reflection and 25.1% for rotation, demonstrating improved robustness and accuracy.
Key insights
- Vision-language models implicitly encode valuable symmetry information from their diverse pre-training data, which can be effectively extracted for geometric tasks.
- Integrating robust semantic cues from CLIP with a principled rotation-equivariant decoder dramatically improves symmetry detection performance and consistency under transformations.
- Semantic-Aware Prompt Grouping (SAPG) provides an effective strategy for guiding foundation models to abstract geometric concepts like symmetry, addressing data scarcity challenges.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.