Reproduce “ID-Sim: An Identity-Focused Similarity Metric” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Julia Chae, Nicholas Kolkin, Jui-Hsien Wang, Richard Zhang, Sara Beery, Cusuh Ham
ID-Sim is a new identity-focused similarity metric designed to emulate human selective sensitivity for visual identities, distinguishing intrinsic object properties despite contextual changes. It consistently outperforms seven baseline models across 49 evaluation setups in identity recognition, retrieval, and preservation tasks, while offering more efficient and deterministic evaluations than MLLMs.
Key insights
- Human-like "selective sensitivity" to visual identities, balancing invariance to context with sensitivity to identity-altering changes, can be effectively learned by a deep metric.
- A robust training strategy combining carefully curated real-world instance data with strategically generated synthetic identity-preserving and identity-altering examples is key to developing such a metric.
- Jointly learning global (CLS token) and spatially-aware local (patch token, via optimal transport) embeddings within a contrastive framework is crucial for achieving both high contextual invariance and fine-grained discrimination.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.