Reproduce “Visual Recognition by Request” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chufeng Tang, Lingxi Xie, Xiaopeng Zhang, Xiaolin Hu, Qi Tian
A new paradigm, Visual Recognition by Request (ViRReq), enables AI systems to achieve arbitrarily fine-grained visual understanding through an iterative, query-driven process guided by a hierarchical text-based knowledge base. This approach efficiently learns from incomplete annotations and supports incremental learning, significantly outperforming traditional methods on complex hierarchical datasets like ADE20K.
Key insights
- Decomposing visual understanding into atomic, interactive requests, guided by a flexible knowledge base, resolves the inherent conflict between recognition granularity and annotation certainty.
- A hierarchical, text-based knowledge base, coupled with language-driven queries, facilitates open-domain recognition and allows for dynamic expansion of visual concepts.
- Models can effectively learn from highly incomplete hierarchical annotations by selectively processing requested segments and identifying undefined regions as 'others'.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.