Reproduce “Agile Deliberation: Concept Deliberation for Subjective Visual Classification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Leijie Wang, Otilia Stretcu, Wei Qiao, Thomas Denby, Krishnamurthy Viswanathan, Enming Luo, Chun-Ta Lu, Tushar Dogra and 2 more
Agile Deliberation introduces an interactive human-in-the-loop framework designed to assist users in defining subjective visual concepts and training aligned classifiers. The framework yielded an average F1 score improvement of 10.5% over zero-shot baselines and significantly lowered user cognitive effort compared to manual deliberation.
Key insights
- Explicitly supporting 'concept deliberation'—an iterative process of exploring ambiguous cases and refining definitions—is crucial for effective human-AI alignment in subjective visual classification.
- Surfacing semantically ambiguous examples, rather than just model-uncertain ones, significantly reduces user cognitive load and improves classifier performance by guiding attention to critical decision boundaries.
- Integrating automated prompt optimization with user feedback allows non-experts to effectively encode nuanced, evolving concept definitions into VLM-based classifiers without needing to craft precise prompt syntax.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.