Reproduce “Beyond Labels: Information-Efficient Human-in-the-Loop Learning using Ranking and Selection Queries” on OpenResearch

About this paper

Belén Martín-Urcelay, Yoonsang Lee, Matthieu R. Bloch, Christopher J. Rozell

Researchers at the Georgia Institute of Technology introduced an information-efficient human-in-the-loop learning framework that leverages item ranking and exemplar selection queries instead of traditional labels. This approach demonstrated up to an 85% reduction in human interactions needed to train classifiers, alongside optimized learning speed when accounting for annotation costs.

Key insights

• Richer queries, particularly ranking and exemplar selection, inherently provide significantly more information per human interaction than binary labels, accelerating the learning process.
• An approximately linear relationship exists between a human's perceived item score and its inner product with the ground truth classifier in common off-the-shelf embedding spaces, enabling effective modeling of richer responses.
• Optimizing for information rate (information gain per unit cost) using empirically derived human response times leads to different optimal query strategies than optimizing solely for sample complexity, yielding practical time savings in HITL systems.

Summary by alphaXiv.