Reproduce “Learning Loss for Active Learning” on OpenResearch

About this paper

Donggeun Yoo, In So Kweon

The authors present Learning Loss for Active Learning, which selects unlabeled examples for annotation by predicting the supervised loss a task model would incur. A small module ranks examples, and people label the top-scoring ones. Tests cover classification, object detection, and pose estimation. On CIFAR-10, accuracy is 0.9101 versus 0.9059 for entropy sampling at matched label counts; annotation time and computation were not measured.

Key insights

• Directly learning to predict the loss of unlabeled data provides a universal, task-agnostic criterion for active data selection, eliminating the need for task-specific uncertainty heuristics.
• Training the loss prediction module with a pairwise ranking loss effectively captures the relative informativeness of data points, overcoming challenges associated with varying loss scales during model training.
• Integrating a small, parametrically learned module (LPM) into the main network allows for efficient active learning without significant computational overhead, making it practical for large deep models.

Summary by alphaXiv.