Reproduce “Scene Graph Prediction with Limited Labels” on OpenResearch

About this paper

Vincent S. Chen, Paroma Varma, Ranjay Krishna, Michael Bernstein, Christopher Re, Li Fei-Fei

Researchers at Stanford University developed a semi-supervised framework that uses image-agnostic features and a generative model to automatically complete visual knowledge bases by generating probabilistic labels for missing relationships. This approach, requiring as few as 10 labeled examples per relationship, significantly improves scene graph prediction performance, outperforming traditional transfer learning and addressing the 'long tail' problem in visual datasets.

Key insights

• Image-agnostic features (categorical and spatial) are more robust and generalize better than raw deep features for learning visual relationships with limited labeled data.
• The data programming paradigm, leveraging a generative model to aggregate noisy heuristics, can effectively generate high-quality probabilistic labels for visual knowledge base completion.
• The 'complexity' of a visual relationship, defined by its spatial and categorical subtypes, strongly correlates with the performance advantage of the proposed semi-supervised method over transfer learning.

Summary by alphaXiv.