Reproduce “LLM4SGG: Large Language Models for Weakly Supervised Scene Graph Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kibum Kim, Kanghoon Yoon, Jaehyeong Jeon, Yeonjun In, Jinyoung Moon, Donghyun Kim, Chanyoung Park
LLM4SGG leverages Large Language Models to enhance weakly-supervised scene graph generation by producing higher-quality and denser pseudo-labels from image captions. The method significantly improves the prediction of fine-grained relationships and boosts data efficiency for existing WSSGG models.
Key insights
- Large Language Models (LLMs) can effectively replace conventional rule-based parsing and static knowledge bases in weakly-supervised scene graph generation to produce richer and denser pseudo-labels.
- Paraphrasing input captions before triplet extraction, combined with direct extraction, significantly increases the density and diversity of unlocalized triplets.
- LLM-based semantic alignment enables more flexible and accurate mapping of extracted linguistic terms to target entity and predicate classes, retaining more valuable information than traditional knowledge base lookups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.