Reproduce “Pair then Relation: Pair-Net for Panoptic Scene Graph Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jinghao Wang, Zhengyu Wen, Xiangtai Li, Zujin Guo, Jingkang Yang, Ziwei Liu

Researchers at Nanyang Technological University's S-Lab developed Pair-Net, a Panoptic Scene Graph Generation framework that addresses the critical bottleneck of inter-object pair recall through a novel "Pair then Relation" strategy. This approach achieved new state-of-the-art performance, with over a 10.2% absolute gain in mR@20 on the PSG benchmark and a 24.1% absolute gain in Pair Recall@20.

Key insights

  • The primary bottleneck in Panoptic Scene Graph Generation is not individual object segmentation but the accurate recall of inter-object pairs, which directly correlates with overall triplet recall.
  • Explicitly learning to propose and filter subject-object pairs, rather than inferring dense all-to-all interactions, is crucial for improving PSG performance due to the inherent sparsity of relationships.
  • Differentiating between subject and object roles via distinct projectors and employing specialized loss functions like positive weight adjusted BCE for pair learning and Seesaw Loss for relation classification effectively handles sparsity and long-tailed distributions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.