Reproduce “DDS: Decoupled Dynamic Scene-Graph Generation Network” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

A S M Iftekhar, Raphael Ruschel, Satish Kumar, Suya You, B.S. Manjunath

The Decoupled Dynamic Scene-Graph Generation Network (DDS) from researchers at the University of California Santa Barbara and the Army Research Laboratory introduces a multi-branch, Transformer-based architecture that explicitly decouples object and relationship feature learning. This approach significantly enhances the model's ability to generalize to unseen relationship triplets, achieving 4-24 times higher recall for such triplets on the Action-Genome dataset and setting new state-of-the-art performance on Action-Genome, HICO-DET, and UnRel datasets.

Key insights

  • Explicitly decoupling the feature learning processes for objects and relationships is crucial for enabling robust generalization to unseen relationship triplets in dynamic scene graphs.
  • A multi-branch architecture with separate encoders, spatio-temporal decoders, and queries for objects and relations effectively reduces feature dependency.
  • Utilizing distinct temporal and spatial decoders within each branch allows for effective aggregation of both inter-frame temporal dependencies and intra-frame spatial context, critical for video-based SGG.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.