Reproduce “Synthetic Object Compositions for Scalable and Accurate Learning in Detection, Segmentation, and Grounding” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Weikai Huang, Jieyu Zhang, Taoyang Jia, Chenhao Zheng, Ziqi Gao, Jae Sung Park, Winson Han, Ranjay Krishna

Researchers from the University of Washington and Allen Institute for AI developed Synthetic Object Compositions (SOC), an object-centric pipeline that generates high-quality synthetic images with pixel-perfect annotations for visual grouping tasks. This method improved LVIS open-vocabulary detection AP by 9.7 points and significantly boosted visual grounding performance, outperforming prior synthetic data approaches and complementing real datasets.

Key insights

  • An object-centric composition approach, combined with advanced generative harmonization and 3D geometric layouts, can yield synthetic data that is more accurate, diverse, and photorealistic than prior synthetic methods.
  • Decorrelating object scale, position, and appearance from semantic categories through randomized 3D layouts and camera augmentations helps models learn robust, non-spurious features.
  • Precisely controlled synthetic data can effectively target and improve performance on specific diagnostic tasks (like intra-class referring) and mitigate long-tail distribution problems in real datasets.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.