Reproduce “Learn2Augment: Learning to Composite Videos for Data Augmentation in Action Recognition” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shreyank N Gowda, Marcus Rohrbach, Frank Keller, Laura Sevilla-Lara

Learn2Augment, developed by researchers at the University of Edinburgh and Meta AI, presents a reinforcement learning-based method to intelligently select and composite video pairs for data augmentation, enhancing action recognition model performance across various data availability settings. The framework substantially improves accuracy on datasets like UCF101 and HMDB51 while significantly reducing computational overhead compared to random augmentation strategies.

Key insights

  • Learning to select valuable video pairs for augmentation is more effective and computationally efficient than random sampling or heuristic approaches, reducing the candidate search space by over three orders of magnitude.
  • Combining semantic matching with a learned selector is crucial for generating realistic composite videos and making large-scale video augmentation feasible by pruning implausible combinations.
  • High-quality video compositing, involving precise segmentation and background inpainting, yields more visually plausible augmented data that significantly contributes to learning robust representations for action recognition.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.