Reproduce “Synthetic Human Action Video Data Generation with Pose Transfer” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Vaclav Knapp, Matyas Bohacek

A systematic method generates synthetic human action videos by combining pose transfer with 3D Gaussian avatar models, addressing the scarcity and bias in real-world training data. The approach improved action recognition model accuracy by over 30 percentage points on the Toyota Smarthome dataset and over 34 percentage points on the NTU RGB+D dataset when synthetic data was added to training.

Key insights

  • The method successfully integrates 3D human modeling with generative AI techniques, achieving both precise pose control and photorealistic rendering for synthetic video generation.
  • The introduction of the RANDOM People dataset, comprising 100 diverse human identity videos, provides a foundation for generating avatars with varied characteristics, directly addressing demographic bias in datasets.
  • Synthetic data generated by the approach significantly improves model performance in data-limited scenarios, including one-shot and few-shot learning settings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.