Reproduce “InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yichen Peng, Jyun-Ting Song, Chen-Chieh Liao, Kris Kitani, Hideki Koike, Erwin Wu
A research collaboration from Institute of Science Tokyo and Carnegie Mellon University introduces InterPet4D, the first large-scale, multimodal 4D dataset capturing naturalistic human-dog interactions, alongside InterPetMoGen, a framework generating realistic pet motion from human gestures and audio. The framework achieved a 47.2% reduction in kinetic FID and higher alignment recall against baselines, with generated motions perceived as more natural and responsive in user evaluations.
Key insights
- Capturing synchronized multimodal data, including both human and animal 3D motion, along with audio and varied video perspectives, is essential for advancing human-pet interaction research.
- Utilizing discrete motion representations through a VQ-VAE (PetVAE) significantly contributes to the stability of motion generation and the realism of pet movements.
- A modality-aware attention mechanism is effective in integrating diverse human input signals (body, hand, audio) to generate contextually appropriate and responsive pet motions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.