Reproduce “Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunxiang Luo, Constantino Álvarez Casado, Xiaoting Wu, Nhi Nguyen, Le Nguyen, Manuel Lage Cañellas, Miguel Bordallo López
The authors train a shared classifier for contactless breathing monitoring that handles RGB video, depth video, and radar in any of seven non-empty combinations. It uses frozen V-JEPA2 features and marks absent streams rather than reconstructing them. All-seven-combination training raises activity Avg-7 macro-F1 to 75.23%, versus 73.31% with balanced supervision; this supports the training protocol, not a query-pooling effect.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.