Reproduce “PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Uyoung Jeong, Jonathan Freer, Seungryul Baek, Hyung Jin Chang, Kwang In Kim
PoseBH trains across datasets with different skeletons and incomplete labels. It maps keypoints into a shared embedding space and uses agreement between two predictions to generate training labels for unlabeled keypoints. On ViT-Base, it scores 75.0, 87.2, and 57.9 on AP-10K, APT-36K, and COCO-WholeBody, averaging 73.4 versus 69.1 for ViTPose++. These results do not establish zero-shot performance on unseen skeletons.
Key insights
- Nonparametric keypoint prototypes provide a flexible and scalable mechanism to integrate and interpret diverse keypoint definitions and anatomical differences across datasets.
- Cross-type self-supervision effectively mitigates the issue of sparse ground truth annotations in multi-dataset training by generating reliable supervision signals without requiring additional teacher models or input duplication.
- A unified framework can achieve superior generalization across challenging, diverse domains like animal and whole-body pose estimation while simultaneously maintaining or surpassing state-of-the-art results on established human pose benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.