Reproduce “UniDexTok: A Unified Dexterous Hand Tokenizer from Real Data” on OpenResearch

About this paper

Dong Fang, Youjun Wu, Yuanxin Zhong, Rui Zhang, Yunlong Wang, Xiaosong Jia, Yu-Gang Jiang

UniDexTok introduces a unified dexterous hand tokenizer and a unified hand model (UDHM) that enable learning a shared, retargeting-free state representation directly from heterogeneous real-world data, effectively addressing data fragmentation across diverse dexterous hand hardware. The system achieves sub-millimeter accuracy in hand state reconstruction and demonstrates robust generalization capabilities to unseen hand embodiments with efficient few-shot adaptation.

Key insights

• A semantically aligned unified hand model (UDHM) effectively standardizes heterogeneous real-world dexterous hand data, allowing direct learning of state representations without introducing retargeting errors.
• A conditional state tokenizer architecture with a factorized vector quantization codebook enables learning a robust, unified discrete token space that supports strong zero-shot transfer and efficient few-shot adaptation to new hand embodiments.
• Including UDHM-processed human hand data significantly expands pose coverage and improves the learning capability of the tokenizer for robot hands, contributing to better overall performance.

Summary by alphaXiv.