Reproduce “FlexPose: Pose Distribution Adaptation with Limited Guidance” on OpenResearch

About this paper

Zixiao Wang, Junwu Weng, Mengyuan Liu, Bei Yu

FlexPose, developed by researchers from The Chinese University of Hong Kong, ByteDance Inc., and Peking University, introduces a framework to adapt pre-trained generative models to new human pose distributions using only a few target examples. It represents poses as 'skeleton images' and employs targeted regularization on StyleGAN's affine layers, leading to superior pose distribution matching (lowest Fréchet Distance and MMD^2) and improved performance in downstream unlabelled pose dataset annotation compared to baselines, while avoiding privacy issues inherent in appearance-based methods.

Key insights

• Human poses across various datasets, despite differences, share a fundamental 'hinge-structure prior,' with variations primarily due to specific geometric transformations (e.g., joint rotations, bone lengths).
• Calibrating specific, geometrically meaningful layers (e.g., coarser affine transformation layers) of a pre-trained StyleGAN-based generator is highly effective for few-shot pose distribution adaptation.
• Extending 'mixup' regularization to the keypoint space allows for generating physically plausible intermediate poses, which effectively enriches limited target guidance and prevents overfitting in few-shot settings.

Summary by alphaXiv.