Reproduce “ReFit: Recurrent Fitting Network for 3D Human Recovery” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yufu Wang, Kostas Daniilidis

ReFit introduces a recurrent neural network architecture that mimics iterative optimization to precisely recover 3D human pose and shape from single images. The model achieved an MPJPE of 65.8 mm and PA-MPJPE of 41.0 mm on the 3DPW dataset, outperforming prior methods through spatially-aligned feedback and disentangled parameter updates.

Key insights

  • Embedding the iterative feedback-update structure of optimization into a neural network architecture ("learning-to-optimize") accelerates training convergence and improves accuracy.
  • Spatially-aligned feedback, obtained by reprojecting model keypoints to query localized features on image feature maps, provides direct and effective error signals for iterative refinement.
  • Disentangling parameter updates through parallel recurrent units for specific parameter groups allows for more precise and efficient learning of update rules, mimicking localized gradient-based adjustments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.