Reproduce “Grasp-and-Lift: Executable 3D Hand-Object Interaction Reconstruction via Physics-in-the-Loop Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Byeonggyeol Choi, Woojin Oh, Jongwoo Lim
Researchers at Seoul National University developed a physics-in-the-loop optimization framework, "Grasp-and-Lift," to transform visually aligned human-object interaction data into physically consistent, executable trajectories within a high-fidelity simulator. This method achieved a 71.67% success rate and reduced object translation error to 0.67 cm on the DexYCB dataset, while explicitly recovering 6-DoF contact wrenches.
Key insights
- Optimizing only the hand's control inputs, allowing the object's motion to emerge naturally from the physics simulation, inherently ensures dynamic consistency of the interaction.
- A low-dimensional, data-driven parameterization of the hand combined with a robust black-box optimizer effectively handles the non-differentiable and complex nature of contact physics.
- Including object velocity matching and using the visually-aligned hand 3D joint positions as a strong prior are crucial for achieving both dynamic consistency and visual faithfulness in reconstructed trajectories.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.