Reproduce “Observer-Actor: Active Vision Imitation Learning with Sparse-View Gaussian Splatting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yilong Wang, Cheng Qian, Ruomeng Fan, Edward Johns
Researchers at Imperial College London developed the Observer–Actor (ObAct) framework, which enhances robotic manipulation by using sparse-view Gaussian Splatting to dynamically determine and utilize optimal camera viewpoints. This approach significantly improves imitation learning success rates for both unoccluded and occluded tasks, achieving up to 233% increase over static-camera baselines.
Key insights
- Sparse-view 3D Gaussian Splatting can be effectively leveraged in real-time to generate a 3D scene representation, enabling computationally efficient optimal viewpoint selection through differentiable rendering.
- Dynamically assigning observer and actor roles, coupled with a camera-frame action representation, allows for ambidextrous inference and significantly improves policy generalization and robustness against viewpoint discrepancies.
- Providing consistently optimal, occlusion-free visual observations at test time dramatically enhances the performance and data efficiency of both trajectory transfer and behavior cloning methods in imitation learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.