Reproduce “AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models” on OpenResearch

About this paper

Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, Young Min Kim

AnyCamVLA introduces a zero-shot camera adaptation framework for Vision-Language-Action (VLA) models, utilizing a feed-forward novel view synthesis module to transform real-time camera observations into the VLA's trained configuration. This enables robust VLA performance despite varying camera viewpoints, achieving up to 94.5% success rates on LIBERO benchmarks and maintaining performance in real-world tasks with diverse camera models.

Key insights

• Zero-shot real-time visual input adaptation via photorealistic novel view synthesis effectively bridges the domain gap between varying test-time camera viewpoints and a VLA's fixed training configuration.
• Leveraging fast feed-forward view synthesis models avoids the computational overhead and per-scene optimization limitations of traditional 3D reconstruction methods (e.g., NeRF), making it suitable for online robotic control loops.
• Decoupling viewpoint adaptation from VLA policy learning prevents catastrophic forgetting, preserving the VLA's original learned capabilities while enabling robust generalization to novel camera setups.

Summary by alphaXiv.