Reproduce “PALM: Enhanced Generalizability for Local Visuomotor Policies via Perception Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruiyu Wang, Zheyu Zhuang, Danica Kragic, Florian T. Pokorny
KTH Royal Institute of Technology researchers developed PALM (Perception Alignment for Local Manipulation), a pre-processing framework that enhances visuomotor policy generalizability by aligning visual and proprioceptive inputs. The system reduced average out-of-distribution (OOD) performance degradation to 8% in simulation and 24% in real-world tasks, demonstrating robustness across workspace shifts, camera viewpoint changes, and robot embodiment variations.
Key insights
- Local action distributions exhibit an invariance that can be leveraged to generalize visuomotor policies, despite significant global changes in observation (e.g., robot appearance or workspace position).
- Input-level alignment for both visual and proprioceptive modalities can effectively enhance generalization across diverse OOD shifts without requiring fundamental model modifications or additional data modalities.
- A coarse-to-fine policy decomposition, combined with canonicalizing local observations, allows policies to maintain performance even when deployed in environments significantly different from training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.