Reproduce “Apple: Toward General Active Perception via Reinforcement Learning” on OpenResearch

About this paper

Tim Schneider, Cristiana de Farias, Roberto Calandra, Liming Chen, Jan Peters

The APPLE framework enables autonomous agents to learn general active perception policies by integrating reinforcement learning with supervised learning, optimizing for information-gathering actions and prediction accuracy across various tactile tasks. This approach achieves high performance in classification, regression, and pose estimation, demonstrating improved sample efficiency and robustness over previous methods.

Key insights

• Jointly training an information-gathering policy and a prediction module using a unified objective, which combines RL rewards with a differentiable perception loss, is effective for learning task-agnostic active perception.
• Off-policy actor-critic methods, particularly adaptations like APPLE-SAC and APPLE-CrossQ, offer superior sample efficiency and robustness for active perception compared to on-policy baselines.
• The use of a transformer architecture for sequential observation processing proves more effective and stable than LSTM-based models for learning complex active exploration behaviors.

Summary by alphaXiv.