Reproduce “Continuous Control with Action Quantization from Demonstrations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Robert Dadashi, Léonard Hussenot, Damien Vincent, Sertan Girgin, Anton Raichuk, Matthieu Geist, Olivier Pietquin
Action Quantization from Demonstrations (AQuaDem) introduces a framework that learns a state-dependent action vocabulary from human demonstrations, transforming continuous control problems into discrete ones. This enables the application of robust discrete-action deep reinforcement learning algorithms to continuous tasks, yielding improved performance and sample efficiency across diverse robotic manipulation benchmarks.
Key insights
- Learning a state-dependent action vocabulary directly from demonstrations is more effective and scalable than fixed or uniform discretizations for continuous control problems.
- A soft-minimum loss function enables the action quantization network to explicitly model and capture the multimodality inherent in expert demonstrations, providing a richer action space for the RL agent.
- Transforming continuous control into a discrete problem via a learned, relevant action set allows the exploitation of the computational advantages and stability of discrete-action deep reinforcement learning algorithms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.