Reproduce “Learning to build covering structures with continuous adjustments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gabriel Vallat, Maryam Kamgarpour, Stefana Parascho
Researchers developed an adaptive reinforcement learning framework, Hybrid Soft Actor-Critic (HSAC), that enables robots to autonomously build spanning structures without predefined plans, continuously adjusting block placements based on real-time feedback. The system successfully constructed arches in both simulation and on physical hardware, demonstrating improved robustness and adaptability compared to traditional planning methods.
Key insights
- Using separate dynamic entropy coefficients for discrete and continuous policy components in HSAC is critical for effective exploration in hybrid action spaces, preventing the discrete policy from becoming prematurely deterministic.
- A Q-function architecture that employs unidirectional graph edges structurally enforces independence between Q-values of different discrete actions, addressing over-parametrization in hybrid action spaces.
- Representing the construction state as a graph with geometric invariants (translation, permutation, vertical rotation) significantly improves the generalizability and learning efficiency of the reinforcement learning agent.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.