Reproduce “Bellman Value Decomposition for Task Logic in Safe Optimal Control” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
William Sharpless, Oswin So, Dylan Hirsch, Sylvia Herbert, Chuchu Fan
Bellman Value Decomposition formally breaks down complex Temporal Logic specifications for robotic control into a hierarchy of simpler Value functions. The accompanying VDPPO algorithm trains optimal policies for these decomposed components, outperforming baselines in high-dimensional simulations and demonstrating emergent coordinated behaviors on real robotic hardware for tasks requiring sequential goals and safety.
Key insights
- While logical identities hold for robustness scores, the `max_alpha` operator prevents direct translation to Value function decomposition, necessitating a dedicated decomposition algebra.
- Complex Temporal Logic tasks can be systematically transformed into a hierarchy (or graph) of simpler, solvable Bellman Value functions using specific decomposition rules and novel Bellman operators.
- Parameter sharing across a Decomposed Value Graph (DVG) within a single neural network for both actor and critic significantly boosts training speed (up to N times faster) while preserving high performance for intricate TL objectives.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.