Reproduce “How to discretize continuous state-action spaces in Q-learning: A symbolic control approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sadek Belamfedel Alaoui, Adnane Saoud
Researchers from UM6P developed a symbolic control framework for Q-learning that discretizes continuous state-action spaces by over-approximating reachable sets, yielding two Q-tables that provably bound the true continuous Q-values. This approach enables arbitrary precision in learned policies and successfully stabilizes continuous systems, outperforming traditional uniform discretization methods.
Key insights
- The true Q-values of the continuous system are provably bounded by the learned maximal and minimal Q-values of the symbolic model, providing a quantifiable range of performance.
- The difference between maximal and minimal symbolic Q-values is directly proportional to the state and action discretization parameters and Lipschitz constants, establishing a quantitative link between abstraction granularity and approximation precision.
- The method allows for achieving arbitrary precision (ε-optimality) by appropriately selecting and refining the discretization parameters, offering explicit control over the accuracy-computational complexity trade-off in continuous control problems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.