Reproduce “RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shiqiang Gong
The RS-Diffuser framework integrates risk-sensitive decision-making into offline diffusion planning by guiding trajectory generation with a distributional value critic. This approach enables inference-time control over risk preferences, leading to enhanced worst-case robustness and fewer safety violations in challenging safety-critical tasks.
Key insights
- Integrating the generative capabilities of diffusion models with a distributional value critic provides a principled way to achieve risk-sensitive decision-making in offline reinforcement learning.
- Direct Monte Carlo supervision for the distributional critic provides stable and accurate modeling of the return distribution, avoiding complexities of Bellman bootstrapping in offline settings.
- Gradient-based guidance from tail-aware risk measures during diffusion sampling allows for flexible, inference-time adaptation of behavior from risk-averse to risk-seeking within a single trained model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.