Reproduce “Echoes of Socratic Doubt: Embracing Uncertainty in Calibrated Evidential Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Alex Christopher Stutts, Danilo Erricolo, Theja Tulabandhula, Amit Ranjan Trivedi

Researchers at the University of Illinois Chicago developed Calibrated Evidential Quantile Regression in Deep Q Networks (CEQR-DQN), an algorithm that integrates deep evidential learning and conformal inference for robust uncertainty quantification in distributional reinforcement learning. This method provides explicit and separable estimates of aleatoric and epistemic uncertainty, achieving faster learning and higher scores across multiple MinAtar games compared to previous uncertainty-aware DRL approaches.

Key insights

  • Statistically robust uncertainty quantification, including calibration and modulated confidence upon OOD observations, significantly improves an agent's exploration strategy.
  • Deep evidential learning provides global uncertainty awareness, which is more effective than local estimates derived from methods relying on variance between auxiliary networks.
  • Explicitly disentangling and quantifying aleatoric and epistemic uncertainties allows for more targeted and efficient exploration, leading to faster learning and improved performance in stochastic environments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.