Reproduce “Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Saurabh Daptardar, Paul Schrater, Xaq Pitkow
Inverse Rational Control (IRC) infers task models that explain an agent’s observed actions rather than imitating them. It learns policies across candidate models and selects parameters whose policy best fits behavior. In simulations, nearly all one-dimensional true parameters fell within reported 95% confidence intervals, while two-dimensional estimates recovered parameters and beliefs up to intrinsic uncertainty; the results concern simulated agents.
Key insights
- An agent's behavior, even if suboptimal from an external viewpoint, can be interpreted as rational by inferring its specific internal model parameters, including subjective rewards and assumed world dynamics.
- Optimal policies can be learned and represented as explicit functions of internal model parameters, allowing for efficient inference over a continuous manifold of possible agent models.
- Separating the policy optimization over parameter space from the maximum likelihood estimation of parameters offers a computationally tractable and robust method for inverse rational control in complex, partially observable environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.