Reproduce “Reinforcement Learning: An Overview” on OpenResearch

About this paper

Kevin Murphy

Murphy’s overview maps reinforcement learning for sequential decisions, emphasizing that choices depend on the application and assumptions about the environment. It explains value-based, policy-based, and model-based approaches, alongside partial observability. In one GuessTheNumber setup, fine-tuning raised reward from 0.8 to 1.0 and reduced generated action text from about 900 to about 500 tokens per turn, while turns stayed around three.

Key insights

• Reinforcement Learning provides a universal framework for sequential decision-making, applicable across various problem formulations from simple bandits to complex multi-agent environments.
• RL algorithms can be systematically organized into three core paradigms—value-based, policy-based, and model-based—each addressing distinct aspects of learning and control.
• The integration of RL with Large Language Models represents a critical frontier, with RL fine-tuning LLMs for alignment and reasoning, and LLMs serving as powerful components within RL systems.

Summary by alphaXiv.