Reproduce “Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yun Qu, Yuhang Jiang, Boyuan Wang, Yixiu Mao, Cheems Wang, Chang Liu, Xiangyang Ji
Tsinghua University researchers introduce LaRe, an LLM-empowered framework that generates semantically interpretable "latent rewards" for enhanced credit assignment in episodic reinforcement learning. This approach improves sample efficiency and achieves higher episodic returns than state-of-the-art baselines across single-agent and multi-agent tasks, sometimes outperforming policies trained with dense ground truth rewards.
Key insights
- LLMs can act as effective, generalized encoders to extract reward-relevant features, bridging linguistic knowledge to symbolic representations and reducing state-action space redundancy.
- Multifaceted, semantically interpretable latent rewards provide richer, more accurate feedback for temporal credit assignment and facilitate better allocation of contributions among multiple agents.
- A self-verification process for LLM-generated code is crucial to mitigate randomness and hallucinations, ensuring the reliability and stability of LLM integration into symbolic RL environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.