Reproduce “Reinforcement Learning for Out-of-Distribution Reasoning in LLMs: An Empirical Study on Diagnosis-Related Group Coding” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hanyin Wang, Zhenbang Wu, Gururaj Kolar, Hariprasad Korsapati, Brian Bartlett, Bryan Hull, Jimeng Sun

Researchers from Mayo Clinic and the University of Illinois Urbana-Champaign developed DRG-SAPPHIRE, a reinforcement learning-based model for automated Diagnosis-Related Group (DRG) coding that achieves state-of-the-art accuracy on the MIMIC-IV dataset while providing physician-validated, explainable reasoning.

Key insights

  • RL performance on knowledge-intensive, out-of-distribution tasks such as DRG coding is fundamentally constrained by the domain-specific knowledge instilled in the base model through Supervised Fine-Tuning (SFT) prior to RL training.
  • Scaling Supervised Fine-Tuning (SFT) data to infuse domain knowledge is more computationally efficient and effective for improving model performance in OOD domains compared to solely scaling RL training.
  • For DRG coding, the 'Answer-First' reasoning pattern (predicting the DRG code before the explanation) demonstrated superior performance over explicit CoT-First or Differential Thinking approaches, suggesting implicit knowledge leverage.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.