Reproduce “Alignment with Reflect-Reasoning: Learning How To Follow Principles” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Henry Bell, Caroline Zhang, Dhaval Potdar, Mohammed Mobasserul Haque, Brandon Fain

Reflect-Reasoning trains a model to check draft answers against natural-language principles, keep them or critique and revise them. Deployers can change the principles at use time without retraining. On held-out principles, Tulu-3-8B-SFT won 94.7% ± 1.9 against Base-ICL and 86.9% ± 2.8 against Reflect-ICL. These LLM-judge conformance comparisons are not general preferences; smaller human validation supported the direction.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.