Reproduce “RL Post-Training in Diffusion Transformers Is Activation Steering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tuna Han Salih Meral, Kaan Oktay, Alec Helbling, Atakan Kara, Duen Horng Chau, Pinar Yanardag

The authors fit fixed steering vectors to imitate an RL-tuned generator’s predictions while keeping its weights frozen. Added to internal block outputs, the vectors do not vary by prompt. On SD3.5-M tuned with Flow-GRPO on PickScore, they recover 90.8% of the tuned model’s reward gain. This measures reward, not human preference, and does not show every capability was available in the base model.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.