Reproduce “Steerable Multi-Objective Group-Relative Preference Optimization with Jointly Trained Low-Rank Adapters” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Constantinos Karouzos, Nikolaos Aletras

LambdaMix uses online group-relative reinforcement learning to train one low-rank adapter per objective. One preference controls adapter mixing for generation and reward weighting for updates, letting one model change trade-offs at inference without further optimization. Across four models, it reports higher hypervolume than conditioned and policy-merging baselines on two tasks; Qwen3.5-9B has evaluator-based hypervolume 0.8735 versus 0.7760 for prompt conditioning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.