Reproduce “Square$χ$PO: Differentially Private and Robust $χ^2$-Preference Optimization in Offline Direct Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingyu Zhou, Yulian Wu, Wenqian Weng, Francesco Orabona
The authors propose SquareχPO, an offline preference-optimization method that keeps χPO’s policy parametrization but fits preference probabilities with squared error instead of log-loss. They analyze local privacy and Huber-corrupted labels, deriving comparator-policy guarantees under stated realizability and bounded-reward assumptions. In one synthetic financial task, it reports higher win rates than χPO against an SFT baseline; this is proof-of-concept evidence, not broad validation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.