Reproduce “PACT: Pairwise-Anchored Calibrated Tuning for Single-Token Typed Decisions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yida Lin
PACT trains typed-decision models using contrastive pairs, evidence-removal certificates, answer-code permutations, and ordered rubric scores. It adds pairwise and consistency constraints to cross-entropy training while preserving the single-pass answer interface. On a frozen synthetic, model-labelled holdout of 324 items, PACT reduces answer-code sensitivity and Score error relative to Nimble, but does not improve accuracy in this evaluation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.