Reproduce “Maximizing Signal in Human-Model Preference Alignment” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kelsey Kraus, Margaret Kroll

The authors present a framework for deciding whether disagreement among human judgments is noise or meaningful preference signal, and apply it to two classifiers for a workplace Ask Me Anything feature. They ask 80 participants to assess 50 questions where the classifiers differed. Their judgments led them to choose the more restrictive classifier for this use case, not establish broader safety or user preference.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.