Reproduce “You Are What You Annotate: Towards Better Models through Annotator Representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Naihao Deng, Xinliang Frederick Zhang, Siyang Liu, Winston Wu, Lu Wang, Rada Mihalcea
The authors train classifiers to predict individual annotators’ labels rather than collapsing disagreements into a single answer. The method combines text with learned representations of annotator identity and that annotator’s tendencies in prior training labels. On TID-8, it improves significantly on six of eight datasets, with fewer than 1% additional parameters; this measures annotator-specific prediction, not fairness or social outcomes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.