Reproduce “Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, Markus Strohmaier
The authors compare eight methods for turning open-ended language model responses into valid options in simulated political surveys, using ten open-weight models, four datasets, and 32 million responses. Restricted Generation, especially Restricted Choice, generally improves alignment with individual human answers. For GLES 2017 vote choice, Restricted Choice’s adjusted OLS coefficient is +0.242 macro-average F1 relative to First-Token Probabilities; the findings do not establish broader population validity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.