Reproduce “Evaluating Human-LLM Alignment on Visualization Preference” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tanmay Bakshi, Chenglong Wang, Simra Shahid

The authors test whether language-model judges match human preferences across 24 pairs of valid charts, using choices from 10 volunteers. They compare choice distributions and how concentrated model judgments are. Nine participants chose a stacked area chart over a heatmap, while four tested models usually chose the heatmap. Overall, models diverged from the panel; this small pilot supports caution, not model rankings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.