Reproduce “Dataset Signatures in Human-LLM Interactions and User Modeling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joseph Suh, Serina Chang

The authors test whether conversation datasets leave detectable source signatures and whether those signatures persist in simulated users. A classifier using frozen Qwen2.5-7B representations identifies seven English datasets with 53.3% accuracy, versus 14.3% chance. Matching topic, intent, and structural summaries still leaves above-chance classification. User-model source changes assistant task success, but the study does not establish which source is more realistic.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.