Reproduce “AuthBench: A Large-Scale Multilingual Benchmark for Authorship Representation across Genres and Lengths” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
MaoXun Huang, Zhenxing Zhang, Claire Cardie
Cornell University researchers introduce AuthBench, a large-scale multilingual benchmark for evaluating authorship representation models across diverse genres and document lengths. The benchmark reveals significant performance variability for state-of-the-art models across languages, genres, and document lengths, with short documents posing a consistent challenge for both authorship attribution and verification tasks.
Key insights
- Authorship representation is far from a solved problem, with the best retrieval model achieving only 0.258 S@5 and best verification model achieving 0.076 EER, indicating substantial room for improvement.
- Performance varies dramatically across languages and genres, with no single model exhibiting universal superiority, suggesting that different model architectures may be sensitive to distinct authorial cues in diverse discourse structures.
- Document length strongly correlates with performance; shorter documents (1-10 tokens) represent a critical bottleneck for both attribution and verification tasks, requiring more robust sparse-evidence modeling.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.