Reproduce “Cross-Genre Authorship Attribution via LLM-Based Retrieve-and-Rerank” on OpenResearch

About this paper

Shantanu Agarwal, Joel Barry, Steven Fincke, Scott Miller

Researchers from USC's Information Sciences Institute developed SADIRI-V2, an LLM-based retrieve-and-rerank framework, to address cross-genre authorship attribution. This system achieved absolute gains of 22.3 Success@8 points on the HIATUS HRS1 benchmark and 34.4 Success@8 points on HRS2 over the previous state-of-the-art by carefully fine-tuning LLMs and adapting negative sampling for the reranker.

Key insights

• Fine-tuning LLMs for authorship attribution tasks yields substantial performance improvements, with larger LLM backbones demonstrating better results, surpassing previous RoBERTa-based models.
• Standard Information Retrieval (IR) reranker training strategies, particularly those sampling negatives only near the query document, are counterproductive for cross-genre authorship attribution as they promote an 'anti-topic' heuristic.
• Optimal reranker performance in cross-genre settings requires a balanced negative sampling strategy that includes negatives semantically close to the query, close to the positive document, and random documents.

Summary by alphaXiv.