Reproduce “Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Samuel Paech, Allen Roush, Judah Goldfeder, Ravid Shwartz-Ziv
The Antislop framework introduces an end-to-end system to identify and eliminate repetitive "slop" patterns in Large Language Model outputs. It provides a diagnostic tool to quantify model-specific slop, an inference-time sampler for real-time suppression, and a novel fine-tuning method, Final Token Preference Optimization (FTPO), which reduces slop by approximately 90% while maintaining writing quality and general capabilities with less than 1% degradation compared to baseline models.
Key insights
- "Slop" is a quantifiable and model-specific phenomenon characterized by statistically over-represented patterns in LLM outputs, distinct from simple local repetition.
- Targeted logit-space optimization, such as FTPO's approach with margin-based preference loss and two-part regularization, can achieve precise behavioral alignment (slop suppression) without causing the broader degradation in quality or diversity seen in other preference optimization methods.
- An automated, end-to-end pipeline can effectively identify model-specific slop, generate tailored preference training data, and fine-tune models to eliminate these patterns, offering a comprehensive solution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.