Reproduce “Combining Counting Processes and Classification Improves a Stopping Rule for Technology Assisted Review” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Reem Bin-Hezam, Mark Stevenson
Researchers at the University of Sheffield and Princess Nourah bint Abdulrahman University developed a hybrid framework that integrates text classification with counting process models to improve stopping rules for Technology Assisted Review. Their method achieved significant reductions in manual review cost, such as a 79% reduction on the Legal collection at 0.9 recall, while maintaining high recall and reliability across six diverse datasets.
Key insights
- Integrating global relevance information from a text classifier into a counting process's rate function provides a more robust estimate of remaining relevant documents.
- Cost-sensitive learning during classifier training is critical to prevent premature stopping due to class imbalance in TAR datasets.
- The improved stopping rule leverages existing manual judgments for both components, avoiding any additional annotation burden.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.