Reproduce “CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han and 3 more
CAIL2018 introduces the first large-scale, publicly available dataset of over 2.6 million Chinese criminal cases, annotated with applicable law articles, charges, and prison terms. Benchmark experiments reveal that while existing models achieve high accuracy on charges and articles (up to 97.6%), they exhibit significantly lower macro-precision and macro-recall, particularly struggling with class imbalance and prison term prediction (accuracy 74.6-78.2%).
Key insights
- CAIL2018 provides an unprecedented scale of authentic, richly annotated Chinese legal data, significantly advancing the resources available for LJP research.
- The dataset reveals a pronounced class imbalance, with a few categories dominating the data distribution, which poses a substantial challenge for model generalization.
- LJP remains a difficult task, as baseline models achieve high overall accuracy but significantly lower macro-precision and recall, particularly struggling with minority classes and prison term prediction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.