Reproduce “AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Daniel Braun, Florian Matthes
Researchers from the University of Twente and TU Munich developed AGB-DE, an openly available corpus of 3,764 German consumer contract clauses annotated by legal experts for topic and legal validity. Their work establishes a baseline for automated legal assessment, showing that fine-tuned BERT models achieve an F1-score of 0.54, outperforming zero-shot commercial LLMs like GPT-3.5.
Key insights
- Automated legal assessment of contract validity is a challenging task, with models often struggling with nuanced interpretation beyond basic legal knowledge.
- Fine-tuned language models (e.g., BERT) demonstrate better precision in identifying void clauses compared to zero-shot commercial LLMs, which tend to generate many false positives.
- Dataset imbalance, where potentially void clauses are rare, significantly impacts model performance; undersampling proved an effective strategy to improve F1-scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.