Reproduce “Large Language Models Meet Legal Artificial Intelligence: A Survey” on OpenResearch

About this paper

Zhitian Hou, Zihan Ye, Nanli Zeng, Tianyong Hao, Kun Zeng

Zhitian Hou and colleagues survey the emerging intersection of large language models and Legal AI, organizing 16 legal LLM series, 47 LLM-based frameworks, 15 benchmarks, and 29 datasets across major legal tasks. The review finds that retrieval augmentation, structured reasoning, and domain adaptation are central trends, while hallucination, limited legal data, weak multilingual and multimodal coverage, and inconsistent evaluation continue to constrain deployment.

Key insights

• Legal LLM development generally combines domain-specific legal data with general corpora to preserve broad language ability, while fine-tuning increasingly depends on synthetically generated instructions because expert annotations are limited.
• Smaller domain-adapted models can perform competitively with larger systems on some legal benchmarks, suggesting that training data quality, task coverage, and grounding mechanisms may be as important as parameter count.
• Retrieval and structured reasoning improve access to statutes and precedents and make outputs easier to inspect, but they do not by themselves resolve factual errors, jurisdictional variation, interpretability limitations, or high-stakes reliability concerns.

Summary by alphaXiv.