Reproduce “Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graph” on OpenResearch

About this paper

Xiaochen Kev Gao, Feng Yao, Kewen Zhao, Beilei He, Animesh Kumar, Vish Krishnan, Jingbo Shang

Researchers at UCSD developed the Fine-grained cLAim depeNdency (FLAN) Graph, a domain-specific graph-based method for predicting patent claim approval. This approach, which explicitly models dependencies within and between claims, achieved an AUC of 66.04% and a Macro-F1 of 58.22%, substantially outperforming all tested Large Language Models (LLMs) and the previous BERT-based state-of-the-art.

Key insights

• Scaling up large language models (LLMs) or using advanced prompt engineering does not inherently lead to improved performance on highly domain-specific, structure-intensive tasks like patent approval prediction.
• Explicitly modeling fine-grained dependencies, both within individual claims and between dependent claims, is crucial for capturing the nuances of patent law and novelty assessment.
• Domain-specific structural knowledge, meticulously encoded, can be a more powerful predictor than relying solely on the general-purpose capabilities of massive LLMs.

Summary by alphaXiv.