Reproduce “Explicating Tacit Regulatory Knowledge from LLMs to Auto-Formalize Requirements for Compliance Test Case Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiyi Xue, Xiaohong Chen, Min Zhang
A framework called RAFT automates the formalization of regulatory requirements and generation of compliance test cases by explicating tacit regulatory knowledge from Large Language Models (LLMs). It achieves expert-level performance with a 2.3x speedup over human experts and significantly reduces costs across financial, automotive, and power domains.
Key insights
- Tacit regulatory knowledge embedded within LLMs can be systematically explicated and structured into verifiable artifacts to effectively constrain and guide LLM behavior in formalizing requirements.
- An Adaptive Purification-Aggregation strategy, combining outputs from multiple LLMs based on consensus or aggregation, yields more robust and complete regulatory knowledge artifacts compared to individual LLM outputs.
- Injecting explicit domain meta-models, formal representation rules, and testability constraints into LLM prompts significantly improves the accuracy, interpretability, and testability of automatically generated formal requirements and test cases.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.