Reproduce “(Security) Assertions by Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rahul Kande, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Shailja Thakur, Ramesh Karri, Jeyavijayan Rajendran
Researchers investigated the feasibility of using large language models (LLMs) to automatically generate hardware security assertions from natural language prompts. Their evaluation framework demonstrated that LLMs can produce functionally correct SystemVerilog assertions, achieving up to 93.55% accuracy for optimal prompt configurations, suggesting a method to reduce manual effort in hardware verification.
Key insights
- LLMs are fundamentally capable of generating hardware security assertions, indicating a potential 'shift-left' in vulnerability detection by automating an expert-intensive task.
- Prompt engineering significantly influences LLM performance, with detailed natural language comments and relevant SystemVerilog assertion examples being crucial for achieving high accuracy.
- The LLM's ability to infer from rich natural language context can sometimes outweigh the necessity of providing the full design source code for assertion generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.