Reproduce “Insights from Rights and Wrongs: A Large Language Model for Solving Assertion Failures in RTL Design” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jie Zhou, Youshu Ji, Ning Wang, Yuchen Hu, Xinyao Jiao, Bingkun Yao, Xinwei Fang, Shuai Zhao and 2 more
AssertSolver is an open-source large language model specifically fine-tuned to automate the resolution of assertion failures in Register Transfer Level (RTL) designs. It achieves an 88.54% pass@1 and 90.00% pass@5 on a new SVA-Eval benchmark, surpassing leading commercial models like OpenAI's o1-preview by 11.97% in pass@1, through a methodology that includes novel data augmentation and learning from erroneous responses.
Key insights
- A novel data augmentation method, synergistically combining LLMs with traditional EDA tools, effectively addresses data scarcity in hardware verification, yielding rich training datasets.
- Learning from erroneous responses to challenging cases via Direct Preference Optimization (DPO) substantially enhances an LLM's precision and reliability for critical engineering tasks.
- Domain-specific pretraining and fine-tuning enable LLMs to achieve superior performance in complex hardware design tasks, significantly outperforming general-purpose state-of-the-art models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.