Reproduce “Dataset Construction for Training LLM to Learn Analog Circuit Knowledge” on OpenResearch

About this paper

Zihao Chen, Ji Zhuang, Jinyi Shen, Xiaoyue Ke, Xinyi Yang, Mingjie Zhou, Zhuoyao Du, Xu Yan and 6 more

The authors turn textbook subsections into question-and-answer examples with reasoning and worked solutions, then adapt Qwen2.5-32B-Instruct using supervised fine-tuning with KL regularization. On a fixed text-only analog-circuit benchmark, it reaches 84.59% versus 68.92% for the starting model, a 15.67-point gain. The test contains 1,181 multiple-choice items and does not establish general circuit-design competence.

Key insights

• In these experiments, supervised fine-tuning on generated examples contributed more than continued pre-training: adding CPT before SFT improved accuracy by only 0.17 percentage points over SFT alone.
• KL regularization improved benchmark accuracy across all three reported seeds, while combining analog-circuit and general reasoning data performed better than either data type alone.
• Model initialization mattered: fine-tuning QwQ-32B reduced its benchmark score, whereas the tested Qwen2.5 instruct model benefited from domain adaptation.

Summary by alphaXiv.