Reproduce “ICLayout-Bench: Benchmarking LLM Agents for Custom Circuit Layout Design” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhangmai Li, Tianrui Ma, Zheng Weng, Mo Zou, Zidong Du, Yifan Hao, Yongwei Zhao, Wei Li and 3 more

The authors introduce ICLayout-Bench, a benchmark where agents iteratively build and revise chip layouts with electronic-design-automation feedback, then an independent evaluator checks frozen submissions for validity, electrical quality, and area. Its experiments cover 21 public and 19 private tasks from a 165-task inventory. Mean scores did not consistently rise with reasoning effort, across these selected tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.