Reproduce “Flexible Multiple-Objective Reinforcement Learning for Chip Placement” on OpenResearch

About this paper

Fu-Chieh Chang, Yu-Wei Tseng, Ya-Wen Yu, Ssu-Rui Lee, Alexandru Cioba, I-Lun Tseng, Da-shan Shiu, Jhih-Wei Hsu and 6 more

A flexible multiple-objective reinforcement learning framework, MOPPO, enables generating diverse chip placement solutions by allowing variable weighting of objectives like wirelength, congestion, and anchor distance at inference time. This framework, combined with an efficient clustered netlist reward computation method, provides engineers with rapid design exploration capabilities.

Key insights

• Introducing a flexible preference parameter directly into the DRL agent's architecture allows a single model to generate a Pareto frontier of solutions with varying trade-offs for multiple objectives at inference time.
• Efficient reward computation via clustered netlists, correlated to full netlist results, is crucial for making multi-objective DRL training tractable for complex industrial problems like chip placement.
• MOPPO can effectively learn a continuum of policies across a multi-objective preference space, demonstrating that on-policy DRL can be adapted to provide flexible optimization in complex engineering domains.

Summary by alphaXiv.