Reinforcement Learning within Tree Search for Fast Macro Placement
Zijie Geng, Jie Wang, Ziyan Liu, Siyuan Xu, Zhentao Tang, Mingxuan Yuan, Jianye HAO, Yongdong Zhang
Abstract
Macro placement is a crucial step in modern chip design, and reinforcement learning (RL) has recently emerged as a promising technique for improving the placement quality. However, existing RL-based techniques are hindered by their low sample efficiency, requiring numerous online rollouts or substantial offline expert data to achieve bootstrap, which are often impractical in industrial scenarios. To address this challenge, we propose a novel sample-efficient framework, namely **EfficientPlace**, for fast macro placement. EfficientPlace integrates a global tree search algorithm to strategically direct the optimization process, as well as a RL agent for local policy learning to advance the tree search. Experiments on commonly used benchmarks demonstrate that EfficientPlace achieves remarkable placement quality within a short timeframe, outperforming recent state-of-the-art approaches.
BibTeX
@inproceedings{
geng2024reinforcement,
title={Reinforcement Learning within Tree Search for Fast Macro Placement},
author={Zijie Geng and Jie Wang and Ziyan Liu and Siyuan Xu and Zhentao Tang and Mingxuan Yuan and Jianye HAO and Yongdong Zhang and Feng Wu},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=AJGwSx0RUV}
}