Uncertainty-Aware Congestion Reward Shaping for Multi-Objective IC Placement with Deep Reinforcement Learning

Reinforcement-learning (RL) placers increasingly use learned congestion predictors to densify their reward through potential-based shaping. Existing congestion-aware shaping evaluates an overflow penalty at a point prediction of the final routing demand, although this demand is uncertain while objects remain unplaced and the predictor is unreliable on unfamiliar states. We show that the plug-in potential is optimistically biased by Jensen’s inequality, most strongly early in an episode, and propose UA-CARS: a deep ensemble of heteroscedastic grid-graph GCNs predicts the final g-cell utilisation, the expected overflow penalty is evaluated in closed form, and the potential is shrunk toward a reference potential with a precision weight derived from ensemble disagreement, which preserves policy invariance and adds no tuning parameter. On four OpenROAD designs built from public gate-level netlists, the uncertainty-aware potentials reduce the error of the congestion-to-go estimate by 32–36% and lower their trust automatically where the predictor generalizes poorly. A controlled PPO study with 56 training runs shows, however, that potential accuracy alone does not yield better placements: with a neutral reference, UA-CARS performs on par with the plug-in baseline, and both are outperformed by the exact incremental congestion reward. Anchoring the shrinkage on that reward (UA-CARS-M) matches it and lowers the late-training cost of CARS by 5.6% (11/12 design–seed pairs, p=0.002). We trace these results to the accuracy of potential differences, which governs the benefit of shaping.