Multiagent Based Reinforcement Learning (MA-RL): An Automated Designer for Complex Analog Circuits

Despite the effort of analog circuit design automation, currently complex analog circuit design still requires extensive manual iterations, making it labor intensive and time-consuming. Recently, reinforcement learning (RL) algorithms have been demonstrated successfully for the analog circuit design optimization. However, a robust and highly efficient RL method to design analog circuits with complex design space has not been fully explored yet. In this work, inspired by multiagent planning theory as well as human expert design practice, we propose a multiagent based RL (MA-RL) framework to tackle this issue. Particularly, we (i) partition the complex analog circuits into several sub-blocks based on topology information and effectively reduce the complexity of design search space; (ii) leverage MA-RL for the circuit optimization, where each agent corresponds to a single sub-block, and the interactions between agents delicately mimic the best design tradeoffs between circuit sub-blocks by human experts; (iii) introduce and compare three different multiagent RL algorithms and corresponding frameworks to demonstrate the effectiveness of the MA-RL method. (iv) employing twin-delayed techniques and proximal policy to further boost training stability and accomplish higher performances. (v) The impacts of different reward function definitions as well as different state settings of MA-RL agents are investigated to further improve the robustness of this framework. (vi) Experiments on three different complex analog circuit topologies (GBA, DLL and SAR ADC) and knowledge transfers between two technology nodes are demonstrated. It's shown that MA-RL framework can achieve the best FoM for complex analog circuits' design. This work shines the light for future large scale analog circuit system design automation.