Evaluating Deep Reinforcement Learning for Macromodel Synthesis

This paper presents a novel approach to apply reinforcement learning (RL) to the design of equivalent circuits for modeling passive electronic components. These components are usually characterized by S-parameter measurements in the frequency domain. However, time-domain simulations and other applications require equivalent circuit models. Baseline methods for generating these models have drawbacks, such as requiring prior physical knowledge of the components, generating large models with unnecessary lumped elements, or violating passivity rules. We present an RL-based approach to this problem by developing an RL environment in which the design of an equivalent circuit can be considered as a Markov Decision Process (MDP). In this way, we enable an RL agent to interact with the MDP in a stepwise manner with the goal of generating a best-fitting equivalent circuit for a component. A spice-assisted approach within the RL environment ensures the passivity of the equivalent circuit. With the presented experiment, we provide a first proof of concept for this approach and discuss future possibilities and challenges of this method.