Gray-Box System Identification using Continuous-Time Local Model Networks

Both first-principles-based and data-driven methods provide powerful tools for modeling physical systems. As such, there is a strong push for harnessing both approaches by combining them in the form of gray-box models. Automatic differentiation of ordinary differential equation (ODE) models and physical model parts enables flexible gray-box modeling in system identification tasks. Local model networks (LMNs) with their excellent interpretability and sparing use of model parameters are an ideal model class to provide data-driven elements in a gray-box model. Unfortunately, LMNs’ intricate training procedures currently do not allow for end-to-end training within larger model architectures. To remedy this, we propose a new LMN training algorithm and corresponding model called rectified linear unit model tree (ReLUMoT). ReLUMoT provides fully gradient-based training of LMNs, which permits us to train within gray-box ODE models. In essence, ReLUMoT pioneers the use of piecewise linear neural networks (PLNNs) as a training engine for LMNs. This is accomplished by distilling the locally linear structure of a PLNN into an LMN. We thoroughly review all theoretical concepts needed to train gray-box ODE models, ranging from state space modeling to initializing latent states during the training procedure. In particular, we give a critical review of sensitivity analysis methods, discussing their suitability for system identification tasks. We provide extensive experimental validation of ReLUMoT’s usefulness in practical modeling tasks. ReLUMoT is benchmarked against a variety of state-of-the-art modeling algorithms in learning both static and dynamic processes. In black-box settings, ReLUMoT matches or exceeds the performance of existing LMNs. Properly incorporating prior knowledge is shown to lead to vast improvements in model performance and size.