Optimal control of jump-linear gaussian systems†

This paper investigates the problem of controlling a discrete-time linear system with jump parameters. A review of the literature is presented as well as a development of the application of dynamic programming to this class of control problems. Dynamic programming has been applied by many researchers and it was observed that no closed-form analytical solution could be constructed because of the ‘dual’ aspects of the controller. The main contribution of the present work is an algorithm, suitable for computer implementation, for the optimal dual control. The construction of the algorithm is based on transforming the dynamic programming relations into a space of sufficient statistics and using a finite-dimensional optimization procedure to obtain the optimal control as a function of the statistics. This is achieved by first developing a suitable recursive realization of a ‘filter’ which generates the sufficient statistics for the problem and then embedding this filter into the dynamic programming equations. Several examples are presented to illustrate the use of the algorithm for constructing optimal and suboptimal controllers.