Optimal Adaptive Control of Linear-Quadratic-Gaussian Systems

We consider the problem of adaptively controlling an unknown linear-Gaussian system with a standard quadratic cost criterion, including a control cost. By means of a counterexample, it is shown that a commonly mentioned adapative control scheme can lead to severe problems. To overcome this, a new adaptive control law, based on biasing the usual least-squares parameter estimation criterion with a term favoring parameters associated with lower optimal costs, is introduced. A salient feature of this adaptive control scheme is its imperviousness to the closed-loop identification problem. Properties such as closed-loop system identification, convergence of the adaptive control law to an optimal control law, overall stability of the controlled system and optimality with respect to the long-term average cost of the adaptive controller are proved.