``Sim2real gap", in which the system learned in simulations is not the exact representation of the real system, can lead to loss of stability and performance when controllers learned using data from the simulated system are used on the real system. In this work, we address this challenge in the linear quadratic regulator (LQR) setting. Specifically, we consider an LQR problem for a system with unknown system matrices. Along with the state-action pairs from the system to be controlled, a trajectory of length S of state-action pairs from a different unknown system is available. Our proposed algorithm is constructed upon Thompson sampling and utilizes the mean as well as the uncertainty of the dynamics of the system from which the trajectory of length S is obtained. We establish that the algorithm achieves O (f (S, M_{) T/S}) Bayes regret after T time steps, where M_ characterizes the dissimilarity between the two systems and f (S, M_) is a function of S and M_. When M_ is sufficiently small, the proposed algorithm achieves O (T/S) Bayes regret and outperforms a naive strategy which does not utilize the available trajectory.
Bajaj et al. (Tue,) studied this question.