Mean-Variance Criteria for Finite Continuous-Time Markov Decision Processes

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> This technical note deals with the mean-variance problem (known as the <emphasis emphasistype="italic">average variance (AV)</emphasis> minimization problem) for finite continuous-time Markov decision processes. We first introduce a so-called <emphasis emphasistype="italic">G-condition</emphasis> which is weaker than the well known ergodicity and unichain conditions and sufficient for the finiteness of the AV of a policy. Also, we present an example of a policy having <emphasis emphasistype="italic">infinite</emphasis> AV when the G-condition is not satisfied. Under the G-condition we prove that the AV criterion can be transformed into an <emphasis emphasistype="italic">equivalent</emphasis> mean (or expected) average criterion by using a martingale technique and an observation from the canonical form of a transition rate matrix, and thus the existence and calculation of an AV minimal policy over a class of mean optimal policies are obtained by a policy iteration algorithm in an finite number of iterations. As byproduct, we obtain some interesting new results about the mean average optimality. </para>