Formulae are presented for the variance and higher moments of the present value of single-stage rewards in a finite Markov decision process. Similar formulae are exhibited for a semi-Markov decision process. There is a short discussion of the obstacles to using the variance formula in algorithms to maximize the mean minus a multiple of the standard deviation.
No takes yet. Share an insight, caveat, or question.
Matthew J. Sobel (1982) studied this question.
Synapse has enriched 5 related papers. Consider them for comparative context: