This paper proposes a new approach to the investigation of a class of controlled Markov chains with discrete time on a finite interval, which possess some monotonicity and concavity properties related to the transition function and the utility function of the decision maker. By introducing a stochastic discount factor based on the myopic optimal strategy, it is relatively easy to find an optimal strategy in the process modified by the stochastic discount. In a particular case of the model – multi-stage optimal investment problem under “hard” constraints including a call option model – this approach provides a simple way for calculation of optimal investment strategy in the modified process. In other words, the decision maker adopting the stochastic discount factor into the model can essentially simplify the determination of an optimal control for the Markov chain by using the myopic optimal strategies only, i.e., by solving a sequence of one-stage optimization problems without analyzing the more complicated dynamic programming equations.
A. Y. Golubin (Wed,) studied this question.