Separable Markovian Decision Problems

The special structure of a class of Markovian decision problems is exploited to simplify the determination of optimum policies. For certain pairs consisting of a state i and decision k, the cost c k i separates (c k i = a i + b k ), while the transition probabilities p k ij and transition time distributions F k ij are independent of i. Equivalence of a second Markovian decision problem which exploits this structure is demonstrated for the discounted and averaging cases. In addition, streamlined approaches are presented for dealing directly with the original problem, and a particular inventory model is further simplified.