Some Notes on Dynamic Programming and Replacement

In the first section a modification to Howard's policy improvement routine for Markov decision problems is described. The modified routine normally converges the more rapidly to the optimal policy. In the second section a particular form of recurrence relation, which leads to the rapid determination of improved policies is developed for a certain type of dynamic programming problem. The relation is used to show that the repair limit method is the optimal strategy for a basic equipment replacement problem.