On stochastic optimality of policies in first passage problems

In many stochastic scheduling and optimal maintenance problems that have been consisdered in the literature, the optimization criterion employed has often been equivalent to minimizing the expected first passage times to a set of “desirable” states. A typical method that has been used in establishing the optimality of a certain policy is the method of successive approximations. As an intermediate, reuslt, this techinique often horizon versions of the problem. In this paper we point out that under mild assumptions stochastically, i.e. they are optimal in expectation for all members of a sequence of appropriately defined finite horizon problems. Furthermore, this characterization can reduce a “uniformizable” continuous time problem into a sequence of approximately defined discrete time problems. In the final sections we use this characterization to establish the stochastic optimality of pertinent policies for an optimal repair allocation problem and for a scheduling problem