Dynamically optimized replacement with a Markovian renewal process

We consider a system that gradually deteriorates. At random times T n the system is inspected and may or may not be replaced, but it must be if its state is too bad. The process of deterioration is described by a Markovian renewal process. Under some monotonicity conditions we derive a policy for the finite and the infinite horizon which minimizes the total discounted costs. This paper is a generalization of a result of Feldman (1977).