Continuous-time Markov Chains and their Limit Distribution for Systems with Machine Learning Components

Markov Chains and especially Continuous-time Markov Chains are a well-established technique to model a wide range of systems. Particularly in reliability and availability analysis, they are used to quantitatively evaluate the probability of critical states for infinite time. Traditionally the transition rates are single-valued numbers, based on the failure rates of the system components. With the usage of machine learning models inside the system, this precondition is no longer true. The black box character of most machine-learning models does not allow to retrieve a single-valued numbers for the failure rate. However, it is possible to obtain a confidence interval for the failure on demand, which will cause the transition rates inside a Markov Chain to be given with confidence intervals only. In this contribution, we will outline the changed setup posed by transition rates given in confidence intervals only, with respect to the computations of the Markov Chain stationary distribution. Furthermore, we are going to point out how existing Markov Chain variants can handle this changed paradigm, or put differently, how a small additional computation can handle the output confidence level. Furthermore, we will propose an alternative way to compute the stationary distribution, without the usage of imprecise probability theory, interval arithmetic or similar. This method is based on symbolic computation and therefore considered particularly useful in cases in which one or several equivalent transition rates are given only by their confidence interval.