Single-Valued Risk Estimation in Classification for Dependent Data

The use of machine learning models is increasing rapidly, leading to several safety challenges. One concern is the failure rate obtained from testing, as these are only valid at a certain confidence level. Therefore, many safety assessment models like fault trees or reliability block diagrams cannot be applied in a direct manner. Including confidence in extensions of these models increases the computational effort. Furthermore, their output is not a single-valued number and contains some confidence or uncertainty measurement.To overcome this problem, we present a new approach to retrieve a single-valued upper bound estimation for the failure rate in the case of dependent test data. Consequently, it allows the early usage of classical tool-supported techniques in the system development phase. The estimation is based on concentration inequalities and can also be seen as an optimal confidence level determination. Furthermore, the degree of dependency in the form of the minimal number of partitions in a proper cover is considered. As well as the amount of test data, which makes them more realistic than common independent data-based methods. Our simulation supports the high-confidence validity of these bounds and they therefore present a reasonable estimation for initial assessment in the system design phase.