On the Accuracy Versus Transparency Trade-Off of Data-Mining Models for Fast-Response PMU-Based Catastrophe Predictors

In all areas of engineering, modelers are constantly pushing for more accurate models and their goal is generally achieved with increasingly complex, data-mining-based black-box models. On the other hand, model users which include policy makers and systems operators tend to favor transparent, interpretable models not only for predictive decision-making but also for after-the-fact auditing and forensic purposes. In this paper, we investigate this trade-off between the accuracy and the transparency of data-mining-based models in the context of catastrophe predictors for power grid response-based remedial action schemes, at both the protective and operator levels. Wide area severity indices (WASI) are derived from PMU measurements and fed to the corresponding predictors based on data-mining models such as decision trees (DT), random forests (RF), neural networks (NNET), support vector machines (SVM), and fuzzy rule based models (FuzzyDT and FuzzyID3). It is observed that while switching from black-box solutions such as NNET, SVM, and RF to transparent fuzzy rule-based predictors, the accuracy deteriorates sharply while transparency and interpretability are improved. Although transparent automation schemes are historically preferred in power system control and operations, we show that, with existing modeling tools, this philosophy fails to achieve the “3-nines” accuracy figures expected from a modern power grid. The transparency and accuracy trade-offs between the developed catastrophe predictors is demonstrated thoroughly on a data base with more than 60 000 instances from a test (10%) and an actual (90%) system combined.