Key points are not available for this paper at this time.
The paper describes the development and application of several techniques for knowledge extraction from trained ANN models, such as the identification of redundant inputs and hidden neurons, derivation of causal relationships between inputs and outputs, and analysis of the hidden neuron behavior in classification ANNs. An example of the application of these techniques is given of the faulty LED display benchmark. References of the application of these techniques are given of diverse large scale ANN models of industrial processes.
Boger et al. (Sat,) studied this question.