Layer-wise Prediction for Interpretation of Hidden Representations in Deep Neural Networks through Evolving Probability Distributions

The sequential architecture of DNNs enables accurate predictions by encoding input features, yet their limited interpretability and transparency during forward propagation undermine the credibility of predictions. Therefore, we propose layer-wise prediction for interpretation method (LPI), which involves mapping the representation from each layer to a probability distribution, explaining the hidden-layer prediction process. To ensure interpretation fidelity to DNN mechanisms, we employ a reconstruction strategy to map probability distributions back to hidden layer features while reducing their influence on model prediction. Meanwhile, we define a rank-index metric to quantify the consistency of probability distribution evolution. Furthermore, we show how each hidden layer probability distribution evolves consistently during DNN forward propagation. Experiments on various models show that LPI achieves superior performance in consistency, stability, fidelity and certainty, outperforming AdditionalLayer, SDN, MetaGF, and DFS. Notably, LPI not only enhances deep learning interpretability but also provides valuable insights into reliable and credible model security assessment. Here, the authors introduce a hidden-layer interpretation method that maps intermediate DNN representations to probability distributions and reconstructs features to ensure faithful explanation.