Interpreting Computational Neural Network Quantitative Structure−Activity Relationship Models:  A Detailed Interpretation of the Weights and Biases

In this work, we present a methodology to interpret the weights and biases of a computational neural network (CNN) quantitative structure-activity relationship model. The methodology allows one to understand how an input descriptor is correlated to the predicted output by the network. The method consists of two parts. First, the nonlinear transform for a given neuron is linearized. This allows us to determine how a given neuron affects the downstream output. Next, a ranking scheme for neurons in a layer is developed. This allows us to develop interpretations of a CNN model similar in manner to the partial least squares (PLS) interpretation method for linear models described by Stanton. The method is tested on three datasets covering both physical and biological properties. The results of this interpretation method correspond well to PLS interpretations for linear models using the same descriptors as the CNN models, and they are consistent with the generally accepted physical interpretations for these properties.