Key points are not available for this paper at this time.
In this paper we introduce diagNNose, an open source library for analysing the activations of deep neural networks. diagNNose contains a wide array of interpretability techniques that provide fundamental insights into the inner workings of neural networks. We demonstrate the functionality of diagNNose with a case study on subjectverb agreement within language models.
Jaap Jumelet (2020) studied this question.