Digital Iconology: Classification and Association in Visual Knowledge Systems

To fully exploit the potential of artificial intelligence and computer vision for art historical research, better understanding is required of how artificial intelligence and computer vision interact with human intelligence and observation. Like the automated step-by-step approach of deep learning underlying artificial intelligence and computer vision to recognize which features and patterns in data are more meaningful than others, our human reading of the produced results is a gradual process of understanding images in the context of our knowledge. This raises the question how this process can be organized in an optimal way to set up experiments with artificial intelligence and computer vision for applications in digital art history. To answer this question, different modes of organizing, exploring and analysing visual resources will be compared from a historical perspective: Henri van de Waal’s rather unknown classification of the arts “Beeldleer”, and William S. Heckscher’s Index Iconologicus. Both knowledge systems were inspired by Aby M. Warburg and his followers. My claim is that a historical understanding of the initiatives of Van de Waal and Heckscher combining hierarchical classification and association is useful for experiments in digital iconology. Based on this, I propose a tool for human–computer interaction in the pre-classification, classification and post-classification of visual resources.