Knowledge Gaps: A Challenge for Agent‐Based Automatic Task Completion

The study of human cognition and the study of artificial intelligence (AI) have a symbiotic relationship, with advancements in one field often informing or creating new work in the other. Human cognition has many capabilities modern AI systems cannot compete with. One such capability is the detection, identification, and resolution of knowledge gaps (KGs). Using these capabilities as inspiration, we examine how to incorporate detection, identification, and resolution of KGs in artificial agents. We present a paradigm that enables research on the understanding of KGs for visual-linguistic communication. We leverage and enhance and existing KG taxonomy to identify possible KGs that can occur for visual question answer (VQA) tasks and use these findings to develop a classifier to identify questions that could be engineered to contain specific KG types for other VQA datasets. Additionally, we examine the performance of different VQA models through the lens of KGs.