DoodleIt: A Novel Tool and Approach for Teaching How CNNs Perform Image Recognition

To introduce middle school students to key concepts in image recognition, we created an interactive web application that performs sketch recognition and an afterschool curriculum for its use. Our app, called DoodleIt, was inspired by Google’s Quick, Draw!, and makes use of its accompanying open-source sketch library. With DoodleIt, students make simple line drawings on a canvas area and a previously-trained convolutional neural network (CNN) identifies the object drawn. The application dynamically visualizes the different layers that are involved in the process of CNNs, including a display of kernels, the resulting feature maps, and the percentage of match at output neurons. We used DoodleIt in an 18-hour curriculum to introduce middle school students to artificial intelligence, machine learning, and data science. Four hours of content were related to image recognition and the ethics of using AI. Here, we describe the design of the DoodleIt application, the approach we used to introduce the associated ideas to the students, and how we assessed student learning. Qualitative data collected from students are presented and discussed. Our findings indicate that students were able to understand the functionality of the kernels and feature maps involved in the CNN to perform rudimentary image recognition.