Applying Object Recognition to Improve Image Retrieval by Color Features

As the volume of data in the digital world increases, the need to efficiently extract information from images becomes more and more important. One of the key aspects of image retrieval is the ability to accurately and quickly search for similar images in large collections. Color features are essential to identify objects and regions in images, which makes the integration of object recognition techniques suitable to improve the image retrieval process. It is possible to search for similar images by colors only or by objects only, but common sense suggests that much higher search accuracy could be reached when mixing these two methods of searching. The aim of this work is to test and experimentally prove the assumption that combined search by using color features together with object recognition will improve both search accuracy and results ordering in image retrieval systems. To achieve that we enhance the architecture of our content-based image retrieval system, by adding the YOLO real-time object detection system at the input to recognize the type of the search object and to filter and order results not just by colors only, but by both colors and object types. Results show that adding the object type as a search constraint, together with the color features, more than doubles both precision and recall in case of highly detailed objects and leads to 4 times increase in precision and recall for less detailed or monochromatic objects.