A Content-based Image Retrieval by High-level Features from Self-supervised Learning of Pre-trained Deep Neural Networks Model

This research aims to developed a Content-based image retrieval model for resolve semantic gaps problem where low-level features cannot correctly convey the meaning of images. The result of developed model consists of 3 modules: 1) build the image description set module, it applies a CLIP (Contrastive Language-Image Pre-training) to learn the meaning of images by self-supervised learning from the relationship between images and caption on image encoder and text encoder with cosine similarity before collecting to the image description set and create to an image feature vector. 2) query processing module to learn the meaning of the text and constructs it as a query feature vector, and 3) vectors matching module with similar values between image feature vectors and query feature vectors before sorting by relevance and display the result to the user. The result of image retrieval on the Flickr30k dataset with order-unaware metric had a mean of recall is 0.93 when the result was in the top 10 is very high, but anyway it also found that the main barrier to the accuracy of the results was image variation. When comparing the image retrieval results with the image custom dataset, it was found that the average of recall was in the same direction. And there is no problem that the model's performance is compromised when working with previously unseen data. Demonstrate that the model can retrieve content-based images effectively. It also supports users with search terms in the form of natural language that are based on the meaning of the image rather than the grammar of the language. This impact of results is a guideline for information retrieval in the future.