Medical Image Retrieval using ROI extraction and Hybrid Bag-of-Features model

Abstract Medical image processing has become a major component in the healthcare system, where the Content-Based Medical Image Retrieval (CBMIR) technique is increasingly used. However, CBMIR still has room to improve its ability to handle multiple modalities and a large volume of images to produce satisfactory results. This paper presents a novel CBMIR framework using the Hybrid Bag-of-Features (HBoF) technique for multimodal and scalable datasets. The experiment was conducted using the IRMA 2009 and the Kvasir datasets. The HBoF model in the proposed work is a combination of Bag-of-Visual-Words (BoVW) with shape and texture features. Initially, the Region of Interest was extracted using the Dynamic Delaunay Triangulation method. Binary Robust Invariant Scalable Keypoints (BRISK) descriptor followed by the K-means clustering was used to generate the BoVW. The shape and texture features were extracted using the Edge Histogram Descriptors (EHD) and Gray-Level Co-occurrence matrix (GLCM). Finally, the Euclidean distance similarity measure was used to retrieve the relevant images. The experimental results prove the reliability and robustness of the proposed framework achieved against the state-of-art methods. Thus, our study provides a novel CBMIR framework using the HBoF model to address the challenges related to computational and retrieval efficiency of large medical image datasets.