Enhanced pixel-level detection of food regions using a self-attention mechanism

This study aims to increase the accuracy of pixel-level detection of food regions extraction algorithms based on saliency detection, which predicts noticeable image areas that are attractive to the human eye. A method is proposed that refines food regions, detected from the noticeable areas, based on their characteristics using a graph-theory-based technique. The detected regions are typically based on noticeable portions of food items and their visually similar surroundings. In addition, state-of-the-art saliency detection is applied using a deep neural network, TransalNet, which has transformer layers. An experiment using 209 food images was conducted to demonstrate that the proposed TransalNet-based method significantly increased accuracy, which was measured by F-measure, by 3.38% compared with the conventional method. Additionally, the effectiveness of the proposed TransalNet-based method was extensively confirmed using images of both single and multiple food items.