Enhancing gaze estimation using dilated CNN architectures with transfer learning

In this paper, we present a gaze estimation network with an overall architecture mainly consisting of object detection, head pose detection, and gaze detection. For object detection, we detect objects from an input image and mark the head and two eyes. For head pose detection, we use the Hopenet network to estimate the direction of the head. This network uses separate loss functions for three angles and ensures they do not conflict with each other. The VGG16 convolutional layers in this network are replaced with dilated convolutions to expand the spatial extent of the neural network. Hence, we can better capture detailed information on head posture. For gaze detection, we estimate gaze direction by detecting RGB images of the eyes. We also use dilated convolutions in VGG16 for feature extraction and incorporate the head pose angle. We finally use three fully connected layers to detect the gaze direction from the images and incorporate transfer learning to improve estimation accuracy. Experimental results have demonstrated the effectiveness of the proposed approach compared to the previous methods.