Generating visual questions with hop sequence reasoning

Visual Question Generation has emerged as a captivating problem that has gained considerable attention. The objective of this task is to produce meaningful and relevant questions based on a given image. The current state-of-the-art approaches for the same employ a range of modern techniques in deep learning such as active learning through asking question to an Oracle, using generative networks, reinforcement learning etc. However, these methods treat the neural networks as black boxes, lacking the ability to point out the specific image features used for question generation or to explain the reasoning behind generating a particular question. The proposed method aims to address this limitation by providing a justification for each generated question related to the given image. While other models learn latent concepts that may lead to strong predictive performance and generalization, the objective of proposed model is to emphasize faithful reasoning and interpretability. This is accomplished by employing a sequence of multiple features or hops extracted from the image. The work also proposes novel evaluation metrics for the multi-hop questions. These metrics facilitate a comprehensive analysis of the generated questions in terms of their novelty and accuracy on a hop-by-hop basis.