A Hybrid Generative Adversarial Framework for Image Captioning with Reinforced Attention and Metaheuristic Optimization

Image captioning intends to automatically produce relevant and descriptive text for a specified image, integrating Natural Language Processing (NLP) and Computer Vision (CV) to understand visual content and express it in words. Existing image captioning methods suffer from difficulty in generating accurate and contextually rich captions, which results in captions that lack descriptive quality and alignment with visual content. The objective of this study is to develop an efficient image captioning framework capable of producing accurate and semantically rich captions from images. In this research, a hybrid Attention-reinforced transformer with contrastive learning, Serval-Frigatebird Optimization, Gaussian Error Linear Unit-Long-Short Term Memory (ArCO-SerFO-GLSTM) based Generative Adversarial Image Captioning model is introduced for performing image captioning from a given dataset. The proposed model consists of the ArCO-SerFO generator, the Reinforcement Learning (RL Generator) with a language evaluator and a discriminator. At first, in the ArCO-SerFO generator, the input image is passed through an image encoder to extract visual features and then fed to the caption decoder to generate a sample caption. The generated caption is compared with the ground-truth caption using contrastive loss, which improves the alignment between image features and the caption. In this case, the ArCO model is tuned exploiting Serval- Frigatebird Optimization (SerFO). The system then uses a RL generator, where an image encoder and multi-attention mechanism guide a language decoder to generate refined captions. These captions are evaluated by a language evaluator, and Reinforcement Learning (RL loss) updates the model based on the reward metrics. Finally, both generated captions and groundtruth captions are fed into a GELU-LSTM discriminator, which distinguishes real captions from generated caption. The GELU-LSTM is developed by incorporating a GELU into an LSTM. The developed ArCO-SerFO-GLSTM acquired Recall-Oriented Understudy for Gisting Evaluation-L (Rouge-L) of 60.19%, Mean Average Precision (mAP) of 80.13%, Bilingual Evaluation Understudy (BLEU) of 84.23%, Metric for Evaluation of Translation with Explicit Ordering (METEOR) of 31.99%, Semantic Propositional Image Caption Evaluation (SPICE) of 25.99% and Consensus-based Image Description Evaluation (CIDEr) of 123.3 with the Flickr Image dataset.