Warai Transformer: Humorous Caption Generation from Images Using Transformer-Based AI

In this study, we propose a novel AI model for humorous caption generation, called Warai Transformer (WiT), specifically designed for the Japanese Ogiri task. Conventional approaches to Ogiri generation are typically trained on image–humor caption (Ogiri) pairs. However, they often encounter difficulties such as domain discrepancies between image feature extractors and training data for Ogiri generation, as well as limitations in language generation accuracy due to the reliance on LSTM-based encoders. To address these issues, WiT employs a Transformer-based feature extractor, enabling more accurate visual understanding and the generation of captions, including the intentional incongruities and twists characteristic of Ogiri. Furthermore, we propose a preprocessing pipeline that automatically filters unsuitable data, such as unrealistic images, images containing text, and Ogiris with specific proper nouns. Experimental evaluations demonstrate that WiT surpasses conventional models in terms of training loss, and subjective evaluation through a questionnaire indicates its superiority in generating humorous captions. These results confirm the effectiveness of Transformer-based architectures for creative text generation tasks such as Ogiri.