June 1, 2015
Deep visual-semantic alignments for generating image descriptions
Key Points
- The multimodal recurrent neural network generates novel natural language descriptions across both complete scenes and localized image regions.
- Benchmark evaluations on Flickr8K, Flickr30K, and MSCOCO demonstrate that the visual-semantic alignments produce state of the art retrieval results.
- Pairing convolutional neural networks over regions with bidirectional recurrent neural networks over sentences enables effective multimodal embedding.
Authors
Discussion
No takes yet. Share an insight, caveat, or question.
Implication
Benchmark study demonstrates superior image and region description generation across standard datasets, highlighting the power of deep visual-semantic alignments.
Cite This Study
Karpathy et al. (2015) studied this question.