June 1, 2015
Deep visual-semantic alignments for generating image descriptions
Authors
Discussion
Loading...
Member takes
Implication
Benchmark study demonstrates superior image and region description generation across standard datasets, highlighting the power of deep visual-semantic alignments.
Key Points
- The multimodal recurrent neural network generates novel natural language descriptions across both complete scenes and localized image regions.
- Benchmark evaluations on Flickr8K, Flickr30K, and MSCOCO demonstrate that the visual-semantic alignments produce state of the art retrieval results.
- Pairing convolutional neural networks over regions with bidirectional recurrent neural networks over sentences enables effective multimodal embedding.
Cite This Study
Karpathy et al. (2015) studied this question.