Answers are grounded in this paper and cite their sources.
We introduce a new dataset, SUN-Spot, for localizing objects using spatial referring expressions (REs). SUN-Spot is the only RE dataset which uses RGB-D images. It also contains a greater average number of spatial prepositions and more cluttered scenes than previous RE datasets. Using a simple baseline, we show that including a depth channel in RE models can improve performance on both generation and comprehension.
No takes yet. Share an insight, caveat, or question.
Mauceri et al. (2019) studied this question.