Study on scene classification with unimodal and multimodal deep optimization learning techniques for visual-acoustic data

Scene classification is an emerging need in social applications, such as surveillance security and autonomous actions, in which unimodal learning techniques are commonly used to classify scenes from either audio or visual data.Researchers are advancing advances to develop more accurate scene classifications using multimodal techniques.Here, two modalities of data, i.e., both visual and acoustic, are learned for cross-modal fusion scene classification.This paper presents unimodal learning techniques with either acoustic or visual modalities, and the state-of-the-art in multimodal learning techniques for a comparative study of scene classification results.In unimodal, visual-based learning, scene classification is achieved as well as with acoustic-based learning techniques.The experimental study conducted using publicly available benchmarked datasets, TAU datasets, its results demonstrated that recent multimodal learning techniques, Attention graph convolutional network (AGCN), Context-Aware Proposal-Boundary Network (CAPB), Audio-Visual Semantic Modelling (AVSM), gain higher performance values than other existing techniques.