Video Annotation & Descriptions using Machine Learning & Deep learning: Critical Survey of methods

Video description methods aim to produce the most relevant description of a video. This could be description based on full video, frame based or based on important events of the video. This work focuses on recent advancement of video description methodologies in the form of a survey. The work spans across most of the latest deep learning methods along with other classical algorithm-based methods. The categorization of the methods found is done on various aspects like weakly supervised, Tree/graph based, Attention based, Multi Modal based, Supervised with Attention, Attention on region importance based, Video to Q/A based etc. We have performed a comprehensive comparative analysis of the methods surveyed. The relevant application areas as well their relevancy is also mapped to the categories discussed in this paper. A method's relevancy to the application areas is also complemented with the metrics to be used for evaluation. Lastly, the paper also discusses the future directions pertinent to the business requirements presented in recent Forbes's articles and our insights on the survey.