Deep learning for multiple object tracking: a survey

Deep learning has been proved effective in multiple object tracking, which confronts the difficulties of frequent occlusions, confusing appearance, in‐and‐out objects, and lack of enough labelled data. Recently, deep learning based multi‐object tracking methods make a rapid progress from representation learning to network modelling due to the development of deep learning theory and benchmark setup. In this study, the authors summarise and analyse deep learning based multi‐object tracking methods which are top‐ranked in the public benchmark test. First, they investigate functionality of deep networks in these methods, and classify the methods into three categories as description enhancement using deep features, deep network embedding, and end‐to‐end deep network construction. Second, they review deep network structures in these methods, and detail the usage and training of these networks for multi‐object tracking problem. Through experimental comparison of tracking results in the benchmarks in total and by group, they finally show the effectiveness of deep networks for tracking employed in different manners, and compare the advantages of these networks and their robustness under different tracking conditions. Moreover, they analyse the limitations of current methods, and draw some useful conclusions to facilitate the exploration of new directions for multi‐object tracking.