Street view imagery： Methods and applications based on artificial intelligence

： Street view imagery is a promising and growing big geo-data that provides current and historical images in more than 200 countries for urban physical environment representation and audit. Such data not only describes the visual details of the urban physical environment but also contains information about urban functions, socioeconomics, and human dynamics. Street view imagery has the potential to complement new and traditional data, such as remote sensing imagery and social sensing data. However, traditional digital image processing techniques for street view imagery handling are limited. Extracting rich semantic information from street view imagery efficiently has always been a challenging issue. Until recently, the development of artificial intelligence has led to numerous breakthroughs in image processing and machine learning. Indeed, the last few years have witnessed the fast development of deep learning and computer vision techniques, which facilitate the understanding of scene semantics from street view imagery and the quantitative representation of the urban physical and built environments. Many new applications, novel methods, and thoughts regarding street view imagery have emerged, covering research fields, such as geography, urban planning, urban design, urban economics, public health, environmental psychology, and energy. This trend has provided new perspectives for big geo-data-driven urban environment analysis, human – land relationship study, and spatial data mining and knowledge discovery. To summarize this research trend, this paper reviews the recent works on urban physical environment analysis using street view imagery. The key supporting techniques for street view imagery analytics are discussed in terms of two dimensions: deep learning and computer vision. Deep learning has been applied recently to various computer vision tasks, such as image classification, image segmentation, and object detection. The success of deep learning techniques is attributed to their ability to learn rich high-level image representations as opposed to the hand-designed low-level features used in other image understanding methods. Additionally, this paper summarizes the street view imagery applications in three aspects: place representation, sense of place, and place semantics reasoning. “ Place representation ” includes works that extract visual elements that constitute the urban physical environment; “ sense of place ” refers to works that use street view imagery to understand how people respond to their surrounding environments regarding perceptions and emotions; and “ place semantics reasoning ” refers to studies that attempt to infer and estimate invisible factors, socio-economics, demographics, and human dynamics from street view imagery. More importantly, the issues of this research field, such as the spatio-temporal uniformity of street view image data, and the lack of solid workflow of data analytics, are highlighted.Finally, the development prospects of street view imagery are discussed. Crowdsourcing platforms and the field of the autonomous vehicle will increase the number of street view image sources. More issues, including how physical environments are involved and whether a universal law exists regarding the distribution of physical elements in space, are expected to be explored in future works.