A multi-modal social media data analysis framework: Exploring the complex relationships among urban environment, public activity, and public perception—A case study of Xi’an, China

• This study explores the complex relationships between public activity, perception, and the urban built environment using multi-modal social media data. • It reveals how the built environment influences public activity patterns over time and space. • Certain spatial elements play a crucial role in evoking emotional resonance and fostering cultural identity. • There is a close dynamic feedback loop between public activity and perception. Understanding the intricate relationships between the urban built environment, public activity, and public perception is vital for effective urban planning strategies and social sustainability. This study develops a multi-modal data analysis framework, incorporating social media data, topic modeling, and spatial statistical techniques. Using Xi’an, China, as a typical case, it examines how public activities change over time and space, how they interact with urban spaces, as well as how different aspects of the built environment shape public perception.The main findings are as follows:(1) Public activity varies significantly across day and night, interacting in complex ways with urban spatial elements;(2) Public activity is driven by multiple interconnected factors;(3) The built environment affects public perception unevenly, with certain spatial elements evoking strong emotional resonance and reinforcing cultural identity;(4) A dynamic feedback loop exists between public activity and perception.This research offers novel insights into precision-oriented, human-centered, and sustainable urban development.