Visual attention as a design variable: AI-integrated eye-tracking and semantic segmentation in urban perception analysis

Urban design often underrepresents how pedestrians perceive and visually engage with their surroundings. This exploratory study introduces a computational, evidence-based framework that integrates real-world eye-tracking data with AI-powered semantic segmentation to assess visual attention across urban streetscapes. Gaze behavior was recorded using wearable eye-trackers along pedestrian routes in a mid-density American city and categorized into seven Area of Interest (AOI) classes representing key urban elements such as sidewalks and roads, vegetation, built structures, vehicles, humans, objects and sky. Gaze-based metrics, including fixation duration, visit frequency and pupil diameter were aggregated and normalized to compute a composite Urban Attention Index (UAI), quantifying perceptual engagement per AOI class. Additional sensitivity analysis and normalization comparison were included to examine the influence of weighting and scaling choices on the UAI results. A deep learning based semantic segmentation model was trained and validated using pixel-level performance metrics to automatically label urban images, enabling the generation of attention-weighted heatmaps that spatially visualize gaze salience. The resulting AI-driven tool enables users to upload street-level imagery and receive intuitive, data-informed insights into how urban elements may attract visual attention. Results from the exploratory sample show that sidewalks, vegetation, and built structures consistently attracted higher attention, whereas sky and signage were perceived peripherally. The findings demonstrate a methodological workflow for linking gaze behavior, semantic segmentation, and spatial visualization in urban perception analysis. The proposed framework offers a reproducible basis for future larger-scale studies and supports evidence-informed evaluation of pedestrian visual experience in public spaces.