Applying Transformer-Based Dynamic-Sequence Techniques to Transit Data Analysis

Transit systems play a vital role in urban mobility, yet predicting individual travel behavior within these systems remains a complex challenge. Traditional machine learning approaches struggle with transit trip data because each trip may consist of a variable number of transit legs, leading to missing data and inconsistencies when using fixed-length tabular representations. To address this issue, we propose a transformer-based dynamic-sequence approach that models transit trips as variable-length sequences, allowing for flexible representation while leveraging the power of attention mechanisms. Our methodology constructs trip sequences by encoding each transit leg as a token, incorporating travel time, mode of transport, and a 2D positional encoding based on grid-based spatial coordinates. By dynamically skipping missing legs instead of imputing artificial values, our approach maintains data integrity and prevents bias. The transformer model then processes these sequences using self-attention, effectively capturing relationships across different trip segments and spatial patterns. To evaluate the effectiveness of our approach, we train the model on a dataset of urban transit trips and predict first-mile and last-mile travel times. We assess performance using Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). Experimental results demonstrate that our dynamic-sequence method yields up to a 30.96% improvement in accuracy compared to non-dynamic methods while preserving the underlying structure of transit trips. This study contributes to intelligent transportation systems by presenting a robust, adaptable framework for modeling real-world transit data. Our findings highlight the advantages of self-attention-based architectures for handling irregular trip structures, offering a novel perspective on a data-driven understanding of individual travel behavior.