Attention-based deep learning models for cancer RNA sequence classification

This thesis investigates the application of attention-based deep learning models for cancerrelated RNA sequence classification under realistic laboratory and computational constraints. Although genomic and transcriptomic sequences contain biologically meaningfulsignal, this information is sparse, context-dependent, and embedded within longnucleotide sequences, posing significant challenges for reliable and scalable modelling.The work focuses on sequence-level cancer versus normal classification using curatedRNA transcript sequences, treating upstream sequencing and alignment artefacts as potentialsources of distribution shift rather than modelling targets. To capture both localmotif-level patterns and longer-range contextual dependencies, we propose a hybrid architecture that combines a convolutional feature extraction stem with an efficient Transformer backbone based on linearized attention (Performer with FAVOR+). This designenables global contextual reasoning while maintaining linear computational complexitywith respect to sequence length, making it suitable for deployment on mid-range, sharedlaboratory hardware. Sequences are processed using k-mer tokenization and overlapping windowing, followed by a learnable gated pooling mechanism that aggregates window-level evidence into a sequence-level prediction. In addition to predictive performance, particular emphasis is placed on calibration, operating-point selection, and the production of governanceartefacts such as reproducible data splits, latency and memory profiling, and audit-readyprediction outputs. Experimental results demonstrate that efficient attention models substantially outperform classical, non-attention baselines, while remaining computationally feasible under realistic resource constraints. These findings highlight the ability of efficient attention mechanisms to provide a favorable balance between representational capacity and operational efficiency, without requiring the computational overhead of heavier attention-based architectures. Overall, this thesis presents a pragmatic end-to-end framework for cancer RNA sequence classification that aligns biological considerations, statistical robustness, and deployment feasibility, and provides guidance for the responsible application of deep learning models in genomic research settings.