HybridNTT: A Lightweight Learning-Based Dynamic Path Selection Framework for Fast Number Theoretic Transform

Fast and efficient computation of the Number Theoretic Transform (NTT) is essential for accelerating cryptographic algorithms and large integer multiplications. However, conventional NTT implementations rely on fixed computation paths that fail to adapt to varying input distributions and hardware-specific conditions. In this paper, we propose a HybridNTT that is a hybrid neural network architecture that combines 1D convolutional layers with Transformer encoders to dynamically predict the optimal NTT execution path. The model is trained on a large-scale synthetic dataset comprising various input distributions—uniform, normal, sparse, sorted, bursty, and patterned—alongside multiple NTT parameter settings, including different moduli and transform sizes. Experimental results demonstrate that HybridNTT achieves classification accuracy over 92% and significantly reduces NTT execution time by an average of 34.7%, while maintaining low inference overhead (<5%). These findings highlight the feasibility of machine learning-based optimization for fundamental mathematical operations like the NTT. The proposed method can be seamlessly integrated into cryptographic libraries or hardware accelerators to enable adaptive, context-aware performance tuning, offering a practical step toward intelligent mathematical computing.