GAN Augmented Hybrid Transformer Network (GHTNet) For Ancient Tamil Stone Inscription Recognition

Ancient Tamil inscriptions, like Brahmi and Vattezhuthu, hold immense historical and cultural value. However, the degradation of stone surfaces, occluded characters, and script evolution over centuries pose significant challenges to accurate digitization and translation. This paper proposes GHTNet, a novel GAN-augmented Hybrid Transformer Network capable of recognizing and translating Tamil stone inscriptions into modern Tamil directly on mobile devices. The proposed pipeline begins with DnCNN-based de-noising and perspective projection. To improve recognition color feature-based augmentation is performed using CycleGAN and Conditional GAN. Character recognition is achieved using a combination of Swin Transformer and TrOCR, effectively extracting complex script features. Subsequently, Decoupled Attention Network (DAN) and VisionLAN are employed for word recognition. The final stage involves sentence formation and translation using Graph Attention Networks (GAT) and Neural Machine Translation (NMT) models. Extensive experiments validate that GHTNet significantly reduces character and word error rates while achieving high translation accuracy of 98%.