Visual Representation Quality Matters More Than Multimodal Enhancement in Indonesian Word-level Lipreading

Visual speech recognition (VSR) for low-resource languages remains challenging, particularly for Indonesian lipreading.This study investigates whether adding architectural components or audio information can substantially improve word-level recognition on the IDLRW (Indonesian Lipreading Word) dataset (48,083 clips, 100 classes).Using a controlled pipeline and the same test set (n = 4,893), we compare visual-only and multimodal configurations using Top-1 word accuracy rate (WAR).A convolutional encoder outperforms the initially considered Vision Transformer at this data scale.The largest improvement comes from better visual-temporal representation: domain-specific pretraining adds 3.0 percentage points, while replacing the temporal encoder with a Conformer using corrected relative positional encoding adds a further 4.8 percentage points.Audio-logit distillation alone provides no measurable benefit, whereas representation-level audio knowledge improves performance through cross-modal memory.A homophone analysis indicates a practical visual ceiling of 72-73%, compared with 93.29% for audio.The best model achieves 61.97% WAR, while a four-model ensemble achieves 63.72%, exceeding the published baseline of 60.51%.These results suggest that visual information, rather than model capacity, remains the main limitation.