ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners

Digital learning tools for American Sign Language (ASL) often lack the interactive depth necessary to engage learners effectively. This paper introduces a novel, browser-based word search game designed to facilitate ASL vocabulary familiarization through gamified interaction. The system employs a two-tier architecture consisting of a React-based frontend and a Flask-based backend. At its core, the application integrates a lightweight, skeleton-based Isolated Sign Language Recognition (ISLR) model, utilizing a Stacked Transformer-based Spatial-Temporal Attention Network to enable real-time webcam-based word entry during the configuration phase. This model, trained on the WLASL-100 dataset, achieves a Top-5 test accuracy of 88.48% with an average model inference latency of 141 ms, enabling real-time webcam input without proprietary hardware. Furthermore, we implement a constraint-satisfaction puzzle generation algorithm that achieves a 100% success rate in creating interlocked, multi-directional grids. Our results demonstrate that merging computer vision with pedagogical game mechanics provides an accessible, high-performance tool for the Deaf and Hard-of-Hearing (DHH) community, bridging the gap between static instruction and active linguistic practice.