Development of a Large Language Model-Based System For Enhanced Access to Chess for Visually Impaired Individuals

Chess is one of the miraculous inventions of mankind. However, traditional chess interfaces often pose significant challenges for individuals with visual impairments, limiting their ability to engage with the game independently. The objective of this paper is to develop a computerized system where a visually impaired person can play chess with a machine, with another person, visually impaired or not, and train with chess puzzles. It leverages recent advances in the development of machine learning models, most notably the latest versions of large language models (LLMs), speech-to-text conversion models, and chess AI models, to build a cohesive system. We developed a system that initially converts audio input from the visually impaired user to text representation, leveraging an audio-to-text machine learning (ML) model and an LLM. Subsequently, the text representation is fed to chess AI, Stockfish, which generates the optimal countermove. Finally, the optimal counter-move from Stockfish is passed through a text-to-speech model that is played as audio to the user. The proof-of-concept system developed in this study can make chess accessible to visually impaired people by enabling them to engage with the game in a way that is natural and intuitive.