Pilot user study showed a 70% increase in navigational confidence in visually impaired users, suggesting a viable offline technology solution.
Wearable assistive technologies for the visually impaired have faced a critical trade off: traditional aids offer limited environmental context, while advanced electronic solutions are frequently tethered to the cloud, compromising real-time performance and user privacy. Furthermore, few systems integrate rich conversational feedback for social awareness. This paper addresses this gap by presenting "Captain Hope," an innovative shoulder-mounted wearable device. We demonstrate the feasibility of integrating a local large language model (LLM) with real-time computer vision for concurrent social and navigational cueing on a low-cost, fully offline wearable device. Powered by a Raspberry Pi 5 and an efficient YOLOv8n model, the system achieves an average end-to-audio latency of 485 ms, validating the real-time viability of our on-device LLM integration. A pilot user study (n=5) yielded an 89.2 percent detection accuracy in dynamic environments and, critically, a 70 percent increase in navigational confidence, underscoring the qualitative value of locally-processed, context-aware audio cues. By presenting a complete system blueprint and its real-world performance benchmarks, this paper offers a significant contribution toward creating accessible, self-reliant assistive technologies that improve quality of life.
No takes yet. Share an insight, caveat, or question.
Sidhi et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: