Web-based object detection and sound feedback system for visually impaired people

In this paper, we proposed a low cost and flexible system that can be used in mobile devices to achieve the conversion from the visual environment to the auditory environment, which could help the blind or visually impaired people better understand their surroundings. There are two necessary parts of this system: a web-based object detection application which is achieved by SSD-MobileNetV2 object detection algorithm, can recognize the objects including persons, cars, etc. in front of the camera and a sound conversion application, which can select and play the corresponding sounds based on the detected results. Experiment results show that our proposed system achieved high accuracy in the case of single object detection. When the number of objects of the same category increases, the accuracy of the system decreases. In the case of multiple multi-category objects, our proposed system showed low accuracy in object detection. However, the accuracy of sound feedback application had good performance. Our proposed solution can be easily adapted to detect other classes of objects that we need. It is promising in developing real assistive applications for supporting visually impaired people.