Learning to operate a sensorized bionic hand assessed with a portable computer-vision system combined with eye tracking and prosthetic sensors

As prosthetic technologies advance, the demand for methods to objectively assess their impact on user function is growing. Current evaluation tools rely mostly on subjective questionnaires and task performance metrics that are insufficient to capture real-life user interactions of prosthetic devices with external objects. Here we developed a portable system that integrates prosthetic sensors, eye-tracking, and computer vision-based object segmentation to assess the visuomotor behavior involved in operating a bionic hand. The bionic hand is equipped with tactile and positional sensors, and their signals are converted to neurostimulation to produce somatic sensations. This system enables a comprehensive analysis of visual attention and motor performance across distinct phases of object manipulation. Its mobile design permits deployment in everyday environments. We propose that this approach be employed to enhance patient training to operate sensorized prosthetic limbs by providing comprehensive analysis of the performance. In support of this proposal, our results reveal subject-specific responses to sensory feedback in both able-bodied individuals receiving transcutaneous electrical stimulation and one amputee implanted with peripheral-nerve electrodes.