The Computer Vision Box & Block Test in Rehabilitation Assessment

Post-stroke patients very commonly present upper limb deficits, while their rehabilitation comprises regular monitoring and kinematic assessments to evaluate motor recovery. One of the most used and recommended tools to objectively measure upper limb dexterity is the Box and Block Test (BBT). However, the test itself is time consuming and very cumbersome.After briefly presenting a literature review, this paper proposes a computer vision (CV-BBT) approach to the traditional BBT, as a virtual alternative of the real world procedure. Our CV-BBT integrates all the original BBT’s guidelines and procedures into an interactive computer vision experience that utilizes bleeding edge technologies such as MediaPipe Hands for hand and finger tracking. This innovative tool require neither any additional computer peripherals (smart gloves, VR headsets) nor any kind of extra physical equipment (wooden box, blocks), but works instead with just a mid-range PC and a camera. Our system can be deployed in residential spaces and the test results can be sent remotely to any physician or rehabilitation expert. The application implementation is also demonstrated and conferred in-depth.Finally, we shortly discuss some technical issues of our computer vision approach, essentially being the hand’s pose prediction accuracy and processing times, as well as present some future directions regarding our tool’s score normalization of healthy patients against those achieved with the original BBT.