Estimation of Fingertip Force Direction With Computer Vision

This paper presents a method of imaging the coloration pattern in the fingernail and surrounding skin to infer fingertip force direction (which includes four major shear-force directions plus normal force) during planar contact. Nail images from 15 subjects were registered to reference images with random sample consensus (RANSAC) and then warped to an atlas with elastic registration. With linear discriminant analysis, common linear features corresponding to force directions, but irrelevant to subjects, are automatically extracted. The common feature regions in the fingernail and surrounding skin are consistent with observation and previous studies. Without any individual calibration, the overall recognition accuracy on test images of 15 subjects was 90%. With individual training, the overall recognition accuracy on test images of 15 subjects was 94%. The lowest imaging resolution, without sacrificing classification accuracy, was found to be between 10-by-10 and 20-by-20 pixels.