Egocentric Action Recognition in Machine Inspection

In this paper, a method for the egocentric action recognition in the machine inspection is proposed. Based on our observation, it is found that the actions in machine inspection relates to the position and motion of the hands in the egocentric view. Our method consists of the three steps: data preparation, pre-processing, and building model. In data preparation, the temporal sequence of the hand's joints is converted to the three-dimension tensor. In pre-processing, the data is standardized at each dimension by using the statistics of the converted data. In building model, the data is used for training the model which combines pre-trained convolutional neural network and a few fully connected layers. We designed a task simulating the machine inspection, and collected the egocentric view data of the subjects doing the task. The experimental results show that accuracy of the recognition by the proposed method reaches 76.6% at most.