Three Challenges in Utilizing Machine Learning to Predict Human Behavior from Observational Data

This paper outlines the three principal challenges encountered during the machine learning efforts of the Real-Time Adaptive Systems (R-TAPS) project to learn the behavior of chemical plant workers and provides recommendations for future HRI projects that face similar problems. This paper specifically focuses on data labeling, annotation processes, and model evaluation. The R-TAPS machine learning efforts aimed to predict worker behavior during task execution in real-time. It employed a step-level label system, which caused difficulties in predicting worker behavior on a timestamp level. The annotation process that was carried out lacked uniformity, leading to inconsistencies in the data entries. The model performance presentation caused confusion due to multiple performance values and a lack of understanding of what metric to evaluate. In response, this paper offers recommendations that address each challenge for future efforts.