Reproducibility in Activity Recognition Based on Wearable Devices: a Focus on Used Datasets

Reproducibility of proposed approaches is a crucial element in scientific fields, in order to let other researchers trust published works. Moreover, in order to let authors compare the effectiveness of a novel method to the state of the art, benchmark datasets should be commonly used. Concentrating on the task of activity recognition using data coming from wearable devices with inertial sensors, we have analyzed the reproducibility of proposed approaches with a focus on used datasets. In this work, with a literature review, we have measured what percentage of works in the literature verified their approach using public datasets or sharing the ones created on purpose. At the same time, we have also examined the characteristics of considered datasets, with attention to the amount of data recorded, involved population, and studied activities. Starting from 1289 works retrieved on Scopus, we analyzed in detail 146 of them and found out that approximately one out of three (~33%) used public datasets and that less than one out of three (~28%) of the specially made datasets were shared with the public. Moreover, considering all the examined datasets, 13% of them had restricted access (e.g. requiring requests to authors or subscriptions to websites for a fee) or were offline.