Key result
A deep learning system using multimodal audio and video data estimated users' Quality of Life with a total average correct answer rate of 65%.
A deep learning-based telecare system can estimate user Quality of Life from multimodal audio and video conversational data with moderate accuracy.
Answers are grounded in this paper and cite their sources.
May support AI telecare for QoL monitoring; hypothesis-generating and requires prospective validation before clinical use.
Quality of life (QoL) is an effective index of well-being, including physical health as well as mental state of individuals. In this paper, audio and video data were collected from healthy participants during a natural language conversation with a conversational agent. This yielded ten hours of audio and video data. A system that uses a deep learning architecture was constructed to estimate the user's QoL. This system was built using a 3D convolutional neural network and an SVM for multimodal data. In order to evaluate the accuracy of the estimation system, two experiments were conducted. In the first experiment, the QoL question-answer estimation experiment, the accuracy of “Physical functioning”, “Role physical”, “Body pain”, “General health”, “Vitality”, “ Social functioning”, “Role emotional”, and “Mental Health” scale was 84.0%, 68.0%, 67.8%, 62.3%, 51.3%, 68.0%, 54.8% and 47.0% in order. The total average of the correct answer rate was 65%. In the second experiment, the QoL score-regression experiment, the distribution of the difference between the actual score and the estimated results, known as error, was investigated. These results imply that the features necessary for QoL estimation can be extracted from audio and video data except for the “Mental Health” domain. These results also show the usefulness of the estimation system using multimodal data. Finally, suggestions and discussions regarding the plausible behavior of the estimation results were made from the viewpoint of human agent interaction in the field of elderly welfare. From the results, we concluded that it is possible to extract high dimensional information regarding the QoL of a human including their satisfaction level towards daily life and social activities from multimodal data.
No takes yet. Share an insight, caveat, or question.
Nakagawa et al. (2018) studied Quality of Life. Deep learning system (3D CNN and SVM) was evaluated on Total average correct answer rate for QoL question-answer estimation. A deep learning system using multimodal audio and video data estimated users' Quality of Life with a total average correct answer rate of 65%.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: