Toward computers that recognize and respond to user emotion

For a long time emotions have been kept out of the deliberate tools of science; scientists have expressed emotion, but no tools could sense and respond to their affective information. This paper highlights research at the MIT Media Laboratory aimed at giving computers the ability to comfortably sense, recognize, and respond to the human communication of emotion, especially affective states such as frustration, confusion, interest, distress, anger, and joy. Two main themes of sensing—self-report and concurrent expression—are described, together with examples of systems that give users new ways to communicate emotion to computers and, through computers, to other people. In addition to building systems that try to elicit and detect frustration, our research group has built a system that responds to user frustration in a way that appears to help alleviate it. This paper highlights applications of this research to interface design, wearable computing, entertainment, and education, and briefly presents some potential ethical concerns and how they might be addressed.