See Me, Teach Me: Facial Expression and Gesture Recognition for Intelligent Tutoring Systems

Many software systems would significantly improve performance if they could adapt to the emotional state of the user, for example if intelligent tutoring systems, ATM's and ticketing machines could recognise when users were confused, frustrated or angry they could provide remedial help so improving the service. This paper presents research leading to the development of Easy with Eve, an affective tutoring systems (ATS) for mathematics. The system detects student emotion, adapts to students and displays emotion via a lifelike agent called Eve. Eve's is guided by a case-based system which uses data that was generated by an observational study. This paper presents the observational study, the case-based method, and the ATS