Casper: An Assistive Kitchen Robot to Promote Aging in Place1

A rapidly aging population creates significant issues in maintaining the health and wellbeing of the elderly demographic [1]. Cognitive impairment can progressively diminish a person's memory, orientation, verbal skills, visuospatial ability, abstract reasoning, and attentional skills [2], hence, increasing the need for assistance with everyday activities. In general, this population overwhelmingly prefers to stay in their homes and age-in-place as independently as possible [3]. However, a decline in cognitive abilities may make it difficult to maintain such independence in the comfort of their own homes.In order to facilitate independent living, it is important that older adults be able to perform instrumental activities of daily living (IADLs). Due to the prevalence of malnutrition in elderly individuals, particularly those with cognitive impairments, promoting healthy meal preparation and eating habits is pertinent to maintaining quality of life. To date, even though a number of smart home technologies have been developed to assist the elderly with IADLs, few technologies have been targeted for the kitchen environment.In this paper, we propose the development of a unique assistive kitchen robotic system to enable the elderly with cognitive impairments to independently carry out regular kitchen activities. The Casper robot will assist the users with kitchen activities such as remembering mealtimes, choosing a meal option, remembering the locations of stored items, and assisting step-by-step in the meal preparation process, via the use of speech, facial expressions, and a touchscreen interface.This project addresses the need for an assistive kitchen robotic system for the home that incorporates a cognitive assistance feature to help a user overcome initiation, planning, attention, and memory deficits, while performing regular kitchen activities. The advantage of using the robot is that it is a proactive autonomous system that can find people in the home at mealtimes, escort them to the kitchen, and provide them with assistance in choosing and preparing a meal.The Casper robot, Fig. 1, is a human-like robot consisting of a wheeled omnidirectional mobile base onto which a torso with two arms and a head is placed. The robot's arms (three degrees-of-freedom each), emotive head, and tablet display are used for direct verbal and nonverbal (i.e., greeting and pointing gestures, facial expressions, video, and text instructions) communication. Using a combination of light-emitting diode (LEDs) for both the robot's eyebrows and mouth, Casper can display a standard set of facial expressions (happy, sad, surprised, angry, and neutral) that assist in creating social context during interactions. In Fig. 1, a happy facial expression is displayed. The robot's main processing unit is a mini-PC placed in its base that runs the robot operating system framework [4].The intended assistive scenario (demonstrated here) for the robot consists of the following steps, Fig. 2. First, the robot begins searching the home to locate the user to initiate meal preparation. Once the user is identified, the robot escorts him/her to the kitchen area, and guides him/her through a meal preparation task.Casper utilizes a predictive search strategy to find the user by prioritizing search locations based on patterns of user locations (e.g., kitchen, living room) and behaviors (e.g., meal preparation, leisure activity) in the home environment, Fig. 2. Namely, a hidden Markov model (HMM) is used to define relationships between user locations ot and behaviors xt at time t. This model is then used to predict the user's current location using any observed information on the user's past location (e.g., TV left on in living room).The transition matrix T and emission matrix E for the HMM are trained using maximum-likelihood estimation [5] on data provided by self-reported user schedules. T governs state transitions between neighboring behaviors xt and xt−1, while E determines the likelihood of the user being at location ot during behavior xt(1)xt=Txt-l, ot=ExtFrom an observed sequence of location beliefs o1:t, Casper determines the associated behavior beliefs x1:t using the forward-backward algorithm [5], where k = 1:t(2)P(xk|ol:k,ok+l:t)∝P(ok+l:t|xk)P(xk|ol:k)Sequence x1:t is then used to determine likely future behavior states xt+1:∞ via Eq. (1). A ranking of potential search locations osearch can then be determined using a discounted summation of the associated locations (discount factor of y)(3)P(osearch|xt+l:∞,E)=∑k=1∞γkExt+kAs the robot searches the environment, the prior location beliefs and the resulting search prioritization are continuously updated on any new observations of the environment.While searching for the user, Casper determines the route between two locations in the home by performing an A* search [6] on a 2D grid map of the environment. This map is obtained by the robot using scans provided by its laser scanner, which are stitched together using the Gmapping SLAM (simultaneous localization and mapping) technique based on Rao-Blackwellized particle filters [7]. The map is then segmented into rooms. Local obstacle avoidance using 3D data from the RGB-D (red, green, blue plus depth) cameras is used to safely navigate Casper around static and dynamic obstacles within the robot's full workspace.Casper detects people in the environment using its torso-mounted RGB-D camera, by determining human-shaped geometries in the point cloud data and evaluating the geometries' RGB values with a histogram of oriented gradients feature detector [8]. Once a person is detected, the robot uses images from its 2D tablet camera to perform user identification via the OKAO facial recognition library [9], which matches facial features in the image against a database of known users.When the user has been identified, Casper escorts him/her to the kitchen (via the aforementioned planning and navigation approach) to prepare a meal. Once in the kitchen, Casper presents a list of potential recipes for the meal on its tablet and requests the user to select one from this list using the tablet touchscreen interface, shown in Fig. 2. The robot then assists the user in locating the food items needed for meal preparation by displaying a list of the items on its tablet and pointing to where the items are located in the kitchen environment. Subsequently, the robot prompts the user through the preparation steps of the selected recipe by showing him/her prerecorded videos of each step (Fig. 2). Once the user finishes all of the preparation steps, the meal is ready to be eaten.We conducted a system performance evaluation of Casper and its corresponding modules. We evaluated people's ability to recognize Casper's facial expressions during interactions with 27 individuals, who selected observed robot emotions from a list of potential emotions. Perceived usefulness and perceived ease of use constructs from the technology Acceptance models [10] were used to rate the meal preparation interface on a five-point Likert scale (5—strongly agree, 3—neutral, 1—strongly disagree). In addition, the user search and identification modules were also evaluated in a personal home setting for the proposed application.Recognition rates for all of Casper's facial expressions were 80% and higher as shown in Table 1. In addition, positive ratings were obtained with respect to perceived usefulness (μ = 4.0, σ = 0.83) and perceived ease of use (μ = 4.5, σ = 0.93) for the meal preparation interface. The user search method was used to predict the current location of a user based on a priori observations of the user and typical daily schedules in a single floor home with five rooms (70 m2). The predictive accuracy of the HMM-based user search improved with inputs of increasing information about prior user locations, Table 2. When compared to a random search method (with no prior information), the HMM search prioritization consistently located the user with fewer number of rooms explored. The user detection and identification method was found to work effectively within a range of 3.2 m from the robot for detection and 1.8 m for identification, with an overall field of view of 57 deg from the front of the robot.The goal of our research is to develop an at-home autonomous assistive robot to promote the aging-in-place of elderly individuals suffering from mild cognitive impairments. The Casper robot can actively communicate mealtimes to users and provide assistance in selecting and preparing a meal actively. A system performance evaluation study showed that the robot was effective in displaying a range of emotions that are needed for social interactions, and potential users had positive attitudes towards the robot for the intended activity. In addition, the robot is capable of searching, finding, and identifying a user of interest in a home environment to initiate the meal preparation activity. Future work consists of conducting pilot user studies with older adults with cognitive impairments using an integrated, autonomous Casper robot.