Prosody-based detection of the context of backchannel responses

ABSTRACT Current spoken dialogue systems lack positive feedback such asbackchannels, which are common in human-human conversa-tions. To develop more natural human-computer interfaces, theinvestigation of backchannel-responses are indispensable. In thispaper, we propose a method for detecting the precise timing forbackchannel responses in Japanese and aim at incorporating suchmethod in future spoken dialogue systems. The proposed methodis based on machine learning technique with a variety of prosodicfeatures. It is shownto be effectivein automatically derivingrulesfor detecting the contexts of backchannels. The performance ofour method is considerably better than previous methods. 1. INTRODUCTION Many researchers have reported that people hesitate to talk withspokendialogue systems due to the lack of positivefeedback fromthe systems such as backchannels, which are common in human-human conversations 3, 6. To develop more natural human-computer interfaces, the investigation of backchannel-responsemechanisms are indispensable. In this paper, we propose amethodfordetecting the precisetiming forbackchannel responsesin Japanese and aim at incorporating such method in future spo-ken dialogue systems.In the proposed method, the contexts for backchannels are de-tected by using only prosodic features such as fundamental fre-quency and energy, which are relatively easy to handle by currentspeech technology. In contrast to the existing methods, whichuse very limited number of features and hand-made heuristics, weemploy a machine learning method with a varietyof prosodic fea-tures which might be relevant to the detection of the backchannelcontext. It will be shown that our method is effective in automati-cally deriving rules for detecting the contextsof backchannels andthat it performs considerably better than previous methods.In Section 2, we review related works on backchannels inJapanese conversation and automatic detection of the timing forbackchannels. In Section 3, we describe the spoken dialogue cor-pus used in our study and provide our definition of backchannels.In Section 4, we conduct a psychological experiment in order tocategorize positive and negative contexts for backchannels whichare common to average humans. In Section 5, we obtain, by us-ing decision tree learning method, prosodic cues which best dis-criminate the positive and negative contexts for backchannels. InSection 6, we summarize the paper.