Evaluation of an automatic word recognition system over dialed-up telephone lines

An evaluation of an automatic word recognition system F. Itakura, IEEE Trans. Acoust. Speech Signal Process. ASSIP-23, 67–72 (1975) has been carried out over dialed-up telephone lines using a laboratory computer on line. Thirteen speakers participated in the evaluation, calling up the system once a day over a five-month period. In each experimental session speakers were instructed by voice prompt to provide utterances of 12 words spoken in isolation. These words were randomly selected in each session from an 84-word vocabulary, 50 of whose entries are North American cities, designed to give the speaker access to airline flight time-table information. At each trial the speaker was informed whether his utterance was correctly recognized or not. There are two categories of error: an incorrect match or a rejection (no match). Speakers were requested to repeat words not recognized on the first attempt a second and, if required, a third time. The average number of trials per speaker over the entire experimental period was 840. The median percentage of incorrect matches on the first attempt over the 13 speaker was 2.7% while the median percentage of rejections was 5.7%. The median percentage of words still not recognized after 3 attempts was 1.5%.