Key result
Bag of hallucinations post-processing reduces word error rate and safeguards against non-speech audio hallucinations.
Why the study?
Hallucinations of deep neural models are amongst key challenges in automatic speech recognition.
Post-processing text transcriptions using a bag of hallucinations can reduce word error rate and mitigate hallucinations in the Whisper ASR model.
May improve ASR reliability for clinical notes; leaves open validation in diverse cardiovascular datasets.
Hallucinations of deep neural models are amongst key challenges in automatic speech recognition (ASR). In this paper, we investigate hallucinations of the Whisper ASR model induced by non-speech audio segments present during inference. By inducting hallucinations with various types of sounds, we show that there exists a set of hallucinations that appear frequently. We then study hallucinations caused by the augmentation of speech with such sounds. Finally, we describe the creation of a bag of hallucinations (BoH) that allows to remove the effect of hallucinations through the post-processing of text transcriptions. The results of our experiments show that such post-processing is capable of reducing word error rate (WER) and acts as a good safeguard against problematic hallucinations.
No takes yet. Share an insight, caveat, or question.
Barański et al. (2025) studied Whisper ASR hallucinations. Bag of hallucinations (BoH) post-processing was evaluated on Word error rate (WER). Post-processing text transcriptions using a bag of hallucinations (BoH) reduced word error rate (WER) and safeguarded against problematic hallucinations induced by non-speech audio.
Synapse has enriched 4 closely related papers on similar clinical questions. Consider them for comparative context: