Peter A. Heeman

Segmenting speech into utterances and resolving speech repairs is intimately intertwined with the speech recognition task of determining what words the speaker is speaking. Speech repairs and utterance boundaries cause disruptions in the local context, both acoustic and in predicting the next word given the previous words. We have proposed that speech repairs and utterance boundaries (as well as part-of-speech tags and discourse markers) be resolved during the speech recognition task. Towards this end, we have become interested in language models that can be used during speech recognition to help the speech recognizer to prune alternative acoustic hypothesis, and augmenting them to take into account phenomena of spontaneous speech. This work centers around using machine learning techniques, such as decision trees, to learn spontaneous speech phenomena.

In understanding the user's contributions to the dialogue, the dialogue system needs to make use of the discourse state to narrow down the possible interpretations. This usage of discourse knowledge needs to be applied as early on processing as possible, even during the task of speech recognition. Peter Heeman's prior work in incorporating modeling speech repairs and utterance segmentation is a first step towards incorporating more knowledge into the speech recognition task and in enlarging this task to incorporate processing that typically is done after recognition. Peter's work in automatically identifying discourse markers, words such as ``so'' and ``but'' that relate the current utterance to the discourse state, also shows the potential of incorporating discourse processing into the speech recognition problem of language modeling.