Kaltenbacher, T. 1, 2 & Hummer, P. 1, 2
1 Department of Linguistics, University of Salzburg
2 Centre for Neurocognitive Research, University of Salzburg
In search of a psycholinguistic theory that might explain the origin of the phonological deficit - generally believed to be the cause of dyslexia - we have created an experimental paradigm that allows us to test subjects' abilities to process phonological stimuli in visual- only, acoustic- only and audiovisual conditions. The aim of the study was to examine the perception of audiovisual speech in adolescent/adult dyslexics and detect correlations between deficient audiovisual and deficient phonological processing. Based on current models of audiovisual speech processing, we provide a conceptual and analytical framework of bimodal speech perception able to explain such correlations. Subjects/controls (n=50) were tested with standardized dyslexia tests. Eye movements were tracked while they had to a) visually identify syllables and target words (lip-readable stimuli), b) process audiovisually congruent and incongruent stimuli (Mc Gurk stimuli, both with syllables and words) from a talking face and c) read out pseudowords aloud. The acoustic-only conditions consisted of a syllable/word/pseudoword repetition task. We obtained data from our dyslexic subjects that supported our assumption that the Mc Gurk effect cannot be as robustly triggered as in controls. The eye-movement analysis focussed on lip- readable areas of interest (AOIs) in the talking faces (conditions a and b) and the cumulative fixation duration within the AOIs. The pseudoword reading task (c) was analysed according to fixation duration, cumulative fixation duration and regressions within words. Subjects performed poorest in the pseudoword reading task, resulting in significantly longer cumulative fixation durations and regressions within words (which did not occur in controls). No significant differences were found in the audio- only and visual- only task. We interpret non- susceptibility to the McGurk effect and the difficulty with the pseudoword reading task as multimodal integration deficits occurring in audiovisual speech integration tasks as well as in a grapheme- phoneme 'translation' tasks.