Feldker, K. 1 , Hirschfeld, G. 1, 2 & Zwitserlood, P. 1, 2
1 University of Muenster
2 Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience
This event-related potential (ERP) study examines the influence of presentation modality on semantic integration, measured by the N400. Highly constraining sentences were presented audio-visually except for the final word, a noun that was either a highly predictable or an anomalous continuation. This noun was either presented audio-visually (speech and the speakers mouth), auditorily only or visually only. The time course and strength of the context effect (predictable vs. anomalous) differed depending on the modality of presentation. A context effect was present between 200 and 350ms for the audio-visual and the auditory-only conditions, but not for the visual-only condition. The N400 time window (350 to 600ms post-onset) discriminated between predictable and anomalous continuations in all three presentation modalities. The 600 to 800ms time window revealed a context effect for auditory-only and visual-only conditions, but no longer for the audio-visual modality. In contrast to what might be extrapolated from earlier studies on phonological processing, the context effect was stronger in the auditory-only than the audio-visual condition. Our results thus show a modality dependence of the context effect, and, most interestingly, a modulation of the ERPs by context even when the critical word could only be identified by lip reading.