Wendler, K. 1 , Burigo, M. 1 , Schack, T. 1 & Knoeferle, P. 2
1 Neurocognition & Action Group, CITEC (Cognitive Interaction Technology Excellence Cluster), Bielefeld University
2 Department of German Language and Linguistics, Humboldt University Berlin
When people listen to sentences, they inspect the location of mentioned objects even if these objects are no longer present. Arguably then, humans can exploit scene-like memory representations during comprehension. The present research examined to which extent a ?blank screen? can substitute a real scene. Therefore, we conducted an experiment, in which we observed gaze behavior to picture-sentence mismatches in the blank screen versus visual world paradigm. Crucially, we examined potential differences in eye movements for different mismatch types (i.e. verb-action versus thematic role relations mismatches). Motivating our gaze predictions from EEG research, we expected that role and action assignment would be associated with distinct gaze patterns, indexing different comprehension mechanisms.
When the picture was visible, analyses of the eye-gaze data revealed distinct gaze pattern for all mismatches compared to matches. By contrast, no action mismatch effect emerged in the blank screen, while role mismatch effects persisted. To the extent that these findings replicate, the presence of actions seems to play an important role in comprehension and locations on a blank screen are - in some cases - used to simulate an actual scene.