Cunillera, T.
University of Barcelona, Spain
In a recent study conducted by Cunillera, Laine, Càmara and Rodríguez-Fornells (2010) we raised the question of how second language learners are able to segment words and map them to a meaning. Can these two processes occur simultaneously? We explored this unresolved issue by using a new multimodal learning paradigm (see Cunillera et al., 2009) that tracked the first steps in learning new words and their mappings to visual referents. It encompassed a continuous audiovisual stream in which transitional probability of syllables was the only acoustic cue available to segment the stream into words, and a visual stream of object images that accompanied the novel words. The object images were systematically varied in terms of constancy of word-picture association and meaningfulness. The results of the experiments indicated good word-referent mapping and word segmentation after short exposure to the audiovisual stream, and suggest that i) mapping words with pictures is more effective when the visual referents are meaningful objects, ii) in word segmentation, the consistency of the word-picture association affects segmentation performance, iii) the effect of associative strength on segmentation performance was most prominent with meaningful objects, and iv) detection of temporal contiguity between multimodal stimuli may be useful in second-language learners not only for facilitating speech segmentation, but also for detecting word-object relationships in natural environments. All in all the present results suggest that word segmentation and word-referent mapping are closely related processes: word segmentation is affected by the consistency of the mapping relationship and both segmentation and mapping can be accomplished under the same short exposure.