Weissbart, H. 1 , Kandylaki, K. 1, 2 & Reichenbach, T. 1
1 Imperial College London
2 University of Maastricht
Spoken language comprehension relies on continuous prediction of the stimuli at different level of processing. Both acoustic properties of speech and linguistic content are predicted by the brain to lead to comprehension. We aimed at disentangling acoustic and linguistic processing of speech by measuring EEG responses to predictability of words in language with continuous cortical recordings. Acoustic information was described through the word onset. Linguistic information was included first as word frequencies, as obtained from Google Ngrams. We then added another feature, the surprisal of a word, that denotes the negative logarithm of the probability of each word conditioned on its preceding context. The surprisal was computed with a deep recurrent neural network language model, trained to predict incoming word given a context. From the neural network output, we also extracted the entropy of word predictions and used it as a third linguistic feature. We then employed linear regression with ridge regularisation to predict the EEG responses from the linguistic and acoustic features. We considered shuffled features, where the word onsets were left unchanged but the values of the linguistic features were taken from an unrelated text, as well as EEG responses to a foreign language as a control.
We found that both the acoustic feature as well as the linguistic features elicited distinct neural responses. In particular, we obtained a specific electrophysiological correlate of the surprisal of a word in its sequence as computed from a deep neural network. This neural response was found in the delta and beta frequency bands and could not be explained by the acoustic properties or by word frequency alone. The cortical response to the surprisal of a word and entropy of predictions supports the predictive coding hypothesis for language processing.