Raghavan, V. S.; Madsen, J.; Nentwich, M.; Leszczynski, M.; Falchier, A.; Bickel, S.; Russ, B. E.; Parra, L. C.

We sample visual scenes with short gaze fixations separated by saccades. While low-level transsaccadic integration is known, semantic integration across multiple fixations remains unclear. We hypothesized that the brain predicts semantic information from one fixation to the next, and therefore postulated a neural signal associated with semantic novelty for each saccade. Novelty was measured using a deep network on foveal vision. Novelty modulated frontal and occipital fixation-related potentials in human EEG during natural viewing of full-length movies (3.4x106 saccades). Intracranial recordings in humans (9.0x104 saccades) and non-human primates (3.3x104 saccades) revealed broadband high-frequency activity modulations in ventromedial/dorsal visual streams and frontal brain areas. This modulation was stronger for movies than static images, and frontal modulation preceded occipital modulation, suggesting top-down effects. This ubiquitous modulation of fixation-related activity with novelty suggests that foveal representations are integrated across saccades in primates.