C02 Masatoshi Yoshida

C02 Understanding active vision from the perspective of active inference　

Our perception of the external world with our eyes is not a passive sensing of the visual scene, like a camera. It is an active construction of the visual scene sampled from the fovea by moving the eyes (saccades). This aspect of vision is called ‘active vision’. According to the Free Energy Principle (FEP)/Active Inference (AIF), what eye movements do is to update beliefs about what is in the external world from sensory information, and not to reconstruct a camera-like image from sensory information. In this respect, FEP/AIF is consistent with the concept of active vision.

According to AIF, the variational free energy G(a) (expected free energy) expected from an action a can be expressed as in the figure. Here, the Bayesian surprise represents how much future beliefs will change if a particular behavioural choice a is made in the future. Then, choosing an action a that maximises the Bayesian surprise maximises the amount of mutual information MI and minimises the expected free energy G(a). In other words, what has conventionally been explained by the visual salience model as ‘the gaze is attracted to salient locations’ can be explained by using Bayesian surprise as ‘the gaze is attracted to more informative locations (in scene understanding)’, thus integrating scene understanding and eye movement.

Therefore, the aim of this study is to test whether ‘Bayesian surprise’ is processed in the brain. To this end, we perform a behavioural task in marmosets and simultaneously measure neural activity from the parietal association cortex PPC and the frontal eye field FEF using Ca imaging. 1) In a visual mismatch task, we will find neural activity related to surprise for different stimulus features. 2) In a free-viewing task, we will build encoding models of visual features, Bayesian surprise and eye movements. 3) Establish a model of active vision using a visual scene recognition task in humans.