Audiovisual Rehabilitation in Hemianopia: A Model-Based Theoretical Investigation

Hemianopic patients exhibit visual detection improvement in the blind field when audiovisual stimuli are given in spatiotemporally coincidence. Beyond this “online” multisensory improvement, there is evidence of long-lasting, “offline” effects induced by audiovisual training: patients show improved visual detection and orientation after they were trained to detect and saccade toward visual targets given in spatiotemporal proximity with auditory stimuli. These effects are ascribed to the Superior Colliculus (SC), which is spared in these patients and plays a pivotal role in audiovisual integration and oculomotor behavior. Recently, we developed a neural network model of audiovisual cortico-collicular loops, including interconnected areas representing the retina, striate and extrastriate visual cortices, auditory cortex, and SC. The network simulated unilateral V1 lesion with possible spared tissue and reproduced “online” effects. Here, we extend the previous network to shed light on circuits, plastic mechanisms, and synaptic reorganization that can mediate the training effects and functionally implement visual rehabilitation. The network is enriched by the oculomotor SC-brainstem route, and Hebbian mechanisms of synaptic plasticity, and is used to test different training paradigms (audiovisual/visual stimulation in eye-movements/fixed-eyes condition) on simulated patients. Results predict different training effects and associate them to synaptic changes in specific circuits. Thanks to the SC multisensory enhancement, the audiovisual training is able to effectively strengthen the retina-SC route, which in turn can foster reinforcement of the SC-brainstem route (this occurs only in eye-movements condition) and reinforcement of the SC-extrastriate route (this occurs in presence of survived V1 tissue, regardless of eye condition). The retina-SC-brainstem circuit may mediate compensatory effects: the model assumes that reinforcement of this circuit can translate visual stimuli into short-latency saccades, possibly moving the stimuli into visual detection regions. The retina-SC-extrastriate circuit is related to restitutive effects: visual stimuli can directly elicit visual detection with no need for eye movements. Model predictions and assumptions are critically discussed in view of existing behavioral and neurophysiological data, forecasting that other oculomotor compensatory mechanisms, beyond short-latency saccades, are likely involved, and stimulating future experimental and theoretical investigations.


APPENDIX: Equations describing the sensory module
Each area is made up of N = 181 neurons, having preferred retinal location at a distance of 1° from each other, from -90° to + 90°, 0° representing the current gaze position (i.e. the position of the fovea).
In the following, each neuron is labelled by a subscript i representing its preferred retinal position (i = -90°, -89°, …-1°, 0°, +1°, …, +89°, +90°) and by a superscript H denoting its area (H = R, V1, E, A, SC). An external stimulus applied at spatial position p in head-centered coordinates, at a given time t, will match the preferred location for neuron i = p-g(t) in the areas.
In order to avoid edge effects, each area in the network is considered to have a circular structure, so that the first and last neurons are virtually linked. This ensures that all neurons in each area behave in the same way regardless they are located at the border or at the center.
We use rate-coding models of neurons, which are defined via the following equations: . 2 ( ) denotes the input to the generic neuron i in area H, and ( ) denotes neuron's output or neuron's activity (representing neuron's firing rate), computed from its input via a sigmoidal activation function (Eq. S2) and a first order dynamics (Eq. S1) with time constant . According to Eq. S2, maximum neuron's activity is normalized to 1.
The input ( ) to a neuron can be generally written as the sum of three contributions: The term ( ) in Eq. S3, represents the external input due to the external stimulus, and it is different from 0 only in areas R and A that directly receive the stimulus. The external input in each modality is mimicked as a Gaussian function of the distance between the stimulus position (in eyecentered coordinates) and neurons' preferred position. By denoting with D the duration of the external stimulus (applied at t = 0) and with p its position (in head-centered coordinates), we have: . are Gaussian noises with 0 mean and standard deviation equal to 10% of the stimulus strength.
The term ( ) in Eq. S3 represents the lateral input each neuron receives from other neurons in the same area via lateral synapses. The lateral synapses are arranged according to a "Mexican Hat" disposition, obtained as the difference between an excitatory and inhibitory contribution, each mimicked as a Gaussian function of the distance between neurons' preferred positions. Hence, in . 9 is the weight of the lateral synapse from the pre-synaptic neuron at position k inside area H to the post-synaptic neuron at position i inside the same area H The distance among neurons within the same area H ( . 10) is computed accounting for the circular structure of each area.
The term ( ) in Eq. S3 represents the input a neuron in area H may receive from neurons in other areas via inter-area excitatory synapses. In their basal configuration (i.e. before training), they have a Gaussian shape, their strength depending on the distance between preferred retinal positions of neurons in the two connected areas. We have: , in Eq. S11 is the weight of the inter-area synapse from the pre-synaptic neuron at position j in area Q to the post-synaptic neuron at position i in area H, and the first sum extends to all areas Q in the network sending projections to area H. Since area R does not receive synapses from other areas, ( ) = 0, ∀ . , and , in Eq S.12 are the central weight and width of the Gaussian function, and the computation of the distance , among neurons' positions in different areas in eq. S13 accounts for the circular structure.

Figure S1
Figure S1: Pattern of all inter-area synapses within the sensory module at the end of training A for patient # 9 (the same as in Figure 6). In each color map, x-axis denotes the position (j) of the presynaptic neuron in area Q, y-axis denote the position (i) of the post-synaptic neuron in area H, and the color value at the intersection (j,i) indicates the strength of the synapse , . In each color map, scale color ranges between 0 and the maximum value , (Table 1) for the represented synapses. , is the central weight of the pre-training Gaussian pattern.  Figure 8A). (B) Pattern of all inter-area synapses within the sensory module at the end of training A for patient # 19 (the same as in Figure 8B). The meaning is the same as in Figure S1.   Figure 12A). (B) Pattern of all inter-area synapses within the sensory module at the end of training B for patient # 19 (the same as in Figure 12B). The meaning is the same as in Figure S1.   Figure 12C). (B) Pattern of all inter-area synapses within the sensory module at the end of training C for patient # 19 (the same as in Figure 12D). The meaning is the same as in Figure S1.   Table S1. Patients are characterized by islands of low or intact vision within the blind hemifield, formed by clusters of close or contiguous spared V1 neurons, rather than having scattered spared V1 neurons as in patients reported in Table 2   A B

A B
C D E F of oculomotor response (the latter include also detections due to islands of intact vision). Saccademediated detections remain a limited fraction of the triggered saccades. The reasons for the limited advantage of eye movements in post-training performances are due to the strict requirements that the visual stimulus must meet for being detected via a saccade (as in the set of patients examined in the main text): i) The visual stimulus must be applied at a position where the synapses are strongly reinforced (e.g. at or very close to the positions stimulated during training) and the noise superimposed over the external stimulus should favor high activation in the retinal area (and in SC), so that the saccade is triggered; ii) The visual stimulus triggering the saccade must be close enough (within 8° distance from a detection region); iii) The saccade must be triggered early enough so to move the stimulus into the detection region before its removal. Let's consider, for example, patient #5 in Table S1: he has islands of intact vision in the ranges 1°-5° and 42°-48° and region of low vision in the range 6°-16°. After training, he regains visual restitution at position 8° and 40°. Undetected visual stimuli at position 56° or 54° (where synapses had strongly reinforced) can trigger saccades, but only rarely they reach the intact region at 42°-48°, so they mainly remain undetected. A similar effect occurs for stimuli at position 20°-24°; they may occasionally trigger saccades (especially at position 22°, 24°) but only exceptionally elicit detection reaching the low vision region at 6°-16°. (E) and (F) Visual detection accuracy (%,) averaged (mean ± SEM) on the 5 simulated patients in Visual Test 1 and Visual Test 2 in both eye conditions, before training and after training A. The visual detection gain acquired via the training ( = post -pre visual detection) is displayed too and compared with visual detection gain drawn from in vivo studies (Bolognini et al., 2005;Tinelli et al. 2015). In both tests, the model underestimates the visual detection gain in Eye-Movements condition, coherently with results obtained in the other set of patients (main text), suggesting that other oculomotor mechanisms, beyond the execution of short-latency saccades, may contribute to the improvement observed in vivo.