MINI REVIEW article
Sec. Cellular Neurophysiology
Volume 16 - 2022 | https://doi.org/10.3389/fncel.2022.911754
Artificial Visual Information Produced by Retinal Prostheses
- 1Brain Science Institute, Korea Institute of Science and Technology, Seoul, South Korea
- 2School of Electrical Engineering, College of Engineering, Korea University, Seoul, South Korea
- 3Division of Bio-Medical Science & Technology, KIST School, University of Science and Technology, Seoul, South Korea
Numerous retinal prosthetic systems have demonstrated somewhat useful vision can be restored to individuals who had lost their sight due to outer retinal degenerative diseases. Earlier prosthetic studies have mostly focused on the confinement of electrical stimulation for improved spatial resolution and/or the biased stimulation of specific retinal ganglion cell (RGC) types for selective activation of retinal ON/OFF pathway for enhanced visual percepts. To better replicate normal vision, it would be also crucial to consider information transmission by spiking activities arising in the RGC population since an incredible amount of visual information is transferred from the eye to the brain. In previous studies, however, it has not been well explored how much artificial visual information is created in response to electrical stimuli delivered by microelectrodes. In the present work, we discuss the importance of the neural information for high-quality artificial vision. First, we summarize the previous literatures which have computed information transmission rates from spiking activities of RGCs in response to visual stimuli. Second, we exemplify a couple of studies which computed the neural information from electrically evoked responses. Third, we briefly introduce how information rates can be computed in the representative two ways – direct method and reconstruction method. Fourth, we introduce in silico approaches modeling artificial retinal neural networks to explore the relationship between amount of information and the spiking patterns. Lastly, we conclude our review with clinical implications to emphasize the necessity of considering visual information transmission for further improvement of retinal prosthetics.
Vision is unarguably the most critical sensory modality (Hutmacher, 2019) among the five senses of humans. Diverse causes can result in low vision or blindness, and there have been numerous attempts to restore sight to blind individuals by electrically stimulating visual cortex (Dobelle et al., 1974, 1976), lateral geniculate nucleus (Pezaris and Reid, 2007), optic nerve (Gaillet et al., 2019), or retina (Humayun et al., 1996). In the case of retina, the electric stimulation is effective to elicit artificial visual percepts for outer retinal degenerative diseases such as retinitis pigmentosa and age-related macular degeneration, which primarily damage photoreceptors (Bunker et al., 1984; Curcio et al., 1996). Microelectronic retinal prostheses including commercialized ones (e.g., Argus II, Alpha-IMS/AMS, and PRIMA) reported some promising clinical outcomes by electrically stimulating the remaining inner retinal neurons (Humayun et al., 1996; Rizzo et al., 2003; Fujikado et al., 2007; Zrenner et al., 2011; da Cruz et al., 2013; Dorn et al., 2013; Stingl et al., 2013; Shivdasani et al., 2017; Palanker et al., 2020). However, the best visual acuity of elicited artificial vision (20/460) (Palanker et al., 2020) is still far below the level of legal blindness (20/200) as well as normal vision (20/20). To achieve high-resolution visual prosthetics, various research groups have tried to electrically activate cells in only targeted small areas using hardware and software approaches. For example, several groups proposed novel designs of microelectrodes (Flores et al., 2018, 2019; Seo et al., 2020) or photovoltaic arrays (Ferlauto et al., 2018; Wang et al., 2021) to further localize electric current in a smaller region. Also, Jepson et al. (2014) used spatially patterned electric stimulation, and Weitz et al. (2015) demonstrated retinal activation in much smaller area with 25 ms-long pulses.
The other important research topic in retinal prosthetics has long been the cell-type specific stimulation. As the starting point of the most complex sensory system, and the mammalian retinas have numerous types of retinal ganglion cells (RGCs) which are the output spiking neurons sending neural signals to the downstream visual centers (Masland, 2001; Rockhill et al., 2004; Sanes and Masland, 2015; Baden et al., 2016). Among those types, ON and OFF RGCs are known to play a critical role in forming visual percepts (Schiller et al., 1986; Schiller, 1992). In addition to the asymmetricities between light-evoked responses of the ON vs. the OFF pathways (Ölveczky et al., 2003; Margolis and Detwiler, 2007; Liang and Freed, 2012; Freed, 2017), retinal prosthetic studies reported contrasting differences between the two pathways (Freeman et al., 2010; Kameneva et al., 2010; Twyford et al., 2014; Im and Fried, 2015, 2016a; Lee and Im, 2019). However, given the unique mosaic arrangement of each type of RGCs (DeVries and Baylor, 1997; Masland, 2012), it seems almost inevitable to activate every type of RGCs located near a given electrode delivering electric stimulation. Nevertheless, it seems reasonable to aim more biased activation of the ON system because the earlier clinical trials reported dominantly bright sensation (Humayun et al., 1996, 2003; Fujikado et al., 2007; Naycheva et al., 2012). Recent studies demonstrated the ON/OFF response ratio can be increased by modulating several stimulus parameters such as stimulus durations (Im et al., 2018), repetition rates (Cai et al., 2011, 2013; Twyford et al., 2014; Im and Fried, 2016a), waveform shapes (Lee and Im, 2018), and current amplitudes (Lee and Im, 2019). The use of penetrating microelectrode to the specific stratification depth may enhance the cell-type specific activation of either ON or OFF RGCs (Roh et al., 2022). To date, however, it appears extremely challenging to achieve completely selective stimulation of targeted cell type(s) using electrical stimulation. In contrast, optogenetic approaches can selectively activate the ON pathway (Lagali et al., 2008; Gaub et al., 2014; Macé et al., 2015; Lu et al., 2016) but they need to address potential phototoxicity (Grimm et al., 2001; van Wyk et al., 2015; Simunovic et al., 2019) and/or low transfection efficiency of viral vectors (Lagali et al., 2008; Busskamp and Roska, 2011).
In addition to the aforementioned two important features (i.e., the spatial resolution and the cell type-specific stimulation), it may be critical to assess whether retinal prostheses restore ample enough information. It is because the remarkably complex retinal circuits compress the visual world in real-time (Kolb, 2003), making RGCs transmit an incredible amount of visual information to the brain (Figure 1A), which could be estimated as big as 875,000 bits/s (52.5 Mb/min) (Koch et al., 2006). Thus, less artificial information may confound the brain, resulting in unclear artificial visual percepts (Figure 1B), whereas more transferred information may help the brain more precisely recognize artificial visual images (Figure 1C). However, there has been a limited number of retinal prosthetic researches which had studied neural information aspects of electrically evoked spiking activities (Eger et al., 2005; Kang et al., 2021). Contrastingly, in the case of cochlear implants which show great clinical success, information transmission in the auditory system has been well studied (Mino, 2007; Zeng et al., 2008; Hannan et al., 2012; Moroz et al., 2012; Gao et al., 2013). Analyzing the RGC spiking activities using information theory is likely to help understanding how the brain deciphers artificial visual information (Quian Quiroga and Panzeri, 2009). In particular, it has been known that retinal spike trains are precisely structured to efficiently convey visual information (Berry et al., 1997). Therefore, other than simply re-activating RGCs to send spike trains to downstream neurons, it would be essential to understand how much information is encoded (Timme and Lapish, 2018) by the ensemble of prosthetically evoked spikes arising in RGCs (Freeman et al., 2011) to estimate the quality of artificial vision. Also, information researches are likely to offer valuable insights for the improved performance of retinal implants.
Figure 1. Schematic illustration of visual information transfer from the eye to the visual cortex. The visual information flows through the lateral geniculate nucleus (LGN) en route to the visual cortex. But, LGN is not shown in this figure for brevity. (A) Retinal ganglion cells generate spiking activities to visual stimuli and transmit visual information to the brain (visual cortex) through the optic nerve and optic radiation. (B) Insufficient visual information and less natural artificial vision may activate higher visual centers inappropriately. (C) Rich visual information and more natural artificial vision may activate higher visual centers more effectively.
Neural Information of Intrinsic Visual Responses in the Healthy Retina
The retina divides the complex visual world into several parallel pathways (Wässle, 2004; Roska et al., 2006; Nassi and Callaway, 2009) using the remarkable variety of RGC types (Baden et al., 2016). All RGCs in mammalian retinas encode visual information by spatiotemporal structure of spike trains but differently across the RGC types (Berry et al., 1997; Masland, 2001; Field and Chichilnisky, 2007; Zeck and Masland, 2007; Sanes and Masland, 2015). Much work has studied the relationship between spiking and information rates in RGC responses to light stimuli using several computational methods (Koch et al., 2004, 2006; Passaglia and Troy, 2004; Kang et al., 2021) (see a later section regarding how direct and reconstruction methods compute neural information). Particularly, it is notable that distinct RGC types showed different information rates (Koch et al., 2006). For example, in guinea pig retinas, two groups of RGCs with brisk or sluggish responses transmitted information of 21 ± 9 and 13 ± 7 bits/s (n = 19 and n = 23 cells), respectively (Koch et al., 2004). More recently, Kang et al. (2021) analyzed the amount of neural information conveyed through brisk transient (BT) and brisk sustained (BS) subtypes of ON and OFF pathways in rabbit retinas. In responses to spot flashes, 1.83 ± 0.07 and 1.89 ± 0.04 bits/s of information were transmitted by ON BT and OFF BT cells, respectively; while 2.53 ± 0.08 and 3.00 ± 0.22 bits/s of information were transmitted by ON BS and OFF BS cells (n = 15 for each type), respectively.
Compared to laboratory stimuli such as stationary spot flashes, natural stimuli are known to evoke sparser spiking activities in RGCs (Kayser et al., 2003; David et al., 2004; Felsen et al., 2005; Puchalla et al., 2005; Touryan et al., 2005; Talebi and Baker, 2012; Im and Fried, 2016b). For example, about 65% of RGCs do not fire constantly in response to natural stimuli, and it has been argued that the sparse coding can reduce the number of activated neurons, thereby saving energy for information transmission (Wang et al., 2018). Given that natural stimuli contain much bigger visual information than laboratory stimuli, it seems important to compare both amount and efficiency of information conveyed in responses arising from laboratory vs. natural stimuli.
Artificial Visual Information of Electrically Evoked Responses in the Degenerate Retina
Aforementioned examples suggest that, for high-quality artificial vision, it may be crucial to study whether electrically elicited spiking activities of RGCs in the degenerate retina convey visual information at a similar level of visually evoked responses arising in the normal retina (Fried et al., 2006; Freeman et al., 2011). Surprisingly, however, there are very few studies that have discussed the neural information produced by electric stimulation. For example, Eger et al. (2005) stimulated the cat retina electrically while recording neuronal activities at 15 sites in the visual cortex. They estimated 20–160 bits/s of information was transferred at a single recording site when a single electrode was activated (Eger et al., 2005). They also reported 500 bits/s of information was transmitted at 15 recording sites when seven electrodes were activated. However, the information rates varied noticeably between experiments depending on the positions of stimulation and recording sites since it was difficult to place electrodes accurately at corresponding retinotopic loci.
Another example investigated the amount of information elicited by both light and electric stimulation in more sophisticated ways (Kang et al., 2021). First, they classified RGCs into the four major types (i.e., BT and BS subtypes of ON and OFF pathways) in the healthy rabbit retina, and then analyzed spiking activities in each type to compare the neural information between light and electric responses of identical sets of RGCs. When the number of cells increased up to 15, electric responses of ON BT and BS RGCs displayed a similar level of neural information with their light responses, whereas electrically evoked responses of OFF BT and BS cells showed greatly reduced information than those of their light responses. Second, they tried to correlate the neural information and the cell-to-cell heterogeneity of spiking responses. Interestingly, the ON RGCs showed similarly heterogeneous responses regardless of light and electric stimuli. In contrast, the OFF RGCs showed much more homogeneous responses to electric than light stimuli. The reduced information by the homogeneous responses of OFF RGCs is consistent with previous studies which reported the amount of neural information increases with a higher cell-to-cell heterogeneity in naturally evoked neural activities (Chelaru and Dragoi, 2008; Padmanabhan and Urban, 2010). However, it is also noteworthy that there is an optimal level of population response heterogeneity which maximizes the transmission of neural information as well as minimizes the effect of external noises (Tkačik et al., 2010; Tripathy et al., 2013; Im and Kim, 2020). Kang et al. (2021) also demonstrated the ultimate heterogeneity (i.e., completely random spiking) of RGC responses is not ideal to transfer neural information because they are less immune to noise: they showed the increased trial-to-trial variability (i.e., jitter) reduces the population neural information more substantially in random spiking responses than in physiological RGC responses which showed intermediate levels of the cell-to-cell heterogeneity. Given an earlier study that reported RGC spiking consistency is gradually reduced with advancing retinal degeneration (Yoon et al., 2020), the analyses of Kang et al. (2021) suggest that the amount of information transmitted to the brain decreases as the retina degenerates. For further improvement of retinal prosthetics, it may be critical to investigate how information rates can be enhanced in degenerate retinas by altering electric stimulation conditions.
Two Methods for Neural Information Calculation
Information theory can quantify how much information about a given external stimulus is conveyed by neural responses; and there are two representative ways to compute the information from neural spike trains (Borst and Theunissen, 1999; Passaglia and Troy, 2004). First, direct method can be applied to calculate average information transmitted by the difference between total entropy of the neural response and noise entropy (Figure 2A; Koch et al., 2004; Osborne et al., 2008; Stone, 2018; Kang et al., 2021). In this method, before calculating the entropy, spikes of each cell are allocated into time bins in a fixed duration which may differ depending on experimental methods (Koch et al., 2004). Then, if one or more spikes are present in a given time bin, 1 is assigned; while 0 is assigned if there are no spikes (Osborne et al., 2008; Kang et al., 2021). Before using this binary code array to calculate entropy, it is important to choose an appropriate length of binary code combinations depending on the particular context being experimented (Theunissen and Miller, 1995). Then, total entropy is calculated from the probability of particular binary code combinations in the entire recording. Similarly, noise entropy is also estimated but from the probability of particular binary code combinations at a specific given time relative to the identical stimuli (see Osborne et al., 2008; Stone, 2018 for more details).
Figure 2. The direct method and the reconstruction method can be applied to calculate information rates. (A) In the direct method, average information rates are the difference between total entropy and noise entropy. N represents the total number of possible binary code combinations, i represents binary code combination. Pi indicates probability of particular binary code combinations and similarly Pi,t indicates the probability of particular binary code combinations at a specific time, t. (B) In the reconstruction method, average information rates are obtained from the signal to noise ratio (SNR). Signal and noise are calculated differently in each bound (see Passaglia and Troy, 2004 for how signal and noise are calculated). In here, S(f) means signals which are the Fourier transforms of the stimulus, R(f) means responses which are also the Fourier transforms of the response, respectively. Ŝ(f) means the best estimate of stimulus. In upper bound, Ŝ(f) is obtained by averaging R(f) [i.e,. R̄(f)]. In lower bound, Ŝ(f) is obtained by the linear decoder filter, Hs(f). N(f) represents noise, and noise is also different in each bound. In upper bound, noise is the difference between response and average response, while noise is the difference between signals and estimated stimulus in lower bound.
Second, reconstruction method can be used which is a variant of the abovementioned direct method (Figure 2B; Borst and Theunissen, 1999). The reconstruction method can determine the neural responses based on the stimulus or predict what stimulus is given based on the response (Eger and Eckhorn, 2002; Victor, 2006). Unlike the direct method requiring no assumption, the reconstruction method estimates upper or lower bound of information depending on assumptions. When we assume neural responses have a Gaussian distribution over the frequency range then the upper bound is placed since Gaussian distribution has the maximum entropy (Borst and Theunissen, 1999; Passaglia and Troy, 2004). Alternatively, the lower bound is placed when we assume information can be decoded linearly to estimate the best possible stimulus from the neural responses (Borst and Theunissen, 1999). Because we cannot include all of the information with this assumption since neural responses are predominantly non-linear and Poisson process (Felsen and Dan, 2005), it becomes the lower bound of the information. For the computation of information rates using the reconstruction method, the signal to noise ratio in the frequency domain must be calculated (see Passaglia and Troy, 2004 for more details).
In comparison between these two methods, the direct method needs a lot of experimental data to calculate since it has no assumptions with signals and it does not reveal which stimulus aspects are being encoded. In sharp contrast, the reconstruction method needs significantly less data than the direct method, making it useful for the field that has limited amount of available data.
Recent In Silico Computational Neuroscience Approaches for Neural Information Analysis
As an alternative to population responses recorded in vivo or in vitro, we can simulate population codes using computational tools. A few preceding studies suggest that the artificial retina model helps understand how the retina responds to stimuli and how the retina encodes visual information (Wohrer et al., 2006; Pei and Qiao, 2010). Also, Brette (2009) introduced an in silico approach to generate population codes based on the designed firing rate and pairwise correlation of spike trains. With the computational approaches using artificial spike trains, it is possible to more precisely analyze the relationship between the spiking elements and neural information.
Recently, Roh et al. (2021) used a modified version of “Brian 2” (Brette, 2009) to generate various sets of spike trains which have different levels of correlations; spike time tiling coefficients (STTCs) were calculated to quantify the correlations across spike trains (Cutts and Eglen, 2014). Then, they analyzed neural information as a function of the correlation level for a wide range of average STTC values. According to the abovementioned study of Roh et al., the increased spiking heterogeneity across cells can enhance information transmission. Earlier, Hunsberger et al. (2014) reported the heterogeneity may better encode the stimulation by expressing complementary aspects of stimuli. In addition to the cell-to-cell spiking heterogeneity, other spiking features may be also crucial in better encoding visual information. For example, as a follow-up study of Roh et al. (2021) and Kim et al. (2022) further explored the relationship between information and other spiking elements such as mean firing rate and spiking duration. These in silico approaches may expedite future studies regarding how different stimulation parameters of retinal prostheses make RGCs transmit sufficient information to the brain.
Clinical Implications of Artificially Evoked Visual Neural Information
Other than the aforementioned studies (Eger et al., 2005; Kang et al., 2021; Roh et al., 2021; Kim et al., 2022), retinal prosthetics has not paid enough attention on how their microelectronic devices and/or stimulation strategies improve electrically produced neural information. Although there is no direct clinical evidence supporting that prosthetic responses which transmit more information would be better perceived, it is important to note a recent sight restoration study demonstrated better animal behavior responses when the restored spiking activities were more heterogeneous across RGCs (Berry et al., 2017). Given the correlation between the cell-to-cell spiking heterogeneity and the transmitted neural information (Tripathy et al., 2013; Kang et al., 2021), the improved behavior of the animals is likely due to the enhanced visual information transmission. The optical stimulation using optogenetic approaches (Bi et al., 2006; Sahel et al., 2021), photoswitch compound (Tochitsky et al., 2016), and photoactivatable G protein-coupled receptor (Berry et al., 2017) must be promising vision restoration methods because they would not require any surgical implantation of microelectronic devices. However, in those fields as well, it is difficult to find analyses of artificially evoked neural information.
In the past clinical trials of microelectronic retinal prostheses, the most prosthetic users with retinitis pigmentosa were at the near-end stage of degeneration, who showed no light perception or hand motion (Humayun et al., 2003; Zrenner et al., 2011; Stingl et al., 2013; but, see Palanker et al., 2020, 2022 for prosthetic users with age-related macular degeneration who had still periphery vision). In such an advanced stage, their retinas were likely to send less neural information to the brain in response to electric stimulation. It is because both direct and indirect activation of RGCs are likely to result in low information transmission rates as follow: (1) direct activation which can precisely elicit spike at intended timing may produce too homogeneous spiking across RGCs compared to natural spiking activity, and (2) indirect activation generates highly inconsistent (i.e., big trial-to-trial variability in each RGCs) network-mediated responses in severely degenerate retinas (Yoon et al., 2020), increasing noise which reduces information transmission (Kang et al., 2021). Numerous previous literatures have studied synchronous/correlated spiking activities of neighboring RGCs in responses to visual stimuli (Meister et al., 1995; DeVries, 1999; Puchalla et al., 2005; Shlens et al., 2008). Depending on the extent of response synchrony/correlation of neighboring RGCs, the brain might recognize different visual messages as argued earlier (Puchalla et al., 2005). However, systematic understanding is still lacking regarding electrically evoked responses, raising the need for such measurements.
The retina is remarkably complicated in both anatomic and functional aspects. Given the incredible complexity of the retina, it seems insufficient to simply make RGCs fire again for high-quality artificial vision. However, the field of retinal prosthetics has paid little attention on how much artificial visual information could be created by implanted retinal prostheses and transmitted from the retina to the brain, compared to other aspects of electrically evoked retinal responses such as spike counts, firing rates, and so on. For the further improved quality of prosthetic vision, it may be critical to explore whether sufficient amount of visual information is transmitted from the retina to the brain. Probably, more information enhances the perception quality of artificial vision.
SK and HR contributed to drafting the manuscript and figures design. MI conceived the work and revised the manuscript. All authors read and approved the final manuscript.
This work was supported by the KIST (Korea Institute of Science and Technology) institutional grants (2E31821) and National R&D Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT (Nos. 2020R1C1C1006065, 2021M3F3A2A01037366, and 2022M3E5E8017395).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
We thank Joon Ho Kang and Taegon Kim for the helpful discussion on the neural information of retinal ganglion cell spiking activities.
Berry, M. H., Holt, A., Levitz, J., Broichhagen, J., Gaub, B. M., Visel, M., et al. (2017). Restoration of patterned vision with an engineered photoactivatable G protein-coupled receptor. Nat. Commun. 8:1862. doi: 10.1038/s41467-017-01990-7
Bi, A., Cui, J., Ma, Y. P., Olshevskaya, E., Pu, M., Dizhoor, A. M., et al. (2006). Ectopic expression of a microbial-type rhodopsin restores visual responses in mice with photoreceptor degeneration. Neuron 50, 23–33. doi: 10.1016/j.neuron.2006.02.026
Cai, C., Ren, Q., Desai, N. J., Rizzo, J. F. 3rd., and Fried, S. I. (2011). Response variability to high rates of electric stimulation in retinal ganglion cells. J. Neurophysiol. 106, 153–162. doi: 10.1152/jn.00956.2010
Cutts, C. S., and Eglen, S. J. (2014). Detecting pairwise correlations in spike trains: an objective comparison of methods and application to the study of retinal waves. J. Neurosci. 34, 14288–14303. doi: 10.1523/JNEUROSCI.2767-14.2014
da Cruz, L., Coley, B. F., Dorn, J., Merlini, F., Filley, E., Christopher, P., et al. (2013). The Argus II epiretinal prosthesis system allows letter and word reading and long-term function in patients with profound vision loss. Br. J. Ophthalmol. 97, 632–636. doi: 10.1136/bjophthalmol-2012-301525
Dobelle, W. H., Mladejovsky, M. G., Evans, J. R., Roberts, T. S., and Girvin, J. P. (1976). “Braille” reading by a blind volunteer by visual cortex stimulation. Nature 259, 111–112. doi: 10.1038/259111a0
Dobelle, W. H., Mladejovsky, M. G., and Girvin, J. P. (1974). Artifical vision for the blind: electrical stimulation of visual cortex offers hope for a functional prosthesis. Science 183, 440–444. doi: 10.1126/science.183.4123.440
Dorn, J. D., Ahuja, A. K., Caspi, A., da Cruz, L., Dagnelie, G., Sahel, J. A., et al. (2013). The detection of motion by blind subjects with the epiretinal 60-electrode (Argus II) retinal prosthesis. JAMA Ophthalmol. 131, 183–189. doi: 10.1001/2013.jamaophthalmol.221
Eger, M., and Eckhorn, R. (2002). A model-based approach for the analysis of neuronal information transmission in multi-input and -output systems. J. Comput. Neurosci. 12, 175–200. doi: 10.1023/a:1016583328930
Eger, M., Wilms, M., Eckhorn, R., Schanze, T., and Hesse, L. (2005). Retino-cortical information transmission achievable with a retina implant. Biosystems 79, 133–142. doi: 10.1016/j.biosystems.2004.09.030
Ferlauto, L., Airaghi Leccardi, M. J. I., Chenais, N. A. L., Gilliéron, S. C. A., Vagni, P., Bevilacqua, M., et al. (2018). Design and validation of a foldable and photovoltaic wide-field epiretinal prosthesis. Nat, Commun. 9:992. doi: 10.1038/s41467-018-03386-7
Flores, T., Huang, T., Bhuckory, M., Ho, E., Chen, Z., Dalal, R., et al. (2019). Honeycomb-shaped electro-neural interface enables cellular-scale pixels in subretinal prosthesis. Sci. Rep. 9:10657. doi: 10.1038/s41598-019-47082-y
Flores, T., Lei, X., Huang, T., Lorach, H., Dalal, R., Galambos, L., et al. (2018). Optimization of pillar electrodes in subretinal prosthesis for enhanced proximity to target neurons. J. Neural Eng. 15:036011. doi: 10.1088/1741-2552/aaac39
Freeman, D. K., Eddington, D. K., Rizzo, J. F. 3rd., and Fried, S. I. (2010). Selective activation of neuronal targets with sinusoidal electric stimulation. J. Neurophysiol. 104, 2778–2791. doi: 10.1152/jn.00551.2010
Freeman, D. K., Rizzo, J. F. 3rd., and Fried, S. I. (2011). Encoding visual information in retinal ganglion cells with prosthetic stimulation. J. Neural Eng. 8:035005. doi: 10.1088/1741-2560/8/3/035005
Fried, S. I., Hsueh, H. A., and Werblin, F. S. (2006). A method for generating precise temporal patterns of retinal spiking using prosthetic stimulation. J. Neurophysiol. 95, 970–978. doi: 10.1152/jn.00849.2005
Fujikado, T., Morimoto, T., Kanda, H., Kusaka, S., Nakauchi, K., Ozawa, M., et al. (2007). Evaluation of phosphenes elicited by extraocular stimulation in normals and by suprachoroidal-transretinal stimulation in patients with retinitis pigmentosa. Graefes Arch. Clin. Exp. Ophthalmol. 245, 1411–1419. doi: 10.1007/s00417-007-0563-z
Gaillet, V., Cutrone, A., Artoni, F., Vagni, P., Mega Pratiwi, A., Romero, S. A., et al. (2019). Spatially selective activation of the visual cortex via intraneural stimulation of the optic nerve. Nat. Biomed. Eng. 4, 181–194. doi: 10.1038/s41551-019-0446-8
Gao, X., Grayden, D. B., and McDonnell, M. D. (2013). Information theoretic optimization of cochlear implant electrode usage probabilities. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2013, 5974–5977. doi: 10.1109/EMBC.2013.6610913
Gaub, B. M., Berry, M. H., Holt, A. E., Reiner, A., Kienzler, M. A., Dolgova, N., et al. (2014). Restoration of visual function by expression of a light-gated mammalian ion channel in retinal ganglion cells or ON-bipolar cells. Proc. Natl. Acad. Sci. U.S.A. 111, E5574–E5583. doi: 10.1073/pnas.1414162111
Grimm, C., Wenzel, A., Williams, T., Rol, P., Hafezi, F., and Remé, C. (2001). Rhodopsin-mediated blue-light damage to the rat retina: effect of photoreversal of bleaching. Invest. Ophthalmol. Vis. Sci. 42, 497–505.
Humayun, M. S., De Juan, E., Dagnelie, G., Greenberg, R. J., Propst, R. H., and Phillips, D. H. (1996). Visual perception elicited by electrical stimulation of retina in blind humans. Arch. Ophthalmol. 114, 40–46. doi: 10.1001/archopht.1996.01100130038006
Humayun, M. S., Weiland, J. D., Fujii, G. Y., Greenberg, R., Williamson, R., Little, J., et al. (2003). Visual perception in a blind subject with a chronic microelectronic retinal prosthesis. Vision Res. 43, 2573–2581. doi: 10.1016/s0042-6989(03)00457-7
Im, M., and Fried, S. I. (2015). Indirect activation elicits strong correlations between light and electrical responses in ON but not OFF retinal ganglion cells. J. Physiol. 593, 3577–3596. doi: 10.1113/JP270606
Im, M., and Fried, S. I. (2016a). Temporal properties of network-mediated responses to repetitive stimuli are dependent upon retinal ganglion cell type. J. Neural Eng. 13:025002. doi: 10.1088/1741-2560/13/2/025002
Im, M., Werginz, P., and Fried, S. I. (2018). Electric stimulus duration alters network-mediated responses depending on retinal ganglion cell type. J. Neural Eng. 15:036010. doi: 10.1088/1741-2552/aaadc1
Jepson, L. H., Hottowy, P., Mathieson, K., Gunning, D. E., Da̧browski, W., Litke, A. M., et al. (2014). Spatially patterned electrical stimulation to enhance resolution of retinal prostheses. J. Neurosci. 34, 4871–4881. doi: 10.1523/JNEUROSCI.2882-13.2014
Kameneva, T., Meffin, H., and Burkitt, A. N. (2010). Differential stimulation of ON and OFF retinal ganglion cells: a modeling study. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2010, 4246–4249. doi: 10.1109/IEMBS.2010.5627176
Kang, J. H., Jang, Y. J., Kim, T., Lee, B. C., Lee, S. H., and Im, M. (2021). Electric stimulation elicits heterogeneous responses in ON but not OFF retinal ganglion sells to transmit rich neural information. IEEE Trans. Neural Syst. Rehabil. Eng. 29, 300–309. doi: 10.1109/TNSRE.2020.3048973
Kim, S., Roh, H., and Im, M. (2022). Neural Information of Artificial Vision Varies Depending on Mean Firing Rate and Spiking Duration. Accepted to be Presented at ARVO 2022. Rockville, MD: Association for Research in Vision and Ophthalmology.
Koch, K., McLean, J., Berry, M., Sterling, P., Balasubramanian, V., and Freed, M. A. (2004). Efficiency of information transmission by retinal ganglion cells. Curr. Biol. 14, 1523–1530. doi: 10.1016/j.cub.2004.08.060
Lagali, P. S., Balya, D., Awatramani, G. B., Münch, T. A., Kim, D. S., Busskamp, V., et al. (2008). Light-activated channels targeted to ON bipolar cells restore visual function in retinal degeneration. Nat. Neurosci. 11, 667–675. doi: 10.1038/nn.2117
Lee, J.-I., and Im, M. (2018). Non-rectangular waveforms are more charge-efficient than rectangular one in eliciting network-mediated responses of ON type retinal ganglion cells. J. Neural Eng. 15:055004. doi: 10.1088/1741-2552/aad416
Lee, J.-I., and Im, M. (2019). Optimal electric stimulus amplitude improves the selectivity between responses of ON versus OFF types of retinal ganglion cells. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 2015–2024. doi: 10.1109/TNSRE.2019.2939012
Liang, Z., and Freed, M. A. (2012). Cross inhibition from ON to OFF pathway improves the efficiency of contrast encoding in the mammalian retina. J. Neurophysiol. 108, 2679–2688. doi: 10.1152/jn.00589.2012
Lu, Q., Ganjawala, T. H., Ivanova, E., Cheng, J. G., Troilo, D., and Pan, Z. H. (2016). AAV-mediated transduction and targeting of retinal bipolar cells with improved mGluR6 promoters in rodents and primates. Gene Ther. 23, 680–689. doi: 10.1038/gt.2016.42
Macé, E., Caplette, R., Marre, O., Sengupta, A., Chaffiol, A., Barbe, P., et al. (2015). Targeting channelrhodopsin-2 to ON-bipolar cells with vitreally administered AAV Restores ON and OFF visual responses in blind mice. Mol. Ther. 23, 7–16. doi: 10.1038/mt.2014.154
Mino, H. (2007). Encoding of information into neural spike trains in an auditory nerve fiber model with electric stimuli in the presence of a pseudospontaneous activity. IEEE Trans. Biomed. Eng. 54, 360–369. doi: 10.1109/TBME.2006.890486
Moroz, A. S., McDonnell, M. D., Burkitt, A. N., Grayden, D. B., and Meffin, H. (2012). Information theoretic inference of the optimal number of electrodes for future cochlear implants using a spiral cochlea model. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2012, 2965–2968. doi: 10.1109/EMBC.2012.6346586
Naycheva, L., Schatz, A., Röck, T., Willmann, G., Messias, A., Bartz-Schmidt, K. U., et al. (2012). Phosphene thresholds elicited by transcorneal electrical stimulation in healthy subjects and patients with retinal diseases. Invest. Ophthalmol. Vis. Sci. 53, 7440–7448. doi: 10.1167/iovs.12-9612
Osborne, L. C., Palmer, S. E., Lisberger, S. G., and Bialek, W. (2008). The neural basis for combinatorial coding in a cortical population response. J. Neurosci. 28, 13522–13531. doi: 10.1523/JNEUROSCI.4390-08.2008
Palanker, D., Le Mer, Y., Mohand-Said, S., Muqit, M., and Sahel, J. A. (2020). Photovoltaic restoration of central vision in atrophic age-related macular degeneration. Ophthalmology 127, 1097–1104. doi: 10.1016/j.ophtha.2020.02.024
Pezaris, J. S., and Reid, R. C. (2007). Demonstration of artificial visual percepts generated through thalamic microstimulation. Proc. Natl. Acad. Sci. U.S.A. 104, 7670–7675. doi: 10.1073/pnas.0608563104
Rizzo, J. F., Wyatt, J., Loewenstein, J., Kelly, S., and Shire, D. (2003). Perceptual efficacy of electrical stimulation of human retina with a microelectrode array during short-term surgical trials. Invest. Ophthalmol. Vis. Sci. 44, 5362–5369. doi: 10.1167/iovs.02-0817
Rockhill, R. L., Daly, F. J., MacNeil, M. A., Brown, S. P., and Masland, R. H. (2004). The diversity of ganglion cells in a mammalian retina. J. Neurosci. 22, 3831–3843. doi: 10.1523/JNEUROSCI.22-09-03831.2002
Roh, H., Kim, E., Chung, S. M., Kang, J. H., Kim, T., and Im, M. (2021). Neural information of artificial vision varies depending on the level of spiking heterogeneity across retinal ganglion cells. Invest. Ophthalmol. Vis. Sci. 62:3228.
Roh, H., Yoon, Y. J., Park, J. S., Kang, D. H., Kwak, S. M., Lee, B. C., et al. (2022). Fabrication of high-density out-of-plane microneedle arrays with various heights and diverse cross-sectional shapes. Nano-Micro Lett. 14:24. doi: 10.1007/s40820-021-00778-1
Roska, B., Molnar, A., and Werblin, F. S. (2006). Parallel processing in retinal ganglion cells: how integration of space-time patterns of excitation and inhibition form the spiking output. J. Neurophysiol. 95, 3810–3822. doi: 10.1152/jn.00113.2006
Sahel, J. A., Boulanger-Scemama, E., Pagot, C., Arleo, A., Galluppi, F., Martel, J. N., et al. (2021). Partial recovery of visual function in a blind patient after optogenetic therapy. Nat. Med. 27, 1223–1229. doi: 10.1038/s41591-021-01351-4
Sanes, J. R., and Masland, R. H. (2015). The types of retinal ganglion cells: current status and implications for neuronal classification. Annu. Rev. Neurosci. 38, 221–246. doi: 10.1146/annurev-neuro-071714-034120
Shivdasani, M. N., Sinclair, N. C., Gillespie, L. N., Petoe, M. A., Titchener, S. A., Fallon, J. B., et al. (2017). Identification of characters and localization of images using direct Multiple-electrode stimulation with a suprachoroidal retinal prosthesis. Invest. Ophthalmol. Vis. Sci. 58, 3962–3974. doi: 10.1167/iovs.16-21311
Stingl, K., Bartz-Schmidt, K. U., Besch, D., Braun, A., Bruckmann, A., Gekeler, F., et al. (2013). Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS. Proc. Biol. Sci. 280:20130077. doi: 10.1098/rspb.2013.0077
Talebi, V., and Baker, C. L. Jr. (2012). Natural versus synthetic stimuli for estimating receptive field models: a comparison of predictive robustness. J. Neurosci. 32, 1560–1576. doi: 10.1523/JNEUROSCI.4661-12.2012
Tkačik, G., Prentice, J. S., Balasubramanian, V., and Schneidman, E. (2010). Optimal population coding by noisy spiking neurons. Proc. Natl. Acad. Sci. U.S.A. 107, 14419–14424. doi: 10.1073/pnas.1004906107
Tochitsky, I., Helft, Z., Meseguer, V., Fletcher, R. B., Vessey, K. A., Telias, M., et al. (2016). How azobenzene photoswitches restore visual responses to the blind retina. Neuron 92, 100–113. doi: 10.1016/j.neuron.2016.08.038
Tripathy, S. J., Padmanabhan, K., Gerkin, R. C., and Urban, N. N. (2013). Intermediate intrinsic diversity enhances neural population coding. Proc. Natl. Acad. Sci. U.S.A. 110, 8248–8253. doi: 10.1073/pnas.1221214110
Twyford, P., Cai, C., and Fried, S. I. (2014). Differential responses to high-frequency electrical stimulation in ON and OFF retinal ganglion cells. J. Neural Eng. 11:025001. doi: 10.1088/1741-2560/11/2/025001
van Wyk, M., Pielecka-Fortuna, J., Löwel, S., and Kleinlogel, S. (2015). Restoring the ON switch in blind retinas: opto-mGluR6, a next-generation, cell-tailored optogenetic tool. PLoS Biol. 13:e1002143. doi: 10.1371/journal.pbio.1002143
Wang, B. Y., Chen, Z. C., Bhuckory, M., Huang, T., Shin, A., Zuckerman, V., et al. (2021). Electronic “photoreceptors” enable prosthetic vision with acuity matching the natural resolution in rats. bioRxiv [Preprint] 452093. doi: 10.1101/2021.07.12.452093
Weitz, A. C., Nanduri, D., Behrend, M. R., Gonzalez-Calle, A., Greenberg, R. J., Humayun, M. S., et al. (2015). Improving the spatial resolution of epiretinal implants by increasing stimulus pulse duration. Sci. Transl. Med. 7:318ra203. doi: 10.1126/scitranslmed.aac4877
Wohrer, A., Kornprobst, P., and Viéville, T. (2006). “From light to spikes: a large-scale retina simulator,” in The 2006 IEEE International Joint Conference on Neural Network Proceedings 4562–4570, (Piscataway, NJ: IEEE).
Yoon, Y. J., Lee, J. I., Jang, Y. J., An, S., Kim, J. H., Fried, S. I., et al. (2020). Retinal degeneration reduces consistency of network-mediated responses arising in ganglion cells to electric stimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 1921–1930. doi: 10.1109/TNSRE.2020.3003345
Zrenner, E., Bartz-Schmidt, K. U., Benav, H., Besch, D., Bruckmann, A., Gabel, V. P., et al. (2011). Subretinal electronic chips allow blind patients to read letters and combine them to words. Proc. Biol. Sci. 278, 1489–1497.
Keywords: retinal prosthetics, visual information, neural computation, information theory, spike trains
Citation: Kim S, Roh H and Im M (2022) Artificial Visual Information Produced by Retinal Prostheses. Front. Cell. Neurosci. 16:911754. doi: 10.3389/fncel.2022.911754
Received: 03 April 2022; Accepted: 18 May 2022;
Published: 06 June 2022.
Edited by:Stuart C. Mangel, The Ohio State University, United States
Reviewed by:John Troy, Northwestern University, United States
Wei Tong, The University of Melbourne, Australia
Copyright © 2022 Kim, Roh and Im. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
†These authors have contributed equally to this work