5 58 5. Towards biologically plausible phosphene simulation 5.1. Introduction Globally, as per 2020, an estimated 43.3 million people were blind (Bourne et al., 2021). For some cases of blindness, visual prosthetics may provide a promising solution. These devices aim to restore a rudimentary form of vision by interacting with the visual system using electrical stimulation (Bloch et al., 2019; Fernández, 2018; Nowik et al., 2020). In particular, our work concerns prosthetic devices that target the primary visual cortex (V1). Despite recent advances in the field, more research is required before cortical prosthesis will become clinically available. Besides research into the improvement of the safety and durability of cortical implants (Chen et al., 2020; Fernández et al., 2021), a great portion of the research attention is devoted to optimizing the efficacy, efficiency, and practical usefulness of the prosthetic percepts. The artificially induced visual percepts consist of patterns of localized light flashes (‘phosphenes’) with a limited resolution. To achieve a functional level of vision, scene-processing is required to condense complex visual information from the surroundings in an intelligible pattern of phosphenes (de Ruyter van Steveninck et al., 2022a; Granley et al., 2022a; Han et al., 2021; Lozano et al., 2020; Normann et al., 2009; Sanchez-Garcia et al., 2020; Troyk, 2017). Many studies employ a simulated prosthetic vision (SPV) paradigm to non-invasively evaluate the functional quality of the prosthetic vision with the help of sighted subjects (Beyeler et al., 2017; Bollen et al., 2019b; Cha et al., 1992b; 1992a; Dagnelie et al., 2006; 2007; de Ruyter van Steveninck et al., 2022b; Han et al., 2021; Normann et al., 2009; Pezaris & Reid, 2008; Rassia et al., 2022; Sommerhalder et al., 2004; Srivastava et al., 2009; Thompson et al., 2003; Vergnieux et al., 2014; Vurro et al., 2014) or through ‘end-to-end’ approaches, using in silico models (de Ruyter van Steveninck et al., 2022a; Granley et al., 2022a; Küçükog˘lu et al., 2022). Although the aforementioned SPV literature has provided us with important insights, the perceptual realism of electrically generated phosphenes and some aspects of the biological plausibility of the simulations can be further improved by integrating knowledge of phosphene vision and its underlying physiology. Given the steadily expanding empirical literature on cortically-induced phosphene vision, it is both feasible and desirable to have a more phenomenologically accurate model of cortical prosthetic vision. Such an accurate simulator has already been developed for retinal prostheses (Beyeler et al., 2017), which has formed an inspiration for our work on simulation of cortical prosthetic vision. Thus, in this current work, we propose a realistic, biologically inspired computational model for the simulation of cortical prosthetic vision. Biological plausibility, in our work’s context, points to the simulation’s ability to capture essential biological features of the visual system in a manner consistent with empirical findings: our simulator integrates quantitative findings and models from the literature on cortical stimulation in V1. The elements that are modeled in our simulator include cortical magnification, current-dependent spread of activation and charge-dependent activation thresholds. Furthermore, our simulator models the effects of specific stimulation parameters, accounting for temporal dynamics. A schematic overview of the pipeline and some example outputs are displayed inFigure5.1. Our simulator runs in real-time, is open-source, and makes use of fully differentiable functions, which is an essential requirement for the gradient-based optimization of phosphene encoding models with machine learning. This design enables both simulations with sighted participants, as well as end-to-end optimization in machine-learning frameworks, thus fitting the needs of
RkJQdWJsaXNoZXIy MTk4NDMw