7 112 7. Summary Chapter4investigated a more generalized optimization framework for the automated optimization of visual prostheses through deep learning. In addition toevaluationwitha virtual patient using a neural network decoder, the framework employed a neural network encoder for thegenerationof phosphene encodings. Both the encoder and the decoder were trained simultaneously in an end-to-end optimization procedure. Proof-of-principle experiments indicated that the end-to-end auto-encoder architecture can successfully be trained on basic black-and-white stimuli, as well as complex naturalistic visual stimuli. Without further constraints, the model only maximized information transfer, converging to clinically unfeasible solutions. The addition of a sparsity constraint resulted in the parameterized control over the number of active electrodes. The results of further experiments indicated that by changing the objective function the model can converge to different phosphene encodings. Depending on the objective function, either low-level pixel information or more abstract perceptual content was preserved in the phosphenes. With a supervised training procedure, the encoder could be trained to encode labeled target representations. A set of training runs on arbitrary phosphene layouts indicated that the model could be tailored to implant-specific characteristics. Altogether the results indicate that the framework can be used for automated optimization and can be tailored to safety requirements, custom reconstruction tasks, and patient-specific hardware configurations. Although the phosphene simulation and the tasks are relatively basic, the presented proof-of-principle results demonstrate the potential of end-to-end optimization. Chapter5discusses an improved phosphene simulation compared to the simulations in the first chapters. Possible directions are investigated towards a more biologically plausible simulation of cortical prosthetic vision. The simulator makes use of differentiable operations and it explicitly links the location, size, brightness, perception threshold and temporal dynamics of the rendered phosphenes to the electrode locations and specific electrical stimulation parameters. For the calculation of the phosphene size, the simulator takes into account cortical magnification and the spread of activation in the cortical tissue, making use of contemporary models from the literature. The phosphene brightness and the perception thresholds are based on a leaky-integrator model that calculates the estimated effective cortical activation based on the amplitude, frequency pulse width and the duration of the stimulation. The reproduced brightness (R2 =0.950) and detection thresholds (R2 =0.844) adequately reflect empirical data from the clinical literature for different stimulation parameters. Using a second leaky integrator model that tracks the stimulation history, the model adequately reproduces empirical data from the literature on repeated stimulation (R2 =0.930). The simulator demonstrates real-time simulation of up to 10.000 phosphenes. The results of several feasibility experiments demonstrate that the simulator can be used in computational optimization pipelines, including the end-to-end framework described inChapter4. In summary, the simulator presented in this chapter successfully models the findings from a wide range of experimental results, narrowing the gap between simulation and reality. It can operate in real time and uses differentiable operations, which make the simulator a viable option for behavioral experiments with sighted subjects, as well as machine learning frameworks. The simulator fits the needs of fundamental, clinical and computational vision scientists working on cortical prosthetic vision.
RkJQdWJsaXNoZXIy MTk4NDMw