5 76 5. Towards biologically plausible phosphene simulation simulation experiments that further investigate dynamic stimulation. Note that there remain some challenging perceptual considerations for the design of useful dynamical stimulation patterns (for a review on asynchronous stimulation in relation to flicker fusion, form vision and apparent motion perception, seeMoleirinho et al., 2021). Constrained, efficient stimulation for natural stimuli Our second optimization experiment addressed a more natural and realistic context. The results presented inFigure5.8demonstrate that our simulator is well-suited for the optimization of prosthetic vision to natural stimuli and that it can be configured to comply with constraints of the stimulation protocol. Note that the quality of the reconstructions for the constrained version of the encoder indicate that the model can still find an efficient information encoding strategy using a limited set of stimulation amplitudes (10 discrete values between 0 and 128µA. These results are in line with previous results on constrained end-to-end optimization, indicating that task-relevant information can be maximized under sparsity constraints (de Ruyter van Steveninck et al., 2022a). While in the current experiments the stimulation amplitude is maximized for the individual electrodes, future studies could investigate other sparsity constraints, such as a maximum total charge delivered per second across all electrodes. Ultimately, a visual prosthesis may need to prioritize task-relevant information, rather than providing an accurate description of the visual surroundings. For this reason, in recent SPV research with sighted human observers much attention is devoted to semantic (boundary) segmentation for discriminating the important information from irrelevant background (Han et al., 2021; Rasla & Beyeler, 2022; Sanchez-Garcia et al., 2020). Note that the explored image processing strategies in these behavioral studies are equally compatible with the automated optimization through an end-to-end machine learning pipeline. Our experiments exemplify how supervision targets obtained from semantic segmentation data can be adopted to promote taskrelevant information in the phosphene representation. Furthermore, in addition to reconstruction of the input or labelled targets, another recent study experimented with different decoding tasks, including more interactive, goal-driven tasks in virtual game environments (Küçükog˘lu et al., 2022). Although these proof-of-principle results remain to be translated to real-world tasks and environments, they provide a valuable basis for further exploration. Computational optimization approaches can also aid in the development of safe stimulation protocols, because they allow a faster exploration of the large parameter space and enable task-driven optimization of image processing strategies (de Ruyter van Steveninck et al., 2022a; Fauvel & Chalk, 2022; Ghaffari et al., 2021; Granley et al., 2022a; Küçükog˘lu et al., 2022; White et al., 2019). Ultimately, the development of task-relevant scene-processing algorithms will likely benefit both from computational optimization experiments as well as exploratory SPV studies with human observers. The results presented inFiguresS8andS9demonstrate that the end-to-end pipeline can also be used for low electrode counts and that exploratory experimentation with non-linear interaction models is possible. With the presented simulator we aim to contribute a flexible toolkit that can be tailored to specific use cases. Interpretability and perceptual correspondence Besides the encoding efficiency (characterized by the computational decodability of taskrelevant information), it is important to consider the subjective interpretablitity of the
RkJQdWJsaXNoZXIy MTk4NDMw