4 54 4. End-to-end optimization of prosthetic vision for specific phosphene configurations. The results of experiment 4 indicate that our end-to-end approach can successfully optimize phosphene encoding for arbitrary configurations. A reduction of the number of available phosphenes from 650 to 488 or 325 phosphenes was associated with reduced reconstruction performance. The overall reconstruction performance with the customized phosphene mapping was lower compared to the regular phosphene mapping that was used in experiment 3. This reduction may partly be explained by the reduced number of phosphenes, however this was not formally tested. Possibly, the extension of our model to arbitrary phosphene mappings forms an inherently more challenging task. Knowledge about the perceived phosphene coverage, which is unique for every patient, can be informative for finding a suitable encoding strategy (Buffoni et al., 2005; Kiral-Kornek et al., 2013). By including a customizable phosphene simulation module in our end-to-end architecture, we aim to provide a tool that can be employed for tailored optimization to implant- or patient-specific characteristics. 4.4.5. Limitations and future directions Some limitations of the present study provide directions for future research. First, the subjective quality of the phosphene representations is not addressed in the current study. Future research could compare the phosphene encoding strategies found by our proposed model, to existing pre-processing approaches from the current literature, using behavioural experiments. Second, the simulated prosthetic vision that was used in the current study is still a simplified model of the reality, and it does not address several stimulation dynamics, such as pulse frequency, inter-stimulation interval and interactions between electrodes. Accurate simulation of these characteristics require further adaptations to our phosphene simulator. Also, it may be worthwhile to simulate different type of implants. For instance, due to inadvertent activation of underlying axon pathways, phosphenes generated with retinal prostheses may demonstrate distorted shapes that vary across subjects and even individual electrodes (Beyeler et al., 2019). Third, in this paper the model is trained on static images. Future approaches could extend our end-to-end model to process dynamical stimuli, resembling an even more naturalistic setting and addressing dynamical aspects of the stimulation. Finally, the optimization tasks that were used in the current paper, remain basic. Future work could extend the current approach with other or more complex tasks. For instance, with reinforcement learning strategies (see White et al., 2019), the model could be extended to perform tasks that more closely related to the everyday actions that need to be performed by the end-user, such as object manipulation (Levine et al., 2016) or object avoidance (LeCun et al., 2005). 4.5. Conclusion In this paper we present a novel DL-based approach for automated and tailored end-toend optimization of prosthetic vision. Our validation experiments show that such an approach may help to automatically find a task-specific stimulation protocol, considering an additional sparsity requirement. The presented approach is highly modular and could be extended to dynamically optimize prosthetic vision for everyday tasks and requirements of the end-user.
RkJQdWJsaXNoZXIy MTk4NDMw