4.3. Experiments and Results 4 49 Figure 4.6: Results of experiment 3. The model was trained on naturalistic stimuli, comparing three reconstruction tasks. (a) original image (b) Pixel intensity-based reconstruction task with MSE loss (seeEquations(4.1) to (4.3)) (c) Perceptual reconstruction task, using VGG feature loss (seeEquation(4.4); d is set equal to 3) (d) Semantic boundary reconstruction task, using weighted BCE loss (seeEquation(4.5)) between the reconstruction and the ground truth semantic boundary label (i.e., a binary, boundary-based, version of the ground truth label from the dataset). (e) Simulated prosthetic percept after conventional image pre-processing with (left) Canny edge detection (Canny, 1986) and (right) holistically nested edge detection (Xie & Tu, 2017). plausible simulation of phosphene vision that explicitly models this effect of cortical magnification for a specific electrode configuration. In the fourth experiment of our study, we examine an extension of our approach to validate the capacity of our model to optimize for such customized, more realistic, phosphene mappings. Again, we train our end-to-end model, using the ADE20k dataset on the semantic boundary reconstruction task as described in the previous experiment. However, rather than same-sized phosphenes, placed on a distorted rectangular grid, this time, we use a phosphene map that is inspired by the aforementioned study bySrivastava et al. (2009), who simulate phosphenes in the lower left quadrant of the visual field, with phosphene densities and phosphene sizes adjusted in relation to the eccentricity in the visual field to simulate the effect of cortical magnification. For the phosphene simulation, we formalize a custom phosphene map as a set of n pre-defined 256 × 256 greyscale images, P1,P2,...,Pn, that each display a single Gaussian-shaped phosphene at a specific location. In our experiment, the number of phosphenes n is set to 650, 488 or 325. For each imagePi , we generated a phosphene at polar angleφi U(π,π 3 2π), eccentricityri =xi +2x 2 i withxi U(0,1) and sizeσi =2ri +1. After conversion to Cartesian coordinates, Pi covers a square area in the lower left quadrant, bounded by corners (0,-1) and (-1,0). Note that the described procedure reflects an arbitrary example mapping, which may be replaced to yield any pre-specified set of phosphenes. The final SPV image (the output of the simulator) is calculated by taking a weighted sum over all images in the phosphene map: SPV = nX (i=1) wi Pi wi ∈{0,1} (4.6) Here, wdenotes the stimulation protocol which is the output of the encoder. Note that in order to facilitate the simulation of an arbitrary number of freely distributed
RkJQdWJsaXNoZXIy MTk4NDMw