Simulation of prosthetic vision with PRIMA system and enhancement of face representation

TL;DR

The study addresses the gap between high letter acuity and poor face perception in PRIMA prosthetic vision by introducing ProViSim, a non-pixelated simulator that models resolution and contrast losses. It combines a resolution-reduction pipeline with contrast distortions and validates outputs against patient reports, while enabling ML-driven facial feature enhancement and inverse-contrast pre-processing. The authors demonstrate improvements in emotion recognition and response times in sighted participants and project potential benefits for future smaller-pixel implants when paired with targeted contrast optimization and landmark-based feature reinforcement. The work offers a software-focused pathway to upgrade current PRIMA users and informs hardware development for higher-resolution retinal prostheses, with plans for personalized clinical validation of contrast-loss functions.

Abstract

Objective. Patients implanted with the PRIMA photovoltaic subretinal prosthesis in geographic atrophy report form vision with the average acuity matching the 100um pixel size. Although this remarkable outcome enables them to read and write, they report difficulty with perceiving faces. Despite the pixelated stimulation, patients see smooth patterns rather than dots. We present a novel, non-pixelated algorithm for simulating prosthetic vision, compare its predictions to clinical outcomes, and describe computer vision and machine learning (ML) methods to improve face representation. Approach. Our simulation algorithm (ProViSim) integrates a spatial resolution filter based on sampling density limited by the pixel pitch and a contrast filter representing reduced contrast sensitivity of prosthetic vision. Patterns of Landolt C and human faces created using this simulator are compared to reports from actual PRIMA users. To recover the facial features lost in prosthetic vision due to limited resolution or contrast, we apply an ML facial landmarking model, as well as contrast-adjusting tone curves to the image prior to its projection onto the photovoltaic retinal implant. Main results. Prosthetic vision simulated using the above algorithm matches the letter acuity observed in clinical studies, as well as the patients' descriptions of perceived facial features. Applying the inversed contrast filter to images prior to projection onto the implant and accentuating the facial features using an ML facial landmarking model helps preserve the contrast in prosthetic vision, improves emotion recognition and reduces the response time. Significance. Spatial and contrast constraints of prosthetic vision limit the resolvable features and degrade natural images. ML based methods and contrast adjustments prior to image projection onto the implant mitigate some limitations and improve face representation.

Figures (10)

• Figure 1: A. A diagram of the PRIMA system, including augmented-reality glasses with a camera, an image processor and a projector. Processed images are projected onto the implant using 880nm beam. Each pixel in the subretinal implant converts pulsed light into electric current to stimulate the retina. B. Left: A diagram of a photovoltaic array with a projected Landolt C pattern with a gap width of 1.2 pixels. Right: electric potential across bipolar cells in the retina (from 20 to 87µm above the implant). The red contour outlines the stimulation threshold of 11.7 mV.
• Figure 2: A. Tukey window with a radius of 10 cycles per image and an apodization of 30%. B. The filter’s corresponding mask in the frequency domain. C. The resolution reduction pipeline: an input image is transformed into its frequency representation, multiplied by the filter’s mask and transformed back to the image domain.
• Figure 3: A. Gamma curves along with a sample transform of a grayscale bar and the equation. B. Sigmoid curves along with a sample transform of a grayscale bar and the equation for s=0.2.
• Figure 4: A. Campbell-Robson charts (sinusoidal gratings with variable spatial frequencies (cpd) and contrast) at various Gamma transforms. B. The PRIMA-resolution version of the same charts. C. Campbell- Robson charts with various Sigmoid transforms. D. The PRIMA-resolution version of the same charts.
• Figure 5: A. Landolt C letters with gap sizes varying from 1.2 to 0.6 pixels in width. The black square represents the 20x20 pixel grid size. B. The same letters with the resolution reduction associated with the PRIMA implant. C. The same letters for various Gamma transforms. D. The same letters for various Sigmoid transforms.