Prediction of Refractive Error Based on Ultrawide Field Images With Deep Learning Models in Myopia Patients

Summary Ultrawide field fundus images could be applied in deep learning models to predict the refractive error of myopic patients. The predicted error was related to the older age and greater spherical power. Purpose To explore the possibility of predicting the refractive error of myopic patients by applying deep learning models trained with ultrawide field (UWF) images. Methods UWF fundus images were collected from left eyes of 987 myopia patients of Eye and ENT Hospital, Fudan University between November 2015 and January 2019. The fundus images were all captured with Optomap Daytona, a 200° UWF imaging device. Three deep learning models (ResNet-50, Inception-v3, Inception-ResNet-v2) were trained with the UWF images for predicting refractive error. 133 UWF fundus images were also collected after January 2021 as an the external validation data set. The predicted refractive error was compared with the “true value” measured by subjective refraction. Mean absolute error (MAE), mean absolute percentage error (MAPE) and coefficient (R2) value were calculated in the test set. The Spearman rank correlation test was applied for univariate analysis and multivariate linear regression analysis on variables affecting MAE. The weighted heat map was generated by averaging the predicted weight of each pixel. Results ResNet-50, Inception-v3 and Inception-ResNet-v2 models were trained with the UWF images for refractive error prediction with R2 of 0.9562, 0.9555, 0.9563 and MAE of 1.72(95%CI: 1.62–1.82), 1.75(95%CI: 1.65–1.86) and 1.76(95%CI: 1.66–1.86), respectively. 29.95%, 31.47% and 29.44% of the test set were within the predictive error of 0.75D in the three models. 64.97%, 64.97%, and 64.47% was within 2.00D predictive error. The predicted MAE was related to older age (P < 0.01) and greater spherical power(P < 0.01). The optic papilla and macular region had significant predictive power in the weighted heat map. Conclusions It was feasible to predict refractive error in myopic patients with deep learning models trained by UWF images with the accuracy to be improved.


INTRODUCTION
Myopia is one of the most common causes of distance vision impairment with its global incidence continuing to increase each year (1,2). High myopia and its related retinopathies have been reported to be one of the most common causes of blindness (3). Tessellated fundus, lacquer cracks, focal or diffuse chorioretinal atrophy are all typical fundus findings of myopia, especially high myopia, in clinical settings (4). Deep learning models have already been successfully trained by fundus images in assessing glaucoma (5), diabetic retinopathy (6), age-related macular degeneration (7,8) and retinopathy of prematurity (9). With the help of ultrawide field (UWF) imaging, the visualized fundus photograph obtained an unprecedented large-angle view of up to 200 • and achieved a wide application in ophthalmic settings (10)(11)(12)(13).
Optomap UWF images were gained from a green (532 nm) and a red (633 nm) laser wavelength scanning and were superimposed from the red and green channels by the software to yield semirealistic color, different from the true color of the traditional fundus photographs. While UWF imaging was originally utilized to assess the fundus pathologies, Varadarajan et al. explored the feasibility of training a deep learning model for predicting the refractive error via traditional fundus images (14). However, such feasibility has not been verified on UWF images. It remained unknown whether the peripheral retinal area, unavoidable eyelids or eyelashes would produce noise for the prediction or not. Up to now, fundus images utilized for deep learning training were mainly traditional fundus images with a 30 or 45 • view (15,16).
It is worth exploring the potential of predicting the refractive error via UWF images. The purpose of this study is to explore the feasibility of applying UWF images for deep learning training to predict the refractive error of myopic patients.

Datasets and Image Acquisition
Nine hundred and eighty seven fundus images of 987 patients' left eyes were photographed by Optomap Daytona scanning laser ophthalmoscope (Daytona, Optos, UK) under dual lasers at 532nm and 633nm to gain the 200 • pseudo-color fundus images. The images were obtained from November 2015 to January 2019 in Eye and ENT Hospital of Fudan University. Only patients with myopia in both eyes were included in this study. 133 UWF fundus images were collected after January 2021 as the external validation set to test the performance ResNet-50, Inception-v3 and Inception-ResNet-v2 deep learning of the three models. Only left eyes were chosen since data obtained from the both eyes of the same patient were regarded correlated. All the enrolled patients were myopia patients seeking for refractive surgery treatment. Patients with ocular diseases besides myopia (e.g., diseases that affected fundus imaging like cataract or vitreoretinal diseases or glaucoma), history of trauma or any ocular surgery were all excluded. All enrolled images were gradable with the fovea Abbreviations: UWFI, ultrawide field imaging; CI, confidence interval; D, diopter. located in the center. The images were regarded as gradable when there was no blurring of the optic disc or foveal area and less than 50% peripheral retinal area covered by eyelids or eyelashes.
UWF images were exported in JEPG forms and compressed to 512 * 512 pixels for analysis. The training processes of deep learning models by UWF images for spherical equivalent prediction were summarized in Figure 1. This study complied with the requirements of the Ethics Committee of Eye and ENT Hospital of Fudan University (No. 2020107) and was conducted following the principles of the Declaration of Helsinki.

Network Structure
The deep learning neural network models applied pixel values of UWF fundus images in a series of mathematical calculations, which was the process of the models "learning" how to calculate the spherical equivalent. During the training process, the parameters of the neural network were initially set to random values. For each image in the training set, the predictive values given by the model were compared with the known labels, namely spherical equivalent in this study. Refractive parameters were measured by an experienced optometrist through subjective refraction with the phoropter (NIDEK RT-5100, Japan). Spherical equivalence was used as the label for refractive prediction. Spherical equivalence (SE) equaled spherical power (D) plus 1/2 * cylindrical power (D). With proper adjustments and sufficient data, the deep learning model could predict the refractive error on the new image. Three deep learning models (ResNet-50, Inception-v3 and Inception-ResNet-v2 models) were trained by UWF images for refractive error prediction. 987 fundus images were divided into the training data set (790 UWF images) and the internal test set (197 UWF images) with a ratio of 8:2.
The deep learning models were all built following the Apache2.0 license and written in Python 3.6.6. TensorFlow-GPU 1.12.0 was used as the backend. Keras was adopted as the neural network application programming interface. Keras (https://keras. io) is an open-source artificial neural network library written in Python that serves as an application interface to TensorFlow. Keras supports many artificial intelligence algorithms and serves as a platform for building deep learning models of designing, debugging, evaluation and application.
To visualize the weights of the predicted power of each part of the UWF images, the image features were used as input and the weights predicted by each pixel were averaged to generate the heat map that represented the weight of predicted refractive power.

Algorithm Evaluation
Mean absolute error (MAE), mean absolute percentage error (MAPE) and coefficient value (R 2 ) of refractive prediction were calculated in the test set to assess the predictive performance. MAE (Mean Absolute Error) is defined as the average of the absolute difference between the predicted value and the true value as follows. MAPE (Mean Absolute Percentage Error) is another measure of prediction accuracy defined by the following formula. The sample size is n, the true value of each sample is y i , and the predicted value isŷ i .

Statistical Analysis
Kruskal-Wallis rank sum test was used to compare the characteristics of participants between different datasets. Oneway ANOVA was applied to compare the differences in both MAE and MAPE between different refractive error groups. Tukey multiple comparison test was adopted to assess the differences between each two groups. The Spearman rank correlation test was used to perform univariate analysis and multivariate linear regression analysis on variables that affected the MAE value. Statistical analyses were performed with SPSS version 22.0 (IBM Corp, New York). P < 0.05 was considered statistically significant.

Performance of Deep Learning Models in Refractive Prediction
The 29.95, 31.47, and 29.44% of the test set were within 0.75D of deviation from the "true value" measured by subjective refraction. 64.97, 64.97, and 64.47% of the test set were within 2.00D of deviation from "true value". The comparison of predicted spherical equivalent in the test set and that measured by subjective refraction was shown in

Analysis of Factors Affecting the MAE of Refractive Error Prediction
The MAE was related to the older age (P < 0.01) and the greater spherical power (P < 0.01). Univariate and multivariate analyses of parameters associated with MAE were detailed in Table 1. Examples of typical UWF images in the test set with MAE ≤ 0.5D were shown in Figures 4A-D. The macular and optic papilla area of UWF images made a significant contribution to refractive error prediction in the weighted heat map.

DISCUSSION
Though deep learning models trained by fundus imaging have constantly been utilized for the detection and grading of ophthalmic pathologies (15,17,18), few studies have applied fundus photographs for refractive error prediction (14,19).
Deep learning models trained by traditional 45 and 30 • fundus photographs from the UK Biobank and the Age-Related Eye Disease Research Database could reach the MAE of 0.56D (95% CI: 0.55-0.56 D) and 0.91D (95% CI: 0.89-0.93 D) for refractive prediction (14) while most of the patients were low-grade myope or hyperope. Compared with refractive prediction utilizing 7,307 UWFI fundus images, the MAE could reach 1.115 D with more than a half of enrolled eyes were moderate myopia(−6D ≤ SE < −3D) (19).
It is worth noticing that the enrolled patients in this study were with much higher myopia than those of the previous studies.
More than 90% of the patients in this study were more than −6.00D. Thus, the concept of MAPE was introduced in this study. MAPE is of vital importance in clinical practice. For example, MAE of 2.00 D indicated a huge deviation if a patient's "true" SE was −0.50D, while the same 2.00D predictive error was a minor and insignificant error for a patient with SE of −12.00D.
From the perspective of MAPE, the predictive error via UWF images in this study was relatively smaller than that of the previous study utilizing traditional fundus images and comparable to deep learning models trained by 7,307 UWFI fundus images in certain refractive groups. The deep learning models trained in this study were capable of refractive prediction from UWF images, but the obtained MAE so far was not enough for directly guiding the prescription of eyeglasses or as a reference before refractive surgery.
The MAPE of different myopia groups showed that MAPE increased in SE ≤ −12.00D and low-to-moderate myopia (SE > −6.00D) groups, which was attributed to the imbalanced refractive distribution of enrolled samples.
Older age was found to be related to greater MAE in all three deep learning models. It could be attributed to the darkening of the foveal reflection because of aging (20,21). The MAE was also found to be related to spherical power and had barely relation with cylindrical power, which was consistent with clinical experience. It is the excessive elongation of the globe that plays an important role in the development of myopia and certain fundus degenerative changes like posterior staphyloma, lacquer cracks (22). Astigmatism is the result of irregularity of the cornea or lens, the information of axial length, choroid thickness and retinal portraits was rarely "stored" in astigmatism.
The weighted heat map showed that the macular and optic papilla area contributed the most in predicting the refractive error, which is also consistent with the clinical experience. Myopia, especially high myopia, could result in the thinning of the choroid layer at the macula (23). Although the size of the optic papilla has proven to be unrelated to the degree of myopia (24,25), myopia still affects the morphology of the optic disc, e.g., optic disc tilt, rotation, torsion and the angle   between the superior temporal and inferior temporal arteries of the retina (26). Another reason might be that the brightness of the optic papilla area exceeds the rest area in the UWF imaging dual-color channel.
The limitations of this study were, firstly, the sample size was relatively small. The sample size required for the deep model of training may better reach tens of thousands. Secondly, only data augmentation and dropout layer were applied to prevent over fitting in deep learning training without further separating the validation dataset. Thirdly, part of examined eyes could be minimally too close to or too far from the optimal capturing distance, causing the overall image color to be reddish or greenish.

CONCLUSION
Ultrawide field fundus images could be applied in deep learning training to predict the refractive error of myopic patients with the accuracy to be improved.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.