Development of a predictive model for predicting disability after optic neuritis: a secondary analysis of the Optic Neuritis Treatment Trial

Objective The present study aimed to develop a prediction model for predicting developing debilities after optic neuritis. Methods The data for this research was obtained from the Optic Neuritis Treatment Trial (ONTT). The predictive model was built based on a Cox proportional hazards regression model. Model performance was assessed using Harrell’s C-index for discrimination, calibration plots for calibration, and stratification of patients into low-risk and high-risk groups for utility evaluation. Results A total of 416 patients participated. Among them, 101 patients (24.3%) experienced disability, which was defined as achieving or surpassing a score of 3 on the expanded disability status scale. The median follow-up duration was 15.5 years (interquartile range, 7.0 to 16.8). Two predictors in the final predictive model included the classification of multiple sclerosis at baseline and the condition of the optic disk in the affected eye at baseline. Upon incorporating these two factors into the model, the model’s C-index stood at 0.71 (95% CI, 0.66–0.76, with an optimism of 0.005) with a favorable alignment with the calibration curve. By utilizing this model, the ONTT cohort can be categorized into two risk categories, each having distinct rates of disability development within a 15-year timeframe (high-risk group, 41% [95% CI, 31–49%] and low-risk group, 13% [95% CI, 8.4–17%]; log-rank p-value of <0.001). Conclusion This predictive model has the potential to assist physicians in identifying individuals at a heightened risk of experiencing disability following optic neuritis, enabling timely intervention and treatment.


Introduction
Optic neuritis, a demyelinating inflammation of the optic nerve, affects 3-5 people per 100,000 people annually worldwide (1).The Optic Neuritis Treatment Trial (ONTT) found that most patients with optic neuritis in the study had a connection with multiple sclerosis (MS).This condition can cause substantial visual impairment, impacting patients' quality of life.These visual or physical impairments due to MS could potentially evolve into permanent disabilities (2).During the 15-year follow-up of the ONTT, the Optic Neuritis Study Group observed that patients with optic neuritis generally had good visual acuity outcomes throughout the long-run follow-up.Although 61% of these patients regained their vision with a visual acuity of 20/20 or better in both eyes, some of them continued to experience persistent visual impairment in other aspects of visual function, including the visual field, color vision, and contrast sensitivity (3)(4)(5)(6).According to the National Eye Institute Visual Function Questionnaire scores, they still perceived their visual function to be poorer than a disease-free population, especially those who had a visual or physical disability due to MS, whose vision experience was worse than healthy people (3).
A customized treatment strategy must start with the early detection of patients who are at risk for disability.Previous studies have found that factors including cortical fractal dimension, lesion on magnetic resonance imaging (MRI), and retinal measurements are associated with disability after optic neuritis (7)(8)(9)(10)(11)(12)(13)(14).However, single factors, such as current clinical and paraclinical measures, are not sufficient to reliably predict the risk of disability.Therefore, there is a need for developing risk prediction models that can incorporate multiple factors to improve the accuracy and precision of the prognosis.
Our study aimed to develop a predictive model of disability using the ONTT cohort to identify patients with optic neuritis likely to experience disability, identify risk factors predicting significant visual or motor impairment, and develop a predictive model estimating the likelihood of disability over 10-and 15-year timeframes.Our study aimed to provide more accurate and reliable model to identify those at a high risk of disability and to guide treatment decisions.

Method
We employed a publicly available dataset from the ONTT, accessed through the website http://lons.jaeb.org/.The design of the ONTT, a randomized controlled trial, has been described in previous studies (15-19).In short, the ONTT was conducted to determine the effectiveness of corticosteroids for acute unilateral optic neuritis and the proportion of patients who later developed MS.The ONTT enrolled participants between the ages of 18 and 46 years as long as they had experienced visual symptoms for no more than 8 days.
The ONTT randomly assigned patients among three treatment groups: the high-dose intravenous methylprednisolone group (1,000 mg every day for 3 days), the low-dose oral prednisone group (1 mg per kg per day for 14 days), and the oral placebo group.At baseline, 6 months, and 12 months, as well as annually for 5 years after enrollment, standardized ophthalmological and neurological examinations were performed, followed by evaluations at 10 and 15 years.The ONTT conducted a follow-up on the visual prognosis of these patients and recorded the patients' expanded disability status scale (EDSS) scores (20).
In the ONTT, the diagnosis of MS was made based on the diagnostic criteria for MS proposed by Poser et al. (21).MS was classified into four categories: none, possible, probable, and definite.
In this analysis, patients were divided into those with no or negligible disability (an EDSS score less than 3) and those demonstrating clinically notable disability (an EDSS score equal to or exceeding 3).The patients who had no EDSS score during follow-up were excluded from this analysis.
The ONTT strictly adhered to the principles outlined in the Declaration of Helsinki and received approval from the institutional review boards at each of the implementing institutions.All participants in the ONTT willingly provided informed consent when enrolling in the trial.

Statistical analysis
Categorical variables were denoted by precise numerical values and corresponding ratios, whereas continuous variables were indicated by average values accompanied by standard deviations or medians along with interquartile ranges, depending on whether the distribution of the data is normal or non-normal.The Kaplan-Meier method was used to estimate the probability of developing a disability in the study population, and the test of log-rank was employed to contrast variations between groups in disability occurrence and development rates.Variables with more than 50% missing values were removed before starting modeling.In the model, categories of similarity coefficients were combined (for example, possible and probable MS were combined into a single category).We used all available cases of the ONTT for modeling; therefore, we did not consider the sample size calculation.
The candidate variables in the model were selected based on clinical relevance, prior research (22), and data availability.These variables included age, sex, number of lesions on MRI, treatment group (intravenous, placebo, and prednisone), visual acuity of the affected eye, the baseline condition of the optic disk in the affected eye, pain status of the affected eye, the presence or absence of the optic disk or peripapillary hemorrhage in the affected eye, whether the person lived in the north for 10 or more years of the first 15 years of life (with 'north' defined as states predominantly located above latitude 40° north), the baseline classification of MS (none, possible, probable, and definite), and prior neurologic symptoms.
Analysis of the missing data revealed a random distribution of missing information related to brain lesions, as detected on MRI scans.Using the mice package in R, the missing data were estimated as multivariate estimates by chaining equations.Five imputed datasets were generated and modeled separately, and the estimated values were pooled according to Rubin's rules (23).
We employed the Cox proportional hazards regression model to evaluate the association between potential predictors and the development of disability.The results are presented as a 95% confidence interval (CI) using the hazard ratio (HR).Based on the Akaike Information Criterion, we used a backward stepwise selection procedure to select potential predictors.Finally, the baseline classification of MS and the baseline condition of the optic disk in the affected eye were integrated into the multivariable Cox proportional hazards regression model.
Using the rms and Shiny packages, a nomogram and an online calculator were developed to forecast the likelihood of disability development.Harrell's C-index (24) and calibration curves were utilized to evaluate the effectiveness of the predictive model.The cumulative rates of disability development between the high-and low-risk groups were assessed using the Kaplan-Meier survival curves, which were stratified based on individual predicted total

Demographic and clinical features
In the initial ONTT, a total of 457 cases were enrolled for investigation.Two patients were initially misdiagnosed with optic neuritis, and one patient withdrew before completing the baseline neurological examination.Additionally, 38 cases were excluded from our analysis due to unavailable EDSS scores.Therefore, 416 cases were included for building the predictive model (Figure 1).
At baseline, the age of the 416 cases was 32 on average (standard deviations, 6.8 years), with women comprising 79% (327/416) of the cohort.A significant majority of the participants, namely 87% (362/416), identified as white.Out of the 416 cases, 33 cases (7.9%) were diagnosed with definite MS at the time of their initial enrollment.Additionally, 68 cases (16.3%) were classified as possible MS and 24 cases (5.8%) as probable MS.Moreover, 32% (133/416) of the cases were assigned to high-dose intravenous methylprednisolone treatment, 35% (147/416) to low-dose oral prednisone, and 33% (136/416) to the placebo group.Among the 379 participants with baseline MRI data available, 47% (193/379) showed at least one brain lesion in their baseline MRI.Notably, 91% of the participants exhibited eye pain, while optic disk edema was observed in 36% of the participants.A significant proportion of patients, namely 58%, had lived in the north for 10 or more years of the first 15 years of their lives.Neurological impairment was predominantly mild among the participants.A small proportion of patients (16.1%, 67) experienced moderate disability (EDSS score of 3.0 to 5.5) during the follow-up period and an even smaller group (8.2%, 34) exhibited severe disability (EDSS≥6.0).The probability of developing a disability at 15 years stood at 22% (95% confidence interval [CI], 17-26%).

Predictors in the predictive model
Potential predictors were assessed using the univariate Cox proportional hazards regression model (Table 1).Our novel prognostic model incorporated two factors, including the baseline classification of MS (none, possible, probable, and definite), and the baseline condition of the optic disk in the affected eye (Table 1).In the final model, for MS classification, the HR was 2.92 (95% CI, 1.89, 4.52) for "Possible or Probable" and 5.04 (95% CI, 2.92, 8.68) for "Definite, " with "None" serving as the baseline (HR: 1).Regarding optic disk edema, the HR was 2.42 (95% CI, 1.47, 4.00) for "Normal, " with "Edema" as the baseline (HR: 1).These two factors were explicitly identified as the exclusive predictive factors retained in our final model.The nomogram illustrating the model for predicting the probability of disability conversion after optic neuritis is presented in Figure 2.An online calculator for assessing disability probability can be accessed at https://drduyi.shinyapps.io/edss.

Predictive model performance
The model's C-index was 0.71 (95% confidence interval: 0.66-0.76,with an optimism of 0.005).The calibration plots for the predictive model assessing the development of disability demonstrate excellent performance (Figure 3).Using the model, the ONTT patient cohort was divided into two risk groups based on the likelihood of developing a disability within a 15-year period.These risk groups' rates of being disabled differed noticeably: the high-risk group had a rate of 41% (95% CI, 31-49%) and the low-risk group had a rate of 13% (95% CI, 8.4-17%) (log-rank p < 0.001) (Figure 4).

Discussion
Using our novel predictive model, we stratified the ONTT cohort into low-and high-risk groups for disability development after optic neuritis, which allows the patient to understand the potential for future disability and facilitates the development of treatment strategies with the doctor.
It is essential to identify patients at risk for disability as early as possible to develop personalized management strategies.A recent study by Martinez-Lapiscina et al. (26) demonstrated that baseline retinal layer measurements may predict disability worsening over the subsequent 5 years.Another study has shown that the macula may be a retinal region of particular interest when assessing the burden of neurodegeneration, as MS patients with significantly thinner macula showed higher levels of disability (7).Additionally, research highlights that factors such as male gender, older age at symptom onset (27), and the presence of initial symptoms at the onset of MS (28) are significantly linked to a less favorable prognosis (29).The research on the natural history of MS has found that, compared to the slow onset of motor dysfunction, limb ataxia, or balance impairment that suggests cerebellar involvement symptoms in MS, the presence of optic neuritis at the onset of MS is associated with favorable outcomes in MS (30).In the case of MS-related optic neuritis, disability usually develops during the later stages of MS, which is because MS affects various regions of the brain and spinal cord, leading to a deceleration or blockage in nerve signal transmission between neurons.These effects result in neurological symptoms that may ultimately lead to a diminished quality of life and disability (31).Prior studies have shown after 10 years, two-thirds or more of patients with MS-related optic neuritis have a minor disability; however, after 15 years, up to half of the people may need assistance walking (32)(33)(34).In our finding, we observed that patients with optic neuritis who had clinically definite MS at baseline had a higher risk of developing disability compared to those who were not diagnosed with MS.In the ONTT, high-dose intravenous methylprednisolone may reduce the risk of MS occurrence (22), and in theory, this decrease may also lower the risk of disability.However, our analysis did not observe such an occurrence, so further research is needed to investigate the actual impact of corticosteroid therapy on disability.The Optic Neuritis Study Group has established that patients without a swollen optic disk have a higher likelihood of developing MS (35).Similarly, our study has revealed that patients with MS-related optic neuritis who exhibit non-swollen optic disks at baseline are more predisposed to evolving into disability.
Furthermore, various studies shed light on the relationship between treatment measures and the likelihood of disability development.Disease-modifying therapies (DMTs) have shown the ability to delay the progression from optic neuritis to MS (36).An analysis of the impact of interferon therapy on the evolution of disability after optic neuritis reveals that individuals with MS who experience one or more relapses within the first 2 years of interferon therapy tend to face earlier and more persistent disability (37).Motamed et al. (38) found that interferon-beta-1a (Rebif) DMT may effectively halt the development of impairment.
With the advent of high-efficacy DMTs, there have been studies conducting long-term follow-ups on various high-efficacy DMTs, such as natalizumab (39).Among individuals with relapsing-remitting MS, those treated with ocrelizumab exhibited reduced frequencies of relapses, disability advancement, and MRI activity when juxtaposed with patients treated with interferon-beta (40).While interferon was initially a pivotal DMT for MS, the emergence of new and highly effective DMTs has led to a shift in preferences toward these highefficacy options.It is crucial for individuals with MS to receive appropriate medical care and support to help manage their symptoms and improve their overall functioning.Preliminary findings from the UK risk-sharing scheme have highlighted the potential advantages of DMT in mitigating the development of disability (41).Nomogram for predicting the probability of disability after optic neuritis.An online user-friendly calculator of disability probability is available at https:// drduyi.shinyapps.io/edss.Different perspectives exist on the predictive relationship between baseline MRI scans and disability outcomes.Swanton et al. found that the presence and quantity of spinal cord lesions at baseline and new T2 lesions during follow-up were significant independent predictors of high disability rates in the whole clinically isolated syndrome cohort (42).Fisniku et al. (43) discovered that initial brain MRI findings predicted the development of clinically definite.Lesion volume and its early changes are correlated with disability after 20 years.Their findings suggest that the existence and number of lesions observed at baseline brain MRI may serve as predictive factors for disability in MS-related optic neuritis cases.However, Beck et al. (44) found that, in the ONTT cohort, there was no significant correlation between the level of disability after 10 years and the presence or number of lesions in the initial brain MRI scans.The discordance observed between these studies that focus on clinically isolated syndrome and the conclusions of Beck et al. 's study may stem from a variety of factors.These factors include genetic and environmental influences, as well as disparities in the timing of the research.Furthermore, variations in the patient cohorts studied at different time points could potentially have had an impact on the results.In our study, guided by the Akaike Information Criterion, relevant MRI parameters were not included.

Limitations
While our study successfully achieves its objective of developing prognostic model for disability prediction, several limitations warrant consideration.First, the widespread applicability of the research outcomes may be constrained due to a lack of external validation data.Second, in the serotype examination of 177 ONTT participants, no cases of aquaporin-4-IgG+ ON were found, and only a negligible proportion (1.7%) of participants presented myelin oligodendrocyte glycoprotein-IgG+ ON (45), rendering our model inapplicable to patients with either condition.Third, our model does not take into account other potential factors that could affect the prognosis of disability, such as MRI-related parameters, patient lifestyles, health conditions, or data from OCT monitoring.Finally, given that our data originates from a comparatively early period, our model relied on earlier MS diagnostic criteria that differed from the current McDonald criteria.In practical application, we suggest incorporating a re-diagnosis step based on the old criteria into the modeling process before utilizing our model, ensuring its usability under the new standards.Despite these limitations, internal validation indicates that our proposed predictive model maintains robust discrimination and calibration.

Conclusion
Using the dataset from the ONTT, we have developed a predictive model for predicting disability after optic neuritis.Our prediction model can provide the absolute risk likelihood of developing disability after optic neuritis, which facilitates the co-development of disease management strategies by physicians and patients.

FIGURE 3
FIGURE 3 Calibration curves for the proposed predictive model.(A) 10-year calibration curve.(B) 15-year calibration curve.The dotted line represents the ideal fit.Circles represent model-predicted probabilities, and crosses represent the bootstrap-corrected estimates.Error bars indicate 95% confidence intervals.

TABLE 1
Cox proportional hazards regression model showing the association of different variables with the disability after optic neuritis.

Lived in the north for 10 or more of the first 15 years of life
HR, hazard ratio; LogMAR, logarithm of the minimum angle of resolution.