Neuromyelitis Optica Spectrum Disorder With Anti-Aquaporin-4 Antibody: Outcome Prediction Models

Abstract

Background:

Recognizing the predictors of disease relapses in patients with anti-aquaporin-4 antibody (AQP4-ab)-positive neuromyelitis optica spectrum disorder (NMOSD) is essential for individualized treatment strategy. We aimed to identify the factors that predicted relapses among patients with AQP4-ab-positive NMOSD, develop outcome prediction models, and validate them in a multicenter validation cohort.

Methods:

Between January 2015 and December 2020, 820 patients with NMOSD were registered at Huashan Hospital. We retrospectively reviewed their medical records, and included 358 AQP4-ab-positive patients with 1135 treatment episodes. Univariate and multivariate analyses were used to explore the predictors of relapse, severe visual or motor disability during follow-up. A model predicting the 1- and 2-year relapse-free probability was developed and validated in an external validation cohort of 92 patients with 213 treatment episodes.

Results:

Lower serum AQP4-ab titer (< 1:100), higher Expanded Disability Status Scale (EDSS) score at onset (≥ 2.5), and use of intravenous methylprednisolone (IVMP) at the first attack predicted an overall lower annualized relapse rate. Older age (> 48 years), optic neuritis at onset, and higher onset EDSS score (≥ 2.5) significantly increased the risk for blindness, while IVMP at the first attack and maintenance therapy reduced the risk for blindness. Myelitis at onset increased the possibility of motor disability (EDSS ≥ 6.0), severe motor disability or death (EDSS ≥ 8.0), while maintenance therapy reduced these possibilities. Anderson and Gill model identified that the risk factors predicting recurrent relapses under certain treatment status were female gender, high AQP4-ab titer (≥ 1:100), previous attack under same therapy, lower EDSS score at treatment initiation (< 2.5), and no maintenance therapy or oral prednisone lasting less than 6 months. A nomogram using the above factors showed good discrimination and calibration abilities. The concordance indexes in the primary and validation cohort were 0.66 and 0.65, respectively.

Conclusion:

This study reports the demographic, clinical and therapeutic predictors of relapse, and severe visual or motor disability in NMOSD. Early identification of patients at risk of unfavorable outcomes is of paramount importance to inform treatment decisions.

Introduction

Neuromyelitis optica spectrum disorders (NMOSD) are autoantibody-induced inflammatory diseases of the central nervous system, characterized by recurrent optic neuritis (ON) and transverse myelitis (TM), leading to blindness and paralysis (1). Specific serum anti-aquaporin-4 antibodies (AQP4-ab) are pathogenic and identified in most patients with NMOSD (2). A proportion of AQP4-ab-negative NMOSD patients have serum anti-myelin oligodendrocyte glycoprotein antibodies (MOG-ab) and exhibit different characteristics than AQP4-ab-positive patients (3).

Disability in NMOSD patients depends on the relapses; therefore, the primary goal of NMOSD treatment is to prevent or delay attacks. Drugs conventionally used as maintenance therapy in NMOSD include immunosuppressants and monoclonal antibodies, such as azathioprine (AZA), mycophenolate mofetil (MMF), and rituximab (RTX) (4). Recent randomized controlled trials of satralizumab, eculizumab, and inebilizumab have reported promising results for NMOSD treatment (5–8). However, consensus regarding the position of these new preventive drugs in the management algorithm has not been reached. There is insufficient evidence to compare the effectiveness of different maintenance therapies for NMOSD (9, 10). Little is known about the demographic, clinical, and therapeutic predictors of relapse, severe visual or motor disability in NMOSD (11–16). Early identification of patients at risk of unfavorable outcomes is critical for individualized treatment decisions.

The aim of the present study was to investigate the treatment outcomes of NMOSD in China, compare the effectiveness of different maintenance therapies by analyzing the relapses, evaluate the risk factors that predict relapse, and severe visual or motor disability, and design a nomogram that can be used to estimate the probability of 1- and 2-year relapse-free status.

Methods

Study Design and Participants

This retrospective cohort study involved a review of the medical records of 820 consecutive NMOSD patients who presented to the Shanghai Huashan Hospital, China, between January 2015 and December 2020. Patients were included to form a primary cohort in this study if they fulfilled the criteria established by the International Panel for NMOSD diagnosis in 2015 (17), were serum AQP4-ab-positive by cell-based assays, were followed up for at least 2 years, and had treatment episodes for at least 3 months, with definite start and stop dates.

We collected the demographic and clinical data of eligible patients, including onset age, gender, serum AQP4-ab titer, onset attack type (ON, TM, brainstem/cerebral, or mixed attack), subsequent attack (type, start date, treatment, and outcome), Expanded Disability Status Scale (EDSS) score, visual acuity, maintenance therapy (dosage, start and stop dates), and concomitant auto-antibodies. The first available serum AQP4-ab titer detected in a remission status was used in this analysis.

An attack/relapse was defined as worsening of existing symptoms, or occurrence of new symptoms, lasting for at least 24 hours; multiple symptoms within 30 days were regarded as a single attack (18). We defined “first attack” as the inaugural attack that marked the disease onset, “first relapse” as the second attack experienced after the disease onset, and recurrent relapses as all attacks after the first attack.

We identified the predictors of annualized relapse rates (ARRs; calculated from disease onset to end of follow-up, excluding the first attack) for ON, TM, brainstem/cerebral attacks, and overall attacks. We also evaluated the risk factors for first relapse, blindness (visual acuity ≤ 0.1 for > 6 months), motor disability (EDSS score ≥ 6.0), and severe disability or death (EDSS score ≥ 8.0).

To explore the predictors for recurrent relapses despite maintenance treatment, the duration was calculated from the initiation of certain maintenance therapy to the subsequent endpoint. Patients who received no treatment were also recognized as a treatment category and included in the primary cohort. The predictors identified from the primary cohort were validated on an external validation cohort, consisting of 92 AQP4-ab-positive patients from Jing’an District Central Hospital (n = 26), the First Affiliated Hospital of Xinjiang Medical University (n = 18), the Second Affiliated Hospital of Guangzhou University of Chinese Medicine (n = 17), the First People’s Hospital of Yunnan Province (n = 9), Sir Run Run Shaw Hospital (n = 9), the First Affiliated Hospital of Wenzhou Medical University (n = 5), the Fifth Affiliated Hospital of Zhengzhou University (n = 3), Wuhan First Hospital (n = 3), and Jinan Central Hospital (n = 2).

All patients from the Huashan Hospital and nine other study centers underwent serum AQP4-ab and MOG-ab detection with fixed cell-based indirect immune-fluorescence tests. HEK293 cells transfected with either the M1 isoform of AQP4 or full-length human MOG were used. AQP4-ab titer ≥ 1:100 was defined as a high AQP4-ab level. All AQP4-ab-positive patients were not found to harbor MOG-ab.

Treatment Episodes

NMOSD treatments during follow-up were divided into stable (no relapse) and failed (disease relapse) treatment episodes. The effectiveness durations of medications from the time of last administration were as follows: 180 days for RTX; 90 days for mitoxantrone (MIT); 30 days for AZA, MMF, tacrolimus (TAC), cyclophosphamide (CTX), cyclosporine A (CsA), intravenous methylprednisolone (IVMP), intravenous immunoglobulin (IVIG), and plasma exchange (PE); and 7 days for methotrexate (MTX) and oral steroids (11). Treatment episodes were excluded from analyses if they consisted of less than 15 episodes of a certain medication, lasted for < 3 months, without documented start or stop dates, or consisted of patients who had double/overlapping treatments. The duration of certain treatment episodes was prolonged, if there was no relapse. In case of gaps between treatment episodes that lasted shorter than the effectiveness duration, the episodes were considered a single episode.

Statistical Analyses

We performed statistical analyses and constructed figures using the R software (version 4.0.3; R Foundation for Statistical Computing, Vienna, Austria; http://www.r-project.org/), using rms, survival, and ggDCA packages (19–21). Continuous variables are presented as medians (interquartile range, IQR) or means (± 1 standard deviation, SD).

Two datasets were used for the analyses. In Dataset A, generalized linear regression was applied to evaluate the associations between multiple variables and ARRs for ON, TM, brainstem/cerebral and all attack types. Univariate cox regression was applied to analyze the associations of multiple variables with first relapse, blindness, disability, and severe disability. Maintenance therapy was selected as a time-dependent variable. In Dataset B, we selected patient identification as a cluster variable in the Anderson-Gill (AG) proportional-hazards model, and analyzed recurrent events in terms of “time to subsequent relapse” (22). Variables associated with significant rate ratios (RRs) or hazard ratios (HRs) (i.e., p < 0.1) in the univariate analysis were further analyzed with a multivariate model.

Some events were analyzed together to improve statistical power and stability. We analyzed unilateral and bilateral ON, brainstem attacks, and cerebral attacks together because the number of cerebral attacks was insufficient. Similarly, IVMP, IVIG, and PE were combined as acute attack therapy; unilateral and bilateral blindness as well as EDSS score ≥ 8.0 and death were combined as the composite endpoint. If a patient had a mixed attack of ON and TM, the attack would contribute to the ARRs for both ON and TM.

A model was developed to predict disease relapses using the AG model; this model was depicted in a nomogram to estimate the 1- and 2-year relapse-free probability in the primary cohort and validated in the external cohort. Concordance index (C-index), calibration plot, and decision curve analysis (DCA) were used to evaluate the discrimination and calibration ability of the model, as well as its clinical usefulness. Bootstrap with 1000 resampling was performed for the internal and external validation in the primary and validation cohorts, respectively, using the C-index and calibration curve (23, 24). Statistical significance was set at p-value < 0.05.

Data Availability

Anonymized data not presented in the study will be made available upon request from any qualified investigator.

Results

Cohort Study

Figure 1 depicts the flow chart for data inclusion and exclusion. We included 358 patients with 1135 treatment episodes from Huashan Hospital to form the primary cohort, and 92 patients with 213 treatment episodes from other centers to form the external validation cohort. Baseline demographic and clinical characteristics of the two cohorts were comparable (Table 1). The most common onset manifestations were ON and TM; AZA, MMF, and RTX were the most frequently prescribed immunosuppressive drugs.

Figure 1

Predictors of ARRs

The ARRs for each attack type (ON, TM, or brainstem/cerebral attacks) and overall ARRs covering all attack types were calculated. To avoid over-estimating the ARR, the inaugural attack that defined the disease onset was excluded from the calculation. Supplementary Table 1 and Table 2 summarize the effects of predictors on ARRs and certain events in the univariate and multivariate analyses. Patients with high AQP4-ab levels (≥ 1:100) had higher ARRs for TM (p = 0.038) and all attacks (p = 0.016) compared to those with low AQP4-ab levels (< 1:100). Patients with ON at onset had higher ARRs for ON (p < 0.001), while those with TM at onset had lower ARRs for brainstem/cerebral attacks (p < 0.001). Patients with higher EDSS score at onset (≥ 2.5) had lower ARRs for TM (p = 0.003), and all attacks (p = 0.025). Compared to patients not treated with corticosteroids, those treated with IVMP at disease onset had ARRs reduced by 9% for ON (p < 0.001), and 9% for all attacks (p = 0.005). An inverse correlation was observed between maintenance therapy and all attacks (p = 0.18), albeit without statistical significance.

Predictors of First Relapse, Blindness, Motor Disability, and Severe Disability During Follow-Up

We explored the factors that predicted first relapse, blindness, disability, and severe disability during follow-up. Patients with younger onset age (≤ 35), compared to those with older onset age, had lower risk for blindness (p = 0.014). Patients with ON at onset had significantly higher risk for blindness (p = 0.002), while patients with TM at onset had significantly higher risk for disability and severe disability (p = 0.004 and 0.043, respectively). Patients with higher EDSS score at disease onset (≥ 2.5) had higher risk for blindness (p < 0.001). IVMP treatment at the first attack and maintenance therapy significantly reduced the risk for first relapse (p = 0.026 and p < 0.001, respectively), blindness (p = 0.034 and p < 0.001, respectively), while only maintenance therapy significantly lowered the risk for disability and severe disability during follow-up (p < 0.001 and p = 0.005, respectively) (Supplementary Table 2 and Table 3).

Predictors of Recurrent Relapses and Development of Individualized Prediction Model

Recurrent relapses were defined as all attacks after the first attack as mentioned above. We explored the predictors of recurrent relapses under certain treatment episode. The analysis included 358 patients with 1135 treatment episodes. Supplementary Figure 1 is a forest plot for predictors of recurrent relapses during treatment episodes, identified using a univariate AG model. We incorporated treatment episodes with no treatment or prednisone < 6 months as the same category for the HR was 1.01 (95% confidence interval [CI]: 0.81–1.27) (p = 0.92).

Multivariate AG analysis identified that the risk factors predicting recurrent relapses were female gender, high AQP4-ab titer (≥ 1:100), previous attack under same therapy, lower EDSS score at treatment initiation (< 2.5), and no maintenance therapy or oral prednisone lasting less than 6 months (Table 4). We also observed that RTX showed the best effect in preventing relapses (HR = 0.20, p < 0.001), while CTX was the weakest among various drugs (HR = 0.86, p = 0.71), with reference to no maintenance drug or use of oral prednisone less than 6 months.

Figure 2A depicts the nomogram for this Model, including the above factors used to estimate the 1- and 2-year relapse-free probability. The calibration curve demonstrated good agreement between the predicted and actual 1- and 2-year relapse-free probability in the primary cohort (Figures 3A, B). The C-index of the nomogram was 0.66 (95% CI: 0.64–0.69) and 0.68 via bootstrapping validation. Figure 2B depicts the DCA for this Model. The decision curve demonstrated that for threshold probability exceeding 0.15, the nomogram was associated with improved prediction of subsequent relapses and was superior to the treat-all or treat-none strategy.

Figure 2

Figure 3

External Validation of the Prediction Model for Recurrent Relapses

The prediction model for recurrent relapses during certain treatment episode was validated on an external cohort. In the validation cohort, good calibration was observed between the predicted and actual 1- and 2-year relapse-free probability (Figures 3C, D). The C-index of the nomogram was 0.65 (95% CI: 0.60–0.70) and 0.70 with bootstrap resampling.

Discussion

Previous studies have reported the predictors of relapse, blindness, and disability in NMOSD patients using univariate and multivariate regression analysis (11, 14, 25–28). The current study included the largest multicenter cohort of AQP4-ab-positive patients to date and systematically explored the predictors of relapse, and severe visual or motor disability in NMOSD. We demonstrated that independent factors predicting recurrent relapses in NMOSD included female gender, high AQP4-ab titer (≥ 1:100), previous attack under same therapy, lower EDSS score at treatment initiation (< 2.5), and no maintenance therapy or oral prednisone lasting less than 6 months. We also developed a feasible prediction model that could facilitate the selection of appropriate immunotherapy to reduce the probability of subsequent relapse.

We investigated the use of immunotherapies for AQP4-ab-positive patients in China. AZA, MMF, and RTX were the most commonly used first-line immunosuppressive therapies that decrease relapses, consistent with the previous study (9). We also observed that no or inappropriate immunotherapy was used for some NMOSD patients (e.g., use of interferon in NMOSD patients misdiagnosed with multiple sclerosis), indicating antibody detection should be considered as an indispensable examination to improve the diagnostic accuracy of central nervous system idiopathic inflammatory demyelinating diseases.

Previous studies have evaluated factors associated with ARRs and certain events such as first relapse, blindness, and disability in NMOSD (14, 25, 28). These studies reported that onset age, onset attack type, and maintenance therapy were associated with ARRs and disability events, similar to our results. Older patients were found more likely to develop blindness and disability compared to younger patients, which suggested that late-onset NMOSD had a worse prognosis. Seok et al. evaluated the clinical characteristics of late- and early-onset NMOSD and observed that onset age had a significant positive correlation with EDSS score (29). In our study, older onset age was associated with a higher probability of blindness. Additionally, our results emphasized the importance of acute attack therapy, which significantly decreased the risk for first relapse and blindness. As researchers pointed out that IVMP should be administered within 4 days of attack onset for full visual recovery (30).

We evaluated the predictors for recurrent relapses in the study cohort. Time to first relapse under certain treatment was the most commonly used study outcome measure; however, few studies focused on the predictors of recurrent NMOSD events using time to subsequent relapse (26, 31). Consistent with previous reports (11, 26), we found that maintenance therapy predicted subsequent relapse, using a multivariate AG model. RTX was the most potent immunotherapy, consistent with the previous studies that compared the first-line therapies (AZA, MMF, and RTX) (9). The current study included additional immunotherapies (such as prednisone ≥ 6 months, TAC, CTX, and CsA) to provide a more comprehensive comparison. Stellmann et al. reported that previous attacks on the same treatment regimen increase the probability of future relapses, but without any statistical significance (11). In this study, we found this variable to be a strong predictor of future relapses (HR 1.32; p=0.007); therefore, we recommend that patients with attacks on certain treatment should switch to a more potent immunotherapy drug, such as RTX, without delay.

The prediction model was simple and feasible to use, especially with a nomogram that predicted the probability of subsequent relapses. Nomograms are widely used to determine the prognosis in the fields of medicine and oncology (32). The nomogram developed from the primary cohort and the calibration curve exhibited good agreement, while the C-index demonstrated good discrimination ability in the internal and external cohort (0.66 and 0.65). The significance of this nomogram was that it provided an effective reference regarding the prognosis when counseling NMOSD patients about their risk for subsequent relapses. To justify the use of the nomogram, we evaluated the effects of the nomogram on patient outcomes. The novel DCA method uses threshold probability to determine the clinical outcomes and the corresponding net benefit. The nomogram was found to be clinically useful in the DCA.

The primary limitation of this study was its retrospective nature, with the associated possibility of recall bias. The dose of maintenance therapies was not standardized in the primary and validation cohorts. Additionally, the AQP4-ab titers were not determined at fixed time points after NMOSD onset. Future prospective studies, with larger sample sizes and scheduled timings of antibody detection, are needed to validate our findings.

To conclude, this study evaluated the demographic, clinical and therapeutic predictors of relapse, and severe visual or motor disability in NMOSD. Early identification of the patients with unfavorable outcome features is of paramount importance to inform treatment decisions.

Funding

This research was supported by the National Natural Science Foundation of China (Grant No. 82171341, 81771296), the Shanghai Municipal Science and Technology Major Project (No. 2018SHZDZX01) and ZHANGJIANG LAB, and the National Key Research and Development Program of China (2016YFC0901504).

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  JariusSPaulFWeinshenkerBGLevyMKimHJWildemannB. Neuromyelitis Optica. Nat Rev Dis Primers (2020) 6:85. doi: 10.1038/s41572-020-0214-9

  • CrossRef
  • Google Scholar

• 2

  PapadopoulosMCVerkmanAS. Aquaporin 4 and Neuromyelitis Optica. Lancet Neurol (2012) 11:535–44. doi: 10.1016/s1474-4422(12)70133-3

  • CrossRef
  • Google Scholar

• 3

  KitleyJWatersPWoodhallMLeiteMIMurchisonAGeorgeJet al. Neuromyelitis Optica Spectrum Disorders With Aquaporin-4 and Myelin-Oligodendrocyte Glycoprotein Antibodies: A Comparative Study. JAMA Neurol (2014) 71:276–83. doi: 10.1001/jamaneurol.2013.5857

  • CrossRef
  • Google Scholar

• 4

  PapadopoulosMCBennettJLVerkmanAS. Treatment of Neuromyelitis Optica: State-of-the-Art and Emerging Therapies. Nat Rev Neurol (2014) 10:493–506. doi: 10.1038/nrneurol.2014.141

  • CrossRef
  • Google Scholar

• 5

  TraboulseeAGreenbergBMBennettJLSzczechowskiLFoxEShkrobotSet al. Safety and Efficacy of Satralizumab Monotherapy in Neuromyelitis Optica Spectrum Disorder: A Randomised, Double-Blind, Multicentre, Placebo-Controlled Phase 3 Trial. Lancet Neurol (2020) 19:402–12. doi: 10.1016/s1474-4422(20)30078-8

  • CrossRef
  • Google Scholar

• 6

  YamamuraTKleiterIFujiharaKPalaceJGreenbergBZakrzewska-PniewskaBet al. Trial of Satralizumab in Neuromyelitis Optica Spectrum Disorder. New Engl J Med (2019) 381:2114–24. doi: 10.1056/NEJMoa1901747

  • CrossRef
  • Google Scholar

• 7

  PittockSJBertheleAFujiharaKKimHJLevyMPalaceJet al. Eculizumab in Aquaporin-4-Positive Neuromyelitis Optica Spectrum Disorder. N Engl J Med (2019) 381:614–25. doi: 10.1056/NEJMoa1900866

  • CrossRef
  • Google Scholar

• 8

  CreeBACBennettJLKimHJWeinshenkerBGPittockSJWingerchukDMet al. Inebilizumab for the Treatment of Neuromyelitis Optica Spectrum Disorder (N-MOmentum): A Double-Blind, Randomised Placebo-Controlled Phase 2/3 Trial. Lancet (London England) (2019) 394:1352–63. doi: 10.1016/s0140-6736(19)31817-3

  • CrossRef
  • Google Scholar

• 9

  PoupartJGiovannelliJDeschampsRAudoinBCironJMaillartEet al. Evaluation of Efficacy and Tolerability of First-Line Therapies in NMOSD. Neurology (2020) 94:e1645–e56. doi: 10.1212/wnl.0000000000009245

  • CrossRef
  • Google Scholar

• 10

  TaharaMOedaTOkadaKKiriyamaTOchiKMaruyamaHet al. Safety and Efficacy of Rituximab in Neuromyelitis Optica Spectrum Disorders (RIN-1 Study): A Multicentre, Randomised, Double-Blind, Placebo-Controlled Trial. Lancet Neurol (2020) 19:298–306. doi: 10.1016/s1474-4422(20)30066-1

  • CrossRef
  • Google Scholar

• 11

  StellmannJPKrumbholzMFriedeTGahlenABorisowNFischerKet al. Immunotherapies in Neuromyelitis Optica Spectrum Disorder: Efficacy and Predictors of Response. J Neurol Neurosurg Psychiatry (2017) 88:639–47. doi: 10.1136/jnnp-2017-315603

  • CrossRef
  • Google Scholar

• 12

  KitleyJLeiteMINakashimaIWatersPMcNeillisBBrownRet al. Prognostic Factors and Disease Course in Aquaporin-4 Antibody-Positive Patients With Neuromyelitis Optica Spectrum Disorder From the United Kingdom and Japan. Brain: J Neurol (2012) 135:1834–49. doi: 10.1093/brain/aws109

  • CrossRef
  • Google Scholar

• 13

  KimSHHyunJWJoungAParkEYJooJKimHJ. Predictors of Response to First-Line Immunosuppressive Therapy in Neuromyelitis Optica Spectrum Disorders. Mult Scler (Houndmills Basingstoke England) (2017) 23:1902–8. doi: 10.1177/1352458516687403

  • CrossRef
  • Google Scholar

• 14

  PalaceJLinDYZengDMajedMElsoneLHamidSet al. Outcome Prediction Models in AQP4-IgG Positive Neuromyelitis Optica Spectrum Disorders. Brain: J Neurol (2019) 142:1310–23. doi: 10.1093/brain/awz054

  • CrossRef
  • Google Scholar

• 15

  JariusSRuprechtKWildemannBKuempfelTRingelsteinMGeisCet al. Contrasting Disease Patterns in Seropositive and Seronegative Neuromyelitis Optica: A Multicentre Study of 175 Patients. J Neuroinflamm (2012) 9:14. doi: 10.1186/1742-2094-9-14

  • CrossRef
  • Google Scholar

• 16

  KimbroughDJMealyMASimpsonALevyM. Predictors of Recurrence Following an Initial Episode of Transverse Myelitis. Neurol Neuroimmunol Neuroinflamm (2014) 1:e4. doi: 10.1212/NXI.0000000000000004

  • CrossRef
  • Google Scholar

• 17

  WingerchukDMBanwellBBennettJLCabrePCarrollWChitnisTet al. International Consensus Diagnostic Criteria for Neuromyelitis Optica Spectrum Disorders. Neurology (2015) 85:177–89. doi: 10.1212/wnl.0000000000001729

  • CrossRef
  • Google Scholar

• 18

  SchumacherGABeebeGKiblerRFKurlandLTKurtzkeJFMcdowellFet al. Problems of Experimental Trials of Therapy in Multiple Sclerosis: Report by the Panel on the Evaluation of Experimentla Trials of Therapy in Multiple Sclerosis. Ann N Y Acad Sci (1965) 122:552–68. doi: 10.1111/j.1749-6632.1965.tb20235.x

  • CrossRef
  • Google Scholar

• 19

  FrankEHarrellJ. Rms: Regression Modeling Strategies. Nashville: R Package (2021). Available at: http://CRAN.Rproject.org/package=rms. version 6.2-0.

  • Google Scholar

• 20

  TherneauTMLumleyTElizabethACynthiaC. Survival. Rochester: R Package (2021). Available at: http://CRAN.Rproject.org/package=survival. version 3.2-10.

  • Google Scholar

• 21

  VickersAJElkinEB. Decision Curve Analysis: A Novel Method for Evaluating Prediction Models. Med Decis Mak (2006) 26:565–74. doi: 10.1177/0272989X06295361

  • CrossRef
  • Google Scholar

• 22

  AmorimLDCaiJ. Modelling Recurrent Events: A Tutorial for Analysis in Epidemiology. Int J Epidemiol (2015) 44:324–33. doi: 10.1093/ije/dyu222

  • CrossRef
  • Google Scholar

• 23

  WangYLiJXiaYGongRWangKYanZet al. Prognostic Nomogram for Intrahepatic Cholangiocarcinoma After Partial Hepatectomy. J Clin Oncol (2013) 31:1188–95. doi: 10.1200/JCO.2012.41.5984

  • CrossRef
  • Google Scholar

• 24

  HuangYQLiangCHHeLTianJLiangCChenXet al. Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. J Clin Oncol (2016) 34:2157–64. doi: 10.1200/JCO.2015.65.9128

  • CrossRef
  • Google Scholar

• 25

  JiaoYFryerJPLennonVAJenkinsSMQuekAMLSmithCYet al. Updated Estimate of AQP4-IgG Serostatus and Disability Outcome in Neuromyelitis Optica. Neurology (2013) 81:1197–204. doi: 10.1212/WNL.0b013e3182a6cb5c

  • CrossRef
  • Google Scholar

• 26

  KunchokAMalpasCNytrovaPHavrdovaEKAlroughaniRTerziMet al. Clinical and Therapeutic Predictors of Disease Outcomes in AQP4-IgG+ Neuromyelitis Optica Spectrum Disorder. Mult Scler Relat Disord (2020) 38:101868. doi: 10.1016/j.msard.2019.101868

  • CrossRef
  • Google Scholar

• 27

  KimSMParkJKimSHParkSYKimJYSungJJet al. Factors Associated With the Time to Next Attack in Neuromyelitis Optica: Accelerated Failure Time Models With Random Effects. PloS One (2013) 8:e82325. doi: 10.1371/journal.pone.0082325

  • CrossRef
  • Google Scholar

• 28

  Cobo-CalvoARuizAMaillartEAudoinBZephirHBourreBet al. Clinical Spectrum and Prognostic Value of CNS MOG Autoimmunity in Adults: The MOGADOR Study. Neurology (2018) 90:e1858–69. doi: 10.1212/WNL.0000000000005560

  • CrossRef
  • Google Scholar

• 29

  SeokJMChoHJAhnSWChoEBParkMSJooISet al. Clinical Characteristics of Late-Onset Neuromyelitis Optica Spectrum Disorder: A Multicenter Retrospective Study in Korea. Mult Scler (Houndmills Basingstoke England) (2017) 23:1748–56. doi: 10.1177/1352458516685416

  • CrossRef
  • Google Scholar

• 30

  Stiebel-KalishHHellmannMAMimouniMPaulFBialerOBachMet al. Does Time Equal Vision in the Acute Treatment of a Cohort of AQP4 and MOG Optic Neuritis? Neurol Neuroimmunol Neuroinflamm (2019) 6:e572. doi: 10.1212/NXI.0000000000000572

  • CrossRef
  • Google Scholar

• 31

  Cobo-CalvoARuizARollotFArrambideGDeschampsRMaillartEet al. Clinical Features and Risk of Relapse in Children and Adults With Myelin Oligodendrocyte Glycoprotein Antibody-Associated Disease. Ann Neurol (2021) 89:30–41. doi: 10.1002/ana.25909

  • CrossRef
  • Google Scholar

• 32

  BalachandranVPGonenMSmithJJDeMatteoRP. Nomograms in Oncology: More Than Meets the Eye. Lancet Oncol (2015) 16:e173–80. doi: 10.1016/S1470-2045(14)71116-7

  • CrossRef
  • Google Scholar