A Nomogram for the Prediction of Cessation of Migraine Among Patients With Patent Foramen Ovale After Percutaneous Closure

Objective: This study aimed to develop and validate a nomogram to predict cessation of patent foramen ovale (PFO) patients with migraine headache after percutaneous closure. Methods: A total of 247 eligible patients with PFO and migraine after percutaneous closure between May, 2016 and May, 2018 were divided into a development cohort (n = 149) and a validation cohort (n = 98). The primary end point was cessation of migraine at follow-up of 1 year after the procedure measured by the Migraine Disability Assessment Score (MIDAS). In the development cohort, the LASSO regression was used data dimension reduction. A multivariable logistic regression analysis was used to develop the predicting nomogram. The performance of the nomogram was assessed by concordance index (C-index), calibration and clinical usefulness. The results were validated in the validation cohort. Results: Migraine with aura, history of antiplatelet, and the right-to-left shunt (RLS) at rest were identified as significant predictors based on the analysis of multivariate logistic regression. The nomogram incorporating these variables showed good calibration and discrimination in the development cohort with C-index of 0.906 (95% CI: 0.847–0.965), which was confirmed using the validation cohort with C-index of 0.827 (95% CI: 0.751–0.903). The nomogram showed good agreement between prediction by nomogram and actual observation. Furthermore, the decision curve indicated that the novel nomogram was clinically useful. Conclusion: The novel nomogram showed favorable predictive accuracy for cessation of migraine among patients with PFO after percutaneous closure and might provide constructive guidance in clinical decision making.


INTRODUCTION
Migraine is a complex neurological disorder that affects ∼13% of the general population between the ages of 20 and 64 years with a higher prevalence observed in females and preceded by an aura in almost 36% of patients (1,2). Migraine can pose a significantly detrimental effect on quality of life, with negative repercussions on work, social activities, and family life in healthy young adults (3). So far, its etiology and exact pathophysiology remain obscure (2,4). Growing body of evidence reveals an association between mid to large RLS through the PFO and migraine (5)(6)(7). The foramen ovale is a flap-like communication in the atria septum secundum after birth, which is present in ∼25% of adults (8). The underlying concept linking migraine and PFO is that migraine may be triggered by chemicals that reach the brain through PFO-RLS (9). The possible underlying mechanism is that the chemicals passed through the pulmonary circulation via a RLS without the metabolism in the lungs and entered the arterial circulation, which stimulates receptors in susceptible individuals in a concentration and causes the migraine phenomenon. Although the pathophysiology of migraine is not fully understood, the presence of PFO is believed to play a role in the pathophysiology.
Prior nonrandomized observational studies have demonstrated that closure of PFO-RLS reduces the frequency and severity of migraine, particularly in patients with migraine with aura, and about 70-80% of patients who underwent PFO closure reported abolishment or improvement in their migraine attacks after PFO closure (5,(10)(11)(12). However, three randomized clinical trials of PFO closure for refractory migraine have failed to reach their primary endpoints (13)(14)(15). In the PREMIUM (Prospective, Randomized Investigation to Evaluate Incidence of Headache Reduction in Subjects with Migraine and PFO Using the AMPLATZER PFO Occluder to Medical Management) trial, complete migraine remission for 1 year occurred in 8.5% in the closure group (15). In the MIST (Migraine Intervention with STARFlex Technology) trial, the primary end point of migraine headache cessation in closure group was 4.05% at followed up for 6 months (13). Even at 5-year follow-up, 8.8% patients in the PFO closure group still presented with chronic migraine (16). The available evidence suggests that not all migraine patients with PFO benefit from such treatment. Therefore, in this study, we aimed to develop and validate a nomogram to predict cessation of migraine among patients with PFO after percutaneous closure.

METHODS AND MATERIALS Participants
This study was a part of the prospective registered clinical trial (https://register.clinicaltrials.gov/; NCT02777359). The study was approved by the ethics committee of the First Affiliated Hospital of Xi'an Jiaotong University, and all patients or their relatives provided written informed consent. Between May, 2016 and May 30, 2018, a total of 285 patients referred to the department of structural heart disease in our hospital for PFO-related migraine were included. Clinical history of migraine with or without aura was defined according to the International Headache Society criteria (17). The PFO-RLS was diagnosed using contrast transesophageal echocardiography (c-TEE) and contrast transesophageal echocardiography (TEE) examination, as described in our previous studies (18,19). The grading of PFO-RLS was defined based on the following criteria: when no, 1-10 bubbles, 11-30 bubbles, and >30 bubbles (or left atrial opacity) were detected, the RLS was considered to be negative, mild, moderate, and extensive, respectively. All patients diagnosed with substantial RLS under Valsalva maneuver who underwent PFO closure were included.
Migraineurs suffered ≥2 headache days/month averaged over the past 3 months and migraine without acute change or fluctuation were included. Besides, migraineurs with near-daily migraine were excluded from this study since they will tend to report a significant headache improvement even in the absence of migraine elimination. Patients with other cardiovascular defects, pulmonary arteriovenous malformation, the presence of intracardiac thrombi, bacteremia or active infections, active endocarditis, coagulopathy, platelet disorder, portal hypertension or, contraindication to aspirin or clopidogrel were excluded. Patients during pregnancy or with history of allergy or do not wish to participate were also excluded.
The clinical data were obtained via reviewing structured questionnaire recorded these patients' medical information. These data included patient age, gender, body mass index, history of hypertension status, hyperlipidemia, cerebral infarction, transient ischemic attack, history of diabetes mellitus, smoking status, history of preventive medications, history of antiplatelet treatment, migraine with or without aura, PFO with or without atrial septal aneurysm, PFO size, occluder size, type, red blood cell count, white blood cell count, platelets count, and PFO-RLS grading at rest. The validated Migraine Disability Assessment Score (MIDAS) was used to assess the incidence and severity of migraine headache before and after the PFO closure (20). In short, this five-point questionnaire evaluates the burden of migraine episodes in the past 3 months, their impact on working, social, leisure and family activities and the perceived pain.

PFO Closure Procedure and Follow-Up
All procedures were performed with use of an Amplatzer R Septal Occluder (St. Jude Medical, Inc., St. Paul, Minn, USA) and Cardi-O-fix PFO Occluder (Starway Medical Technology, Beijing, China). The detailed surgical procedures were described in our previous study (21). After the procedure, all patients were treated with low-molecular-weight heparin at 10 U/(kg·h) for 48 h, aspirin 100 mg/day for 6 months, and clopidogrel 50-75 mg/day for 3 months following device implantation. All patients were follow-up every 3 months during the first year. All patients were followed at 3, 6, and 12 months after device implantation. During the follow-up, the MIDAS questionnaire was completed over telephone or in outpatients for each patient at each time point ± 5 days. Patients were asked to reexamine c-TTE to assess residual shunts if possible. The clinical endpoint is reached when the complete migraine remission event occurs according to the MIDAS at the follow-up at 1 year.

Variables Selection and of the Nomogram Construction
All included patients who performed the PFO closure were divided into a development cohort and a validation cohort at a ratio of 3:2 using computer-generated random numbers. In practice, the least absolute shrinkage and selection operator (LASSO) model with an L1 penalty, which reduces the regression coefficient by applying a penalty proportional to its size from a large amount of candidates (22), was used to select the optimal predictive features in risk factors from the PFO patients with migraine after closure in the development cohort. Variables with nonzero coefficients in the LASSO regression were selected. Next, multivariable logistic regression analysis was used to construct a predicting model by incorporating the variable selected in the LASSO regression model. Backward step-wise selection was applied by using the likelihood ratio test with Akaike's information criterion as the stopping rule. A P-value for a variable <0.05 in analyses was considered to indicate statistical significance, and the variable was treated as a vital prognostic variable in prognostic prediction. The identified prognostic predictors were used to develop a predicting nomogram for complete migraine remission by using the development cohort.

Assessment and Validation of the Nomogram
To quantify the discrimination performance of the novel nomogram, Harrell's concordance index (C-index) was measured (equivalent to the area under the ROC curve) (23). The Cindex was used to assess the discrimination of the nomogram in both cohorts. Comparison of the C-index of a different model based on risk of paradoxical embolism (RoPE) was performed in two cohorts (24). Calibration plots were used to explore the performance of the nomogram. The calibration plot presented mean prediction probabilities on the x-axis and mean observed probabilities on the y-axis. A calibration plot along the 45 • line would indicate a perfect calibration model between the bootstrap-predicted probabilities and the actual outcomes, any deviation above or below the 45 • line suggests under or overestimation. Hosmer-Lemeshow statistic was also used to test the calibration performance in both cohorts, which tested the fitting precision between the observed and predicted outcome, and a significant test statistic indicates that the model does not calibrate perfectly (25).

Clinical Utility
To explore the implications of the nomogram in clinical practice, we compared performance of the model to RoPE-based model using decision curve analysis (DCA) by quantifying the net benefits at different threshold probabilities in both cohorts (26). The method is based on the principle that the relative harms of false positives and false-negatives can be expressed in terms of a probability threshold. As a comprehensive method for evaluating and comparing different diagnostic and prognostic models, DCA can be used to explore whether the nomogram-assisted decisions would improve patient outcomes.

Statistical Analyses
Continuous variables were expressed as mean and SD or median (interquartile range), and categorical variables were expressed as number and proportion as appropriate. The results were compared using the chi-squared test or Fisher's exact test. Continuous variables were compared using the t-test or Mann-Whitney U test for variables with an abnormal distribution. The Wilcoxon's rank-sum test for continuous variables and the chisquared test for categorical variables were performed between two cohorts. We computed the AUC with a 95% confidence interval by using 1,000 bootstrap re-sampling. A nomogram was generated based on the multivariate prediction model using rms package in R software. ROC analysis was performed to calculate the area under curve (AUC) into evaluating the diagnostic performance of the models. All statistical tests were performed using R statistical software version 3.6.3. All statistical tests were two-tailed, and P-values < 0.05 were considered significant.

Patient Clinical Characteristics
On the basis of all the inclusion criteria, a total of 247 patients underwent transcatheter PFO closure and completed the followup at 1 year. The procedure was successful in all patients, and all patients experienced symptomatic improvement with a mean MIDAS score of 23.15 at baseline and mean MIDAS score of 5.23 at 1 year follow-up ( Figure 1A, P < 0.001). Subgroup analysis was performed to explore the probability of migraine cessation according to the different conditions, such as aura vs. no aura, ASA vs. no ASA, shunt at rest vs. no shunt at rest. As revealed in Figures 1B-F, migraineurs without aura, with ASA, without ASA, with shunt at rest, and without shunt at rest also experienced symptomatic improvement in terms of MIDAS score. In the development cohort, 91 (61.1%) patients with migraine eliminated, and 57 (58.1%) patients with migraine complete eliminated in the validation cohort. The clinical characteristics of patients with PFO and migraine in the two groups are shown in Table 1.

Feature Selection and Nomogram Construction
All available variables were fitted into the LASSO regression model. Variables with a regression coefficient nearly equal to 0 after the shrinkage process are excluded from the model, while variables with nonzero regression coefficient in the LASSO regression model are common strongly related to the outcome variable. As a result, BMI, occluder type, migraine with or without aura, history of antiplatelet, and RLS at rest with nonzero coefficients were identified (Figure 2). Multivariable logistic regression analysis began with the identified candidate variables. A logistic regression analysis identified the migraine with or without aura, history of antiplatelet, and RLS at rest as independent predictors ( Table 2). The model that incorporated the above independent variables was used to develop a nomogram (Figure 3).

Calibration and Validation of the Nomogram
The performance of novel nomogram was validated in the two cohorts, and was compared with the RoPE-based model. Discrimination was excellent for the novel model with AUC of 0.905 (95% CI: 0.847-0.965), sensitivity of 0.828, and specificity of 1 in the development cohort. The discriminative power is obviously higher than the RoPE-based model with AUC of 0.566 ( Figure 4A). Additionally, the satisfactory discrimination was confirmed in the validation cohort with AUC of 0.827 (95% CI: 0.751-0.903), sensitivity of 0.756, and specificity of 0.982 (Figure 4B). The discrimination was still higher than the RoPE-based model with AUC of 0.507. Therefore, in the novel risk nomogram, apparent performance showed a good prediction capability.
The shape of the slope on the calibration plots demonstrate that the novel nomogram is well calibrated in the development cohort ( Figure 5A). The Hosmer and Lemeshow statistical test for the observed data for the nomogram supported the goodnessof-fit of the model (P = 0.101) in the development cohort. Furthermore, the Hosmer-Lemeshow goodness-of-fit test yielded a nonsignificant statistic (P = 0.847) in the validation cohort (Figure 5B), which demonstrated that there was no departure from perfect fit. Therefore, the novel nomogram performed well using both the development and validation cohorts.

Clinical Usefulness of the Nomogram
The clinical usefulness of nomogram was estimated using decision curve analysis (DCA) by quantifying the net benefits for a range of threshold probabilities. The DCA for the established nomogram and that for the model with RoPE integrated is presented in Figures 6A,B. The novel model showed a higher net benefit with a broader range of threshold probability. The decision curve showed that using the nomogram to predict elimination of migraine among patients with PFO after percutaneous closure adds more benefit than either the treat-allpatients strategy or the treat-none strategy, which indicated the novel nomogram to be clinical useful.

DISCUSSION
Migraine is among the most common neurological disorder and has been acknowledged as one of the most important causes of disability burden that often significantly affect patients' lives (27). Migraine and the presence of a PFO have been  associated with each other, although the exact pathophysiological mechanisms linking migraine headache and PFO remain unclear (28). Observational studies reveal that some improvement of migraine would be expected if the PFO were to be closed. The extent of improvement or elimination of migraine headache attack symptoms was variable. There are other studies reporting the useless PFO closure in migraine with and without aura. A prospective study of 80 PFO patients with migraine presented a significant benefit with PFO closure at 2-year follow up, with complete relief in 96.8% of the patients with migraine aura, and significant improvement of migraine in 87.5% of all patients with migraine regardless of with or without aura (29). In a retrospective cohort study, 150 patients suffered from migraine, including 96 patients with migraine with aura, were subjected to PFO closure. After a median follow-up of 4.9 years, among migraine with aura, headaches disappeared in 39% and improved in 49%; while among migraine without aura, headaches disappeared in 26% and improved by at least 50% in 42% patients (30). A recent meta-analysis included eight studies demonstrated a significant improvement of migraine in 81% of migraine with aura patients compared to only 63% of migraine without aura patients (31). According to 20 observational studies that explored the effect of PFO closure on migraine, the elimination of migraine was reported in 10-83% of patients, improvement in 14-83%, and no change in 1-54% patients (32). All three trials failed to meet their primary end points defined as migraine cessation and >50% reduction in migraine days at 1 year (33). Therefore, identifying factors affecting remission and constructing a nomogram is important to predict prognosis of migraine patients with PFO undergoing percutaneous closure. Recent years, nomograms are commonly used as prognostic tools in oncology and medicine. Nomograms provide improved individual prediction accuracy of future clinical outcome by combining the effects of various variables associated with these events to aid better clinical decision making (34). The present study was the first to apply a nomogram to predict cessation of migraine among patients with PFO after percutaneous closure. In this study, we developed and validated a novel  relatively accurate prediction tool for cessation of migraine among patients with PFO after percutaneous closure using a prospective cohort study. The nomogram incorporates three items of the migraine with or without aura, history of antiplatelet, and RLS at rest. Incorporating these easily available factors into an easy-to-use nomogram facilitates the PFO individualized prediction of migraine elimination after percutaneous closure. The nomogram performed well in predicting outcome, and its prediction was supported by the C-index (0.906 and 0.827 for the development and validation cohorts, respectively) and the calibration curve. When compared with the RoPE-based model, the nomogram demonstrated better predictive accuracy for cessation of migraine. Furthermore, the DCA showed that using the nomogram to predict elimination of migraine among patients with PFO after percutaneous closure brings more benefit than the RoPE-based model. In this study, three significant predictors (migraine with or without aura, RLS at rest, and history of antiplatelet) were identified using a multivariable logistic regression analysis model.
There was an obvious difference between the presence of PFO in migraine with aura and migraine without aura. Migraine with aura cessation ranged from 28.6 to 92.3%, whereas migraine without aura elimination varied from 13.6 to 82.9%. Migraine with aura improvement ranged from 4.17 to 64.3%, and migraine without aura improvement varied from 0 to 68.2% (28). PFO closure benefit migraine patients with aura and show better results and aura may be reduced, which demonstrated in observational and RCT studies (33,35). As we known, 5hydroxytryptamine (5HT) has an important role in migraine, through high 5HT venous concentrations as a result of platelet activation, and drugs that act on 5HT receptors are widely used to treat migraine. A randomized study in the UK demonstrated that aspirin, which has an antiplatelet action, weakened migraine compared with controls (36). Migraine with aura after PFO closure in the first few weeks is terminated if the patient is prescribed clopidogrel 75 mg/d for the first 4 weeks after the closure procedure (37). A previous study revealed that MIDAS scores among the migraine patients with moderate or large PFO-RLS were notably higher than those in patients from the mild PFO-RLS group and non-RLS group (P < 0.05) (38).
A history of antiplatelet also found contributed to the relief of migraine. For example, a previous study had reported clopidogrel, a thienopyridine platelet P2Y12 receptor inhibitor, could reduce migraine headache symptoms in certain patients with PFO, which demonstrating a platelet-based mechanism/trigger (39). Clopidogrel may have a primary prophylactic role in migraine patients, and could contribute to select patients who would benefit from PFO closure (40). A recent study further confirmed that clopidogrel could use as an effective complementary prophylactic for PFO migraineurs with poor response to routine prophylactics (41). Furthermore, FIGURE 3 | Nomogram predicting the probability of cessation of migraine at 1 year follow-up in patients with migraine after PFO closure. Using the variable of RLS at rest, a vertical line can be drawn from that variable to the points scale. After repeating the process for other variables, the scores for each variable can be summed and located on the "Total Points" axis. Finally, a vertical line can be drawn straight down from the plotted total point axis to the probability axis to locate the possibility of migraine cessation.   transcatheter PFO closure was effective at improving moderate and severe migraine symptoms in patients with large RLS shunt as well as the elimination of aura (42). All these evidence helps to explain the role of the identified predictors in the model.
Although this is the first nomogram to predict cessation of migraine among patients with PFO after percutaneous closure, the limitations should be considered. First, the sample size is rather small. However, this is the first study developing and validating a nomogram in migraine patients undergoing PFO closure. In addition to being a possible component in medication overuse/chronic migraine, antiplatelet therapy with salicylate may also be a confounder regarding the outcome as all patients were given this after intervention and salicylate daily may be expected to affect migraine either through its antimigraine effect. Besides, it is a single-center analysis, and a multicenter validation study should be performed to confirm the performance of the novel model in future studies.

CONCLUSION
We developed and validated a nomogram incorporating migraine with or without aura, history of antiplatelet, and RLS at rest to predict elimination of migraine among patients with PFO after percutaneous closure in a Chinese population. The novel nomogram showed favorable performance and might provide constructive guidance in clinical decision making.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The study was approved by the ethics committee of the First Affiliated Hospital of Xi'an Jiaotong University, and all patients or their relatives provided written informed consent.

AUTHOR CONTRIBUTIONS
EZ and HX is the principle investigator, conducted statistical analysis and draft the manuscript. EZ performed data management and bioinformatics analysis. EZ, HX, and YZ edited and revised the manuscript. All authors contributed to the article and approved the submitted version.