A Machine Learning Approach to Predict Remission in Patients With Psoriatic Arthritis on Treatment With Secukinumab

Background Psoriatic Arthritis (PsA) is a multifactorial disease, and predicting remission is challenging. Machine learning (ML) is a promising tool for building multi-parametric models to predict clinical outcomes. We aimed at developing a ML algorithm to predict the probability of remission in PsA patients on treatment with Secukinumab (SEC). Methods PsA patients undergoing SEC treatment between September 2017 and September 2020 were retrospectively analyzed. At baseline and 12-month follow-up, we retrieved demographic and clinical characteristics, including Body Mass Index (BMI), disease phenotypes, Disease Activity in PsA (DAPSA), Leeds Enthesitis Index (LEI) and presence/absence of comorbidities, including fibromyalgia and metabolic syndrome. Two random feature elimination wrappers, based on an eXtreme Gradient Boosting (XGBoost) and Logistic Regression (LR), were trained and validated with 10-fold cross-validation for predicting 12-month DAPSA remission with an attribute core set with the least number of predictors. The performance of each algorithm was assessed in terms of accuracy, precision, recall and area under receiver operating characteristic curve (AUROC). Results One-hundred-nineteen patients were selected. At 12 months, 20 out of 119 patients (25.21%) achieved DAPSA remission. Accuracy and AUROC of XGBoost was of 0.97 ± 0.06 and 0.97 ± 0.07, overtaking LR (accuracy 0.73 ± 0.09, AUROC 0.78 ± 0.14). Baseline DAPSA, fibromyalgia and axial disease were the most important attributes for the algorithm and were negatively associated with 12-month DAPSA remission. Conclusions A ML approach may identify SEC good responders. Patients with a high disease burden and axial disease with comorbid fibromyalgia seem challenging to treat.


INTRODUCTION
Psoriatic Arthritis (PsA) is a heterogeneous condition in its clinical presentation and disease course. Immune dysregulation, with altered cytokines expression and cellular phenotypes, is responsible for the typical clinical features of PsA, which involve both peripheral joints and the axial skeleton (1,2). In addition to such manifestations, patients with PsA can often suffer from extra-articular manifestations, which are genetically and immunologically correlated to these features and include psoriasis, inflammatory bowel diseases, and uveitis (3,4). PsA pathogenesis is complex, with nonlinear interaction between genetic and environmental factors, such as obesity and trauma, triggering the preclinical activation of the immune system in patients with psoriasis, mainly involving the activation of the interleukin (IL-)23-IL-17 axis, (2). By interfering with such a pathway, novel therapies with biotechnological agents enable the complete clearing of psoriasis in most patients but a PsA good control in only half the patients (2). Hence, profiling PsA patients more likely to benefit from IL-17 blockade is an unmet clinical need.
Secukinumab (SEC), a human monoclonal antibody that directly inhibits interleukin-17A, has demonstrated efficacy in patients with PsA in phase III FUTURE studies (5), but only sparse data exist about predictors of disease remission in PsA patients in real-life settings (6,7). While logistic regression (LR) was the algorithm of choice to find independent predictors in multivariable models, it must be noted that in previous real-life studies, the hypotheses were usually based on the unreal assumption that the association between the prognostic factors and PsA remission is direct and isolated (8)(9)(10).On the contrary, LR is not suitable for modelling non-independent variables (6), being inadequate to explicitly describe the complex relationship between prognostic factors and remission for complex multifactorial diseases such as PsA. Additionally, previous multivariable models often lacked rigorous internal and external validation (11,12), leaving internal consistency unchecked and raising doubts on model validity in the general PsA population.
Machine learning (ML) is emerging as a promising tool for implementing complex multi-parametric decision algorithms. Supervised ML algorithms have proven effective in predicting treatment responses and disease progression in patients with rheumatic diseases and can handle complex, non-linear relationships between patient attributes that are difficult to model with traditional statistical methods (13,14). Therefore, this study aimed to assess whether a ML approach may be useful to identify those patients more likely to achieve disease remission on SEC.

Data Gathering
Patients with classified PsA according to CASPAR criteria referred to a tertiary centre who underwent Secukinumab therapy from September 2017 to December 2020 with at least 12 months of continuous treatment were included in the analysis. Demographic (age and gender), laboratory (Erythrocyte Sedimentation Rate (ESR), C Reactive Protein (CRP) serum level) and clinical characteristics were retrospectively gathered either at baseline, and 12-month follow up. We recorded the presence/absence of comorbidities [fibromyalgia (FMS) and metabolic syndrome (MetSyn)], Body Mass Index (BMI), disease duration, treatment line, disease domain (oligoarthritis VS polyarthritis, axial involvement), Disease Activity Index in Psoriatic Arthritis (DAPSA), Leeds Enthesitis Index (LEI). Bath Ankylosing Spondylitis Activity index (BASDAI), Psoriasis Area Severity Index (PASI), concomitant conventional synthetic anti-rheumatic drug (csDMARDs) and Health Assessment Questionnaire Disability Index (HAQ-DI). Axial involvement was defined as the presence of radiographic sacroiliitis (as for clinician and/or radiologist judgement) and/or MRI inflammatory changes to the spine and/ or sacroiliac joints, both in the presence of inflammatory pain. The study was approved and reviewed by the local Ethical Committee (Biopure registry, IRB Approval n.5940, Azienda Ospedaliera Universitaria di Bari). This study followed STARD guidelines and the TRIPOD statement. All patients provided written informed consent.

Outcome of Interest
The predictive modelling analysis aimed at forecasting the probability of DAPSA remission at 12 months from Secukinumab onset, represented as a Boolean variable.

Traditional Statistics
Student's t-test or Mann Whitney U test was used as appropriate to identify differences among continuous variables between groups. c 2 test was used to determine the difference among Boolean variables in a contingency table. The significance level at a=0.05 was used.

Attributes Selection
For classification with small training samples and high dimensionality, feature selection plays an essential role in avoiding overfitting and improving classification performance. One commonly used feature selection method for small samples problems is the wrapper feature selection using the recursive feature elimination (RFE) algorithm (1). RFE needs an algorithm to be embedded. Provided with a model with feature coefficients (e.g. regression) or importance factors (e.g. tree algorithms), RFE starts from all features and gradually eliminates the least important feature. Once all features are removed, the algorithm returns the subset that gives the best performance (backward selection). RFE can generate different subsets of features based on various criteria. The subgroup generated in each step will be used to build a model and train the learning algorithm iteratively. This is achieved by fitting the given machine learning algorithm used in the RFE core, ranking features by importance, discarding the least important features, and refitting the model (Supplementary Material). We point out that, by design, the RFE algorithm never relies on validation data to achieve this result. We repeated this process five times using different random seeds to check feature stability for each algorithm. Most often selected attributes were considered part of the final attribute core set (Supplementary Material). The analysis was implemented in a Python 3.9 environment using scikit-learn (ver. 0.22.1) and XGBoost (ver. 1.1.0) libraries (12). Two different linear and nonlinear classifiers were used to train and validate the RFE with 10-fold cross-validation for predicting DAPSA remission. For further details regarding cross-validation, see Supplementary Material. For linear modelling, an LR was used; for nonlinear modelling, a decision tree-based algorithm, namely extreme gradient boosting (XGBoost) (15), was tested. A repeated grid search with cross-validation was used for optimal hyperparameter tuning to maximize the classifiers' performance (Supplementary Material) (16). For each classifier, we plotted ROC curves, and then AUROC was determined. Then, based on the optimal probability cut-off [Youden's Index (17)], classifiers' performance was compared with the following metrics: Accuracy = truepositives + truenegatives truepositives + truenegatives + falsepositives + falsenegatives Recall = truepositives truepositives + falsenegatives Precision = truepositives truepositives + falsepositives Most ML methods are often referred to as black boxes for the complexity of the underlying mechanism. In recent years efforts to improve ML explainability have been made. XGboost methods allow us to obtain the relative importance of tshe prediction of each attribute. For explainability, we plotted the relative importance of each feature, including the training core set. We also determined the odds ratio (OR) for predictors included in the LR model.

Probability Calibration
A classification model generally forecasts a binary outcome for a given observation and class. In predicting, a model may output the probability of an observation belonging to each possible class (13). This case provides some flexibility in the way predictions are interpreted and presented, allowing the choice of a threshold, as mentioned above, Youden's index. For a model to be reliable, the estimated class probabilities should reflect the actual underlying probability of the sample. A diagnostic calibration curve for the candidate best classifier was also plotted to check these assumptions, and, consequently, 10-fold isotonic calibration was carried out.  Table 1.
In general, SEC was well tolerated, with 4/119 patients (3.36%) complaining of upper respiratory tract infections, not requiring treatment discontinuation and 3/119 patients (2.52%) developed rash and pruritus at the injection site. No patients complained of gastrointestinal symptoms suggestive of active inflammatory bowel disease throughout the observation period.

DISCUSSION
PsA is a complex disease. Ideally, while the clinical management of patients with only psoriasis and synovitis may seem to be linear, treatment becomes much less immediate when the clinical phenotype encompasses enthesitis, axial disease, and comorbid conditions such as FMS and MetSyn (18). Indeed, because of the heterogeneous and multifaceted presentation, there is a lack of biomarkers for diagnosis and prognosis (19). Hence, the dearth of clinical and serological predictors remains despite the growing knowledge of PsA pathogenesis. Given that, new approaches such as ML may lead to valuable insights. In this study, we retrospectively evaluated the 12 months DAPSA based remission in PsA patients on treatment with SEC and built different statistical models for predicting the achievement of DAPSA remission.
We found a significant decrease in disease activity at 12month treatment with SEC and remission occurred with a similar frequency as in other Italian and international cohorts (8,20). Furthermore, SEC was proven to improve LEI, PASI, and BASDAI at 12 months. Interestingly SEC did show a good steroid-sparing effect. We also observed a decrease in HAQ-DI, albeit non-significant, with about a fifth of our cohort achieving a HAQ-DI improvement ≥ 0.35. Such improvement is slightly less relevant than recently reported in a large multicenter Italian study (6). Our results should be interpreted in the light of our patient characteristics, depicting a sample with a long disease duration and beyond the 2 nd treatment line for the largest part, both features negatively impacting the quality of life (1). This is may also explain the high rate of patients on steroids at SEC baseline. Interestingly, considering that most of them were anti-TNF therapy inadequate responders, our data support the recent evidence that, in need of a rapid anti-inflammatory effect or bridging of therapies, a trend towards prescribing glucocorticoids in PsA patients still exists (21). We demonstrated that a nonlinear ML approach might outperform linear methods such as LR in predicting 12-month DAPSA remission in patients on SEC, based on similar attribute core sets.
Among features deemed as necessary for prediction, several attributes representative of disease activity in different domains, such as baseline DAPSA, PASI and serum acute phase reactants levels, were identified by both algorithms as negatively associated with 12-month remission. Conversely, the importance of baseline LEI in predicting DAPSA remission was underlined exclusively from the XGBoost algorithm. Of note, enthesitis may be often overlooked as entheseal assessment is not routinely performed in real-life settings (22). Indeed, the XGBoost choice might be expected, considering that enthesitis was associated with a high overall PsA burden in the DANBIO Register (22). It is noteworthy that both RFE wrappers recognized FMS as one of the most important comorbidity to take into account in managing PsA patients on SEC. FMS had been recently recognized as one of the main predictors of drug discontinuation in PsA patients on TNF inhibitors (12), with almost no PsA patients with comorbid fibromyalgia achieving clinical disease remission upon anti-TNF therapy (11). The negative impact of FMS on PsA disease measures in patients on SEC had not been thoroughly investigated yet; as noticeable at a glance by the effect size measures, such as XGBoost feature importance or LR OR, patients with FMS were less likely to achieve 12-month remission. These findings were consistent with a recent study on a cohort of Spanish PsA patients on SEC treatment, reporting the negative impact of depression on drug survival (23). However, FMS was not investigated as a predictor of drug discontinuation in such settings.
Our study also provided insights into the effectiveness of SEC in PsA patients with axial disease. Corevitas (formerly Corrona)    Registry had shown that patients with axial PsA had higher disease activity, reduced quality of life and more impaired physical function and work productivity than those without axial involvement (24). Such evidence might explain why both algorithms found axial disease as a negative predictor of DAPSA remission. In particular, XGBoost recognized such domain as the third most crucial attribute for prediction, underlining the need to routinely assess spine involvement in PsA patients. In this study, overweight patients or patients with obesity having PsA were less likely to achieve DAPSA remission. For each 1-point increase in BMI, the probability of achieving DAPSA remission decreased by 20.3%. This is consistent with a previous report showing in a large prospective cohort that obesity can hinder the clinical response to SEC (11). On the other hand, two recent reports on Spanish and Italian cohorts (6,23) showed that obese patients were less likely to discontinue SEC and a better response to SEC could be achieved in obese individuals compared with normal-weight patients. Potential unknown confounders may account for such discrepancy. However, when focusing on predictive modelling, we noticed that XGBoost overperformed LR without even considering PsA patients BMI. Unlike rheumatoid arthritis, where unidimensional relations are feasible, PsA needs a multidimensional approach to build predicting models. The superiority of ML methods considering nonlinear relationships is essential as it suggests that we should refine our modus cogitandi when dealing with PsA. Far from being a mechanistic chain of cause and effect, rather than simply switching off a specific pathway, it is conceivable that the biologic agents disrupt and modulates the interplay between components of the clinical phenotype resulting in a new balance of interactions. ML approach has the advantage of addressing the clinical issues of PsA management, considering it as a complex system underlying a multifaceted syndrome. Interestingly, the core set of attributes used for training both LR and ML algorithms includes common predictors. However, traditional linear methods such as LR may not be sensitive to perceiving how the attributes interact at baseline and during SEC treatment. When focusing on predictive modelling instead of statistical inference, tree-based algorithms such as XGBoost seem to be a better choice. Another perk of a ML approach with immediate aftermaths into clinical practice is to implement tools readily available on mobile devices and/or desktops capable of providing probability scores, which may help clinicians better identify SECgood responders, potentially saving indirect costs due to treatment failure. Some shortcomings must be acknowledged, such as the retrospective design and the relatively small sample size, even if the applied methods have proven robust also for a small-sized dataset. However, an external validation on an independent cohort is required to adopt the XGBoost algorithm in clinical practice and evaluate the attribute core set in an unbiased manner. Nevertheless, we showed that a ML approach has undisclosed potential for guiding the management of PsA patients on SEC and for unravelling the complex interactions between PsA clinical phenotype and IL-17 blockade.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The study involving human participants were reviewed and approved by the Ethics Committee of the University of Bari (Biopure registry, IRB Approval n.5940, Azienda Ospedaliera Universitaria di Bari). Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin. The patients/participants provided their written informed consent to participate in this study.

AUTHOR CONTRIBUTIONS
VV, FI, and GL conceived the study design, drafted the manuscript, and contributed to discussion. VV performed data analysis and drafted the manuscript. ST supervised machine learning methods. AA, SC, and MM collected the data and contributed to discussion. All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

FUNDING
This study received funding from Novartis Farma SPA. The funder was not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication.