SYSTEMATIC REVIEW article

Front. Immunol., 10 May 2024

Sec. Autoimmune Disorders

Volume 15 - 2024 | https://doi.org/10.3389/fimmu.2024.1395993

The association between the neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, and monocyte-to-lymphocyte ratio and systemic sclerosis and its complications: a systematic review and meta-analysis

  • 1. Department of Biomedical Sciences, University of Sassari, Sassari, Italy

  • 2. Discipline of Clinical Pharmacology, College of Medicine and Public Health, Flinders University, Bedford Park, SA, Australia

  • 3. Department of Clinical Pharmacology, Flinders Medical Centre, Southern Adelaide Local Health Network, Bedford Park, SA, Australia

Abstract

Introduction:

The identification of new, easily measurable biomarkers might assist clinicians in diagnosing and managing systemic sclerosis (SSc). Although the full blood count is routinely assessed in the evaluation of SSc, the diagnostic utility of specific cell-derived inflammatory indices, i.e., neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), and monocyte-to-lymphocyte ratio (MLR), has not been critically appraised in this patient group.

Methods:

We conducted a systematic review and meta-analysis of studies investigating the NLR, PLR, and MLR, in SSc patients and healthy controls and in SSc patients with and without relevant complications. PubMed, Scopus, and Web of Science were searched from inception to 23 February 2024. Risk of bias and certainty of evidence were assessed using validated tools.

Results:

In 10 eligible studies, compared to controls, patients with SSc had significantly higher NLR (standard mean difference, SMD=0.68, 95% CI 0.46 to 0.91, p<0.001; I2 = 74.5%, p<0.001), and PLR values (SMD=0.52, 95% CI 0.21 to 0.83, p=0.001; I2 = 77.0%, p=0.005), and a trend towards higher MLR values (SMD=0.60, 95% CI -0.04 to 1.23, p=0.066; I2 = 94.1%, p<0.001). When compared to SSc patients without complications, the NLR was significantly higher in SSc with interstitial lung disease (ILD, SMD=0.31, 95% CI 0.15 to 0.46, p<0.001; I2 = 43.9%, p=0.11), pulmonary arterial hypertension (PAH, SMD=1.59, 95% CI 0.04 to 3.1, p=0.045; I2 = 87.6%, p<0.001), and digital ulcers (DU, SMD=0.43, 95% CI 0.13 to 0.74, p=0.006; I2 = 0.0%, p=0.49). The PLR was significantly higher in SSc patients with ILD (SMD=0.42, 95% CI 0.25 to 0.59, p<0.001; I2 = 24.8%, p=0.26). The MLR was significantly higher in SSc patients with PAH (SMD=0.63, 95% CI 0.17 to 1.08, p=0.007; I2 = 66.0%, p=0.086), and there was a trend towards a higher MLR in SSc patients with ILD (SMD=0.60, 95% CI -0.04 to 1.23, p=0.066; I2 = 94.1%, p<0.001).

Discussion:

Pending the results of appropriately designed prospective studies, the results of this systematic review and meta-analysis suggest that blood cell-derived indices of inflammation, particularly the NLR and PLR, may be useful in the diagnosis of SSc and specific complications.

Systematic review registration:

https://www.crd.york.ac.uk/PROSPERO/, identifier CRD42024520040.

Introduction

Systemic sclerosis (SSc) is a chronic autoimmune condition that affects primarily women and is characterized by progressive fibrosis of the skin and various organs and systems as well as vascular dysfunction (, ). The estimated incidence and prevalence of SSc globally are between 8 and 56 new cases per million persons per year and between 38 and 341 cases per million persons, respectively (). The diagnosis and overall clinical evaluation of patients with SSc is based on specific findings on physical examination and serological abnormalities (). Such abnormalities include a positive antinuclear antibody (), anti-topoisomerase I antibody (), anticentromere antibody (), anti-RNA polymerase III antibody (), and antibodies to Th/To (). However, the diagnosis of SSc is not always straightforward given the overlap with other autoimmune conditions, particularly in the early stages of the disease (–). These challenges have prompted the search for novel biomarkers of SSc and its complications, e.g., interstitial lung disease (ILD), pulmonary arterial hypertension (PAH), and digital ulcers (DU), to enhance diagnosis and management (–).

An emerging set of inflammatory biomarkers in various autoimmune disorders is represented by specific indices derived from blood cell types that are routinely assessed as part of a full blood count. These cell types, particularly neutrophils, platelets, lymphocytes, and monocytes, have been shown to play an important role in the pathophysiology of SSc in experimental and clinical studies (–). A relatively higher neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), and monocyte-to-lymphocyte ratio (MLR) have been shown to successfully discriminate between the presence and the absence of specific autoimmune conditions as well as the presence and absence of active disease in those affected, e.g., rheumatoid arthritis (–), psoriasis (, ), and systemic lupus erythematosus (–). However, the potential diagnostic role of the NLR, PLR, and MLR in SSc and relevant complications has not been critically appraised.

Therefore, we sought to address this issue by conducting a systematic review and meta-analysis of studies investigating the NLR, PLR, and MLR in patients with SSc and healthy controls and in SSc patients with and without specific complications. We hypothesized that the NLR, PLR, and MLR were significantly higher in SSc patients vs. controls and in SSc patients with complications vs. SSc patients without. Where possible, we conducted meta-regression and subgroup analyses to investigate possible associations between the effect size of the between-group differences in these hematological cell indices and pre-defined study and patient characteristics.

Materials and methods

Search strategy and study selection

We conducted a systematic search for relevant publications in the electronic databases PubMed, Scopus, and Web of Science, from inception to 23 February 2024, using the following terms: “systemic sclerosis” OR “scleroderma” OR “SSc” and “neutrophil to lymphocyte ratio” OR “neutrophil-to-lymphocyte ratio” OR “NLR” OR “platelet to lymphocyte ratio” OR “platelet-to-lymphocyte ratio” OR “PLR” OR “monocyte-to-lymphocyte ratio” OR “monocyte to lymphocyte ratio” OR “MLR”. Two investigators independently performed a review of the abstracts and full text of the publications based on pre-specified inclusion criteria: (i) the investigation of the NLR and/or PLR and/or MLR in patients with SSc diagnosed according accepted guidelines and healthy controls in a case-control study, (ii) the assessment of the NLR and/or PLR and/or MLR in SSc patients with or without specific complications in a case-control study, (iii) the recruitment of adult participants, and (iv) the availability of the full text of the publication in English language. The two investigators also hand searched the references of individual publications to identify additional studies.

The two investigators independently extracted the following variables from each article: year of publication, first author, study design, country where the study was conducted, sample size, age, male to female ratio, NLR, PLR, and MLR values, and presence of relevant complications.

We assessed the risk of bias, the certainty of evidence, and presence of publication bias using validated tools (–). We fully adhered to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement (Supplementary Table 1) (), and registered the study protocol in an official repository (PROSPERO registration number: CRDCRD42024520040).

Statistical analysis

We calculated standardized mean differences (SMDs) and 95% confidence intervals (CIs) and created forest plots to assess possible differences in NLR, PLR, and MLR value between SSc patients and healthy controls and between SSc patients with and without complications (p-value for significance set at <0.05). Data transformations to obtain means and standard deviations from medians and interquartile ranges or medians and ranges were performed using established methods (). Heterogeneity of the SMD was assessed using the Q-statistic (significance level at p<0.10). A random-effect model was used for meta-analyses with high heterogeneity (, 44). Sensitivity analysis was assessed using conventional methods (45).

Meta-regression and subgroup analyses were conducted to assess possible associations between the effect size and study design, study country, age, male to female ratio, and presence of complications. We used Stata 14 for all statistical analyses (Stata Corp., College Station, TX, USA).

Results

The PRISMA flow chart of the study selection is described in Figure 1. After initially identifying 321 articles, 308 were excluded because they were either duplicates or presented data that were not relevant to the study question. After fully revising the remaining13 articles, a further three were excluded because the study design was not case-control. Therefore, ten studies, all with a low risk of bias, were selected for analysis (46–55) (Table 1, Supplementary Table 2). SSc patients did not receive any treatment in three studies (48, 49, 51), 15% received treatment with corticosteroids in one study (47), 33.3%, 21.1%, and 13.2% received treatment with corticosteroids, penicillamine, and methotrexate, respectively, in one study (50), whereas relevant information regarding treatment was not reported in the remaining five studies (46, 52–55). The cross-sectional design of the studies identified downgraded the initial certainly of evidence to low.

Figure 1

Table 1

StudyHealthy controlsPatients with systemic sclerosisStudy
design
nAge
(Years)
M/FNLR
PLR
MLR
(Mean ± SD)
nAge
(Years)
M/FNLR
PLR
MLR
(Mean ± SD)
Esheba NE et al., 2016, Egypt (46)25402/231.52 ± 0.54
NR
NR
25412/232.34 ± 1.04
NR
NR
R
Yolbas S et al., 2016, Turkey (47)554511/442.43 ± 0.99
151 ± 56
NR
39494/356.40 ± 4.43
306 ± 196
NR
P
Yang Z et al., 2017, China (49)1704519/1511.80 ± 0.76
NR
0.21 ± 0.05
33554/292.33 ± 0.69
NR
0.28 ± 0.09
R
Jung JH et al., 2017, South Korea (48)50497/432.00 ± 1.07
126 ± 42
NR
885010/783.95 ± 6.59
164 ± 101
NR
R
Kim A et al., 2020, South Korea (50)304550/3041.32 ± 0.49
120 ± 29
NR
114570/1141.89 ± 0.95
129 ± 46
NR
P
Tezcan D et al., 2020, Turkey (51)129518/1211.59 ± 0.36
114 ± 37
0.19 ± 0.05
129527/1222.18 ± 1.13
136 ± 72
0.29 ± 0.12
R
Yayla ME et al., 2020, Turkey (52)50499/411.75 ± 1.64
NR
0.16 ± 0.25
69537/622.40 ± 5.78
NR
0.21 ± 0.59
R
Sakr BR et al., 2021, Egypt (53)45413/422.45 ± 1.41
NR
NR
35436/294.24 ± 2.45
NR
NR
P
Li H et al., 2022, China (54)NRNRNRNR
NR
NR
2275339/1882.59 ± 1.42
NR
NR
R
Nejatifar F et al., 2023, Iran (55)123NR57/662.10 ± 0.80
NR
0.2 ± 0.2
123NR14/1092.50 ± 1.40
NR
0.2 ± 0.1
R

Characteristics of the studies investigating the NLR, PLR, and MLR values in patients with systemic sclerosis and healthy controls.

NR, not reported; MLR, monocyte-to-lymphocyte ratio; NLR, neutrophil-to-lymphocyte ratio; PLR, platelet-to-lymphocyte ratio.

Neutrophil-to-lymphocyte ratio

Nine studies reported the NLR in a total of 655 SSc patients (mean age 52 years, 92% females) and 951 healthy controls (mean age 50 years, 88% females) (46–53, 55). Three studies were conducted in Turkey (47, 51, 52), two in Egypt (46, 53), two in South Korea (48, 50), one in China (49), and one in Iran (55). Six studies were retrospective (46, 48, 49, 51, 52, 55), and three prospective (47, 50, 53).

The forest plot showed that the NLR values were significantly higher in SSc patients when compared to controls (SMD=0.68, 95% CI 0.46 to 0.91, p<0.001; I2 = 74.5%, p<0.001; Figure 2), with stable results in sensitivity analysis (corresponding pooled SMD ranging between 0.61 and 0.75; Supplementary Figure 1).

Figure 2

There was no evidence of publication bias (Begg’s test, p=0.75; Egger’s test, p=0.60). The “trim-and-fill” method identified two missing studies to be added to the left side of the funnel plot to ensure symmetry (Supplementary Figure 2). The resulting effect size remained significant (SMD=0.59, 95% CI 0.36 to 0.82, p<0.001).

In subgroup analysis, the pooled SMD was not significantly different (p=0.37) between studies in patients aged ≤50 years (SMD=0.89, 95% CI 0.43 to 1.35, p<0.001; I2 = 75.3%, p=0.007), and >50 years (SMD=0.63, 95% CI 0.34 to 0.92, p<0.001; I2 = 73.7%, p=0.010; Supplementary Figure 3). The pooled SMD was not significantly different (p=0.24) between studies with (SSc patient males/SSc patient females)/(control males/control females) ratio <1 (SMD=0.56, 95% CI 0.23 to 0.89, p=0.001; I2 = 81.3%, p<0.001) and (SSc patient males/SSc patient females)/(control males/control females) ratio ≥1 (SMD=0.86, 95% CI 0.69 to 1.03, p<0.001; I2 = 0.0%, p=0.82; Supplementary Figure 4), with a virtually absent between study variance in the second subgroup. Non-significant differences (p=0.65) were also observed between studies conducted in Turkey (SMD=0.72, 95% CI 0.13 to 1.30, p<0.001; I2 = 88.0%, p<0.001), South Korea (SMD=0.64, 95% CI 0.14 to 1.14, p=0.012; I2 = 82.9%, p=0.016), Egypt (SMD=0.95, 95% CI 0.59 to 1.32, p<0.001; I2 = 0.0%, p=0.87), and other countries (SMD=0.50, 95% CI 0.16 to 0.84, p=0.004; I2 = 57.6%, p=0.12; Supplementary Figure 5), with a virtually absent heterogeneity in the Egyptian subgroup. By contrast, we observed a significant difference (p=0.034) in pooled SMD between retrospective (SMD=0.51, 95% CI 0.29 to 0.72, p<0.001; I2 = 58.0%, p=0.036) and prospective studies (SMD=1.01, 95% CI 0.74 to 1.28, p<0.001; I2 = 39.6%, p=0.19; Supplementary Figure 6), with a relatively low between-study variance in both subgroups.

The overall level of certainty remained low after considering the low risk of bias in all studies, the high but partially explainable heterogeneity, the lack of indirectness, the moderate effect size (SMD=0.68) (56), and the absence of publication bias.

Neutrophil-to-lymphocyte ratio and interstitial lung disease

Six studies reported the NLR in a total of 326 SSc patients with ILD and 324 SSc patients without ILD (46, 48, 50–52, 54) (Table 2). Two studies were conducted in Turkey (51, 52), two in South Korea (48, 50), one in China (54), and one in Egypt (46). Five studies were retrospective (46, 48, 51, 52, 54) and one prospective (50).

Table 2

ILD -ILD +PAH -PAH +DU -DU +
nNLR
PLR
MLR
(Mean ± SD)
nNLR
PLR
MLR
(Mean ± SD)
nNLR
PLR
MLR
(Mean ± SD)
nNLR
PLR
MLR
(Mean ± SD)
nNLR
PLR
MLR
(Mean ± SD)
nNLR
PLR
MLR
(Mean ± SD)
Esheba NE et al., 2016, Egypt (46)82.00 ± 0.91
NR
NR
152.51 ± 0.98
NR
NR
222.15 ± 0.42
NR
NR
33.9 ± 0.56
NR
NR
152.01 ± 0.82
NR
NR
102.34 ± 0.99
NR
NR
Yolbas S et al., 2016, Turkey (47)NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
Yang Z et al., 2017, China (49)NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
Jung JH et al., 2017, South Korea (48)482.12 ± 1.71
142 ± 67
NR
406.13 ± 9.18
190 ± 127
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
Kim A et al., 2020, South Korea (50)601.62 ± 0.55
116 ± 35
NR
542.24 ± 1.67
153 ± 62
NR
NRNR
NR
NR
NRNR
NR
NR
791.71 ± 0.70
120 ± 44
NR
352.26 ± 1.34
162 ± 74
NR
Tezcan D et al., 2020, Turkey (51)662.06 ± 0.90
126 ± 61
0.27 ± 0.09
632.31 ± 1.37
147 ± 83
0.32 ± 0.16
1122.10 ± 0.47
141 ± 67
0.29 ± 0.11
173.38 ± 2.80
148 ± 79
0.40 ± 0.22
NRNR
NR
NR
NRNR
NR
NR
Yayla ME et al., 2020, Turkey (52)212.40 ± 5.80
NR
0.21 ± 0.58
482.40 ± 5.30
NR
0.21 ± 0.37
642.39 ± 5.79
NR
0.08 ± 0.47
53.16 ± 4.77
NR
0.05 ± 0.09
542.31 ± 5.79
NR
0.20 ± 0.59
153.16 ± 1.98
NR
0.24 ± 0.32
Sakr BR et al., 2021, Egypt (53)NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
Li H et al., 2022, China (54)1212.50 ± 1.00
139 ± 59
NR
1062.70 ± 1.90
160 ± 72
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
Nejatifar F et al., 2023, Iran (55)NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR
NRNR
NR
NR

Characteristics of the studies investigating the NLR, PLR, and MLR in patients with systemic sclerosis with and without complications.

DU, digital ulcers; ILD, interstitial lung disease; NR, not reported; PAH, pulmonary artery hypertension.

The forest plot showed that SSc patients with ILD had higher NLR values when compared to patients without ILD (SMD=0.31, 95% CI 0.15 to 0.46, p<0.001; I2 = 43.9%, p=0.11; Figure 3). The corresponding pooled SMD values were stable in sensitivity analysis (range between 0.25 and 0.40; Supplementary Figure 7).

Figure 3

The limited number of studies prevented the assessment of publication bias and the conduct of meta-regression analysis. In sub-group analysis, the pooled SMD was significant in studies conducted in South Korea (SMD=0.64, 95% CI 0.36 to 0.92, p<0.001; I2 = 0.0%, p=0.97) but not in Turkey (SMD=0.15, 95% CI -0.14 to 0.44, p=0.30; I2 = 0.0%, p=0.49), or other countries (SMD=0.17, 95% CI -0.08 to 0.42, p=0.19; I2 = 0.0%, p=0.39; Supplementary Figure 8), with a virtually absent heterogeneity in all subgroups.

The overall level of certainty was downgraded to very low because of the lack of assessment of publication bias.

Neutrophil-to-lymphocyte ratio and pulmonary arterial hypertension

Three studies, all retrospective, investigated the NLR in a total of 198 SSc patients without PAH and 25 SSc patients with PAH (46, 51, 52) (Table 2). Two studies were conducted in Turkey (51, 52), and one in Egypt (46).

The forest plot showed that SSc patients with PAH had higher NLR values when compared to SSc patients without PAH (SMD=1.59, 95% CI 0.04 to 3.1, p=0.045; I2 = 87.6%, p<0.001; Figure 4). Assessment of sensitivity, publication bias, meta-regression and sub-group analyses could not be performed because of the small number of studies.

Figure 4

The overall level of certainty was downgraded to very low after considering the low risk of bias in all studies, the high and unexplainable heterogeneity (downgrade one level), the lack of indirectness, the large effect size (SMD=1.59, upgrade one level) (56), and the lack of assessment of publication bias (downgrade one level).

Neutrophil-to-lymphocyte ratio and digital ulcers

Three studies reported the NLR in a total of 148 SSc patients without DU and 60 SSc patients with DU (46, 50, 52) (Table 2). One study was conducted in Egypt (46), one in South Korea (50), and one in Turkey (52). Two studies were retrospective (46, 52), and one prospective (50).

The forest plot showed that the NLR values were significantly higher in SSc patients with DU when compared to SSc patients without DU (SMD=0.43, 95% CI 0.13 to 0.74, p=0.006; I2 = 0.0%, p=0.49; Figure 5). Assessment of sensitivity, publication bias, and meta-regression, and sub-group analyses could not be performed because of the small number of studies.

Figure 5

The overall level of certainty was downgraded to very low after considering the low risk of bias in all studies, the absent heterogeneity, the lack of indirectness, the moderate effect size (SMD=0.43) (56), and the lack of assessment of publication bias (downgrade one level).

Platelet-to-lymphocyte ratio

Four studies reported PLR values in a total of 370 SSc patients (mean age 53 years, 94% females) and 578 healthy controls (mean age 52 years, 95% females) (47, 48, 50, 51) (Table 2). Two studies were conducted in Turkey (47, 51), and two in South Korea (48, 50). Two studies were retrospective (48, 51), and the remaining two prospective (47, 50).

The forest plot showed the PLR values were significantly higher in SSc patients when compared to controls (SMD=0.52, 95% CI 0.21 to 0.83, p=0.001; I2 = 77.0%, p=0.005; Figure 6). Assessment of sensitivity, publication bias, meta-regression, sub-group analyses could not be performed because of the small number of studies.

Figure 6

The overall level of certainty was downgraded to very low after considering the high and unexplained heterogeneity and the lack of assessment of publication bias.

Platelet-to-lymphocyte ratio and interstitial lung disease

Four studies investigated the PLR in a total of 263 SSc patients with ILD and 295 SSc patients without ILD (48, 50, 51, 54) (Table 2). Two studies were conducted in South Korea (48, 50), one in Turkey (51), and one in China (54). Three studies were retrospective (48, 51, 54), and one prospective (50).

The forest plot showed that the PLR values were significantly higher in SSc patients with ILD when compared to SSc patients without ILD (SMD=0.42, 95% CI 0.25 to 0.59, p<0.001; I2 = 24.8%, p=0.26; Figure 7). Assessment of sensitivity, publication bias, meta-regression, and subgroup analysis could not be performed because of the small number of studies.

Figure 7

The overall level of certainty was downgraded to very low after considering the lack of assessment of publication bias.

Platelet-to-lymphocyte ratio and pulmonary arterial hypertension

One study reported that SSc patients with PAH had significantly higher PLR values when compared to SSc patients without PAH (median 147.0 (IQR:111.6) vs. 125.6 (IQR: 82.3), p<0.001) (51).

Platelet-to-lymphocyte ratio and digital ulcers

One study reported that SSc patients with DU had significantly higher PLR values when compared to SSc patients without DU (median 148.07 (IQR, 121.33 to 217.66) vs. 115.67 (IQR, 93.5 to 151.88), p=0.001) (50).

Monocyte-to-lymphocyte ratio

Four studies, all retrospective, reported the MLR in a total of 354 SSc patients (mean age 53 years, 90% females) and 472 healthy controls (mean age 48 years, 77% females) (49, 51, 52, 55). Two studies were performed in Turkey (51, 52), one in China (49), and one in Iran (55).

The forest plot showed that that MLR values were non-significantly higher in SSc patients when compared to controls (SMD=0.60, 95% CI -0.04 to 1.23, p=0.066; I2 = 94.1%, p<0.001; Figure 8). Assessment of sensitivity, publication bias, meta-regression, and subgroup analyses could not be performed because of the small number of studies.

Figure 8

The overall level of certainty was downgraded to very low after considering the high and unexplained heterogeneity and the lack of assessment of publication bias.

Monocyte-to-lymphocyte ratio and interstitial lung disease

Two studies reported the MLR in a total of 111 SSc patients with ILD and 87 SSc patients without ILD (51, 52) (Table 2). Both studies were conducted in Turkey and were retrospective.

The forest plot showed that MLR values were non-significantly higher in SSc patients with ILD when compared to SSc patients without ILD (SMD=0.27, 95% CI -0.02 to 0.55, p=0.071; I2 = 33.4%, p=0.22; Figure 9). Assessment of sensitivity, publication bias, meta-regression, and subgroup analyses could not be performed because of the small number of studies.

Figure 9

The overall level of certainty was downgraded to very low after considering the lack of assessment of publication bias.

Monocyte-to-lymphocyte ratio and pulmonary arterial hypertension

Two studies reported the MLR in a total of 176 SSc patients without PAH and 22 patients with PAH (51, 52) (Table 2). Both studies were conducted in Turkey and were retrospective.

The forest plot showed that PLR values were significantly higher in SSc patients with PAH when compared to SSc patients without PAH (SMD=0.63, 95% CI 0.17 to 1.08, p=0.007; I2 = 66.0%, p=0.086; Figure 10). Assessment of sensitivity, publication bias, meta-regression and sub-group analyses could not be performed because of the small number of studies.

Figure 10

The overall level of certainty was downgraded to very low after considering the lack of assessment of publication bias.

Monocyte-to-lymphocyte ratio and digital ulcers

One study reported that SSc patients with DU had significantly higher MLR values when compared to SSc patients without DU (median 0.24 (IQR:0.43) vs. 0.20 (IQR: 0.79), p=0.007) (52).

Discussion

The results of this systematic review and meta-analysis have shown that a) the NLR and the PLR are significantly higher in SSc patients vs. healthy controls, and b) the NLR, PLR, and MLR are significantly higher in SSc patients with specific complications (ILD, PAH, and DU for the NLR; ILD for the PLR; PAH for the MLR) vs. SSc patients without complications. Non-significant trends towards higher MLR values have also been observed in SSc patients vs. controls and in SSc patients with ILD vs. those without. Subgroup analysis for studies investigating the NLR in SSc patients and controls showed similar effect sizes regardless of age, male to female ratio, and study country. However, there was a significant difference in effect size between retrospective and prospective studies. Subgroup analysis for studies investigating the NLR in SSc patients with vs. without ILD showed differences in the significance of the effect size according to the study country. Subgroup analyses for other hematological indices and specific complications could not be conducted because of the limited number of studies. Taken together, these results suggest that specific blood cell-derived indices of inflammation, particularly the NLR and PLR, may be useful in assisting clinicians to diagnose SSc, the presence of specific complications, e.g., ILD, PAH, and DU, and potentially, to monitor the temporal progression of the disease and the response to pharmacological treatments. The negligible costs associated with the determination of the NLR, PLR, and MLR make their routine use particularly attractive in a patient group that faces significant challenges, including the lack of robust measures of SSc activity and the often fluctuating natural history of the disease (57, 58).

Recent studies have reported that the neutrophil count plays an important pathophysiological role in SSc. For example, in a study of 447 SSc patients, a higher baseline neutrophil count was significantly associated with diffuse skin disease (p<0.001), a higher baseline modified Rodnan skin thickness score (p<0.001), and a lower forced vital capacity (FVC%, p=0.03). Furthermore, a relative neutrophilia predicted lower FVC% during follow-up (point estimate -4.74, 95% CI -8.29 to -1.20, p=0.009), whereas higher lymphocyte counts were significantly associated with higher FVC% over time (point estimate 1.43, 95% CI 0.45 to 2.40, p=0.004). Notably, a higher neutrophil count also independently predicted a higher mortality (p=0.002) whereas a higher lymphocyte count independently predicted a lower mortality (p=0.001), after adjusting for age, sex, and race (). Platelets also exhibit a state of persistent activation in SSc. Such state is likely to be triggered by chronic endothelial dysfunction and vascular damage and the activation of the innate and adaptative immune systems in these patients (–). Therefore, an increase in platelet count is commonly observed in SSc patients, particularly in those with a concomitant state of chronic inflammation (). Experimental evidence also suggests an important role of monocytes in the pathophysiology of SSc (, ). Studies have shown that CD16-positive, non-classical, monocyte count, including monocytes expressing the type II interferon inducible marker, CXCL10, was higher in SSc patients when compared to healthy controls and was associated with an increased risk of fibrotic manifestations., e.g., ILD and a higher modified Rodnan skin score (–). Pending further research to identify the molecular and cellular mechanisms underpinning the detrimental effects of high neutrophil, platelet, and monocyte counts, and the potential protective effects of high lymphocyte counts, the assessment of the NLR, PLR, and MLR may optimally capture the relative alterations of these blood cell types in the assessment of patients with SSc, including the presence of specific complications. However, larger, accurately designed prospective studies are warranted to confirm our findings and to accurately determine the diagnostic performance of the NLR, PLR, and MLR, singly or in combination with clinical parameters and/or other available biomarkers, to justify their routine use in clinical practice.

One important observation in our subgroup analyses of studies investigating the NLR in SSc patients and healthy controls was that the effect size of the between-group difference was similar in studies conducted in different countries. However, it is important to emphasize that our systematic search captured studies from a limited number of countries located in Asia and Africa. Therefore, our results need confirmation in other patient populations, specifically from Europe and North and South America. This is particularly important as epidemiological studies have reported the presence of significant ethnic-based differences in the NLR in subjects without autoimmune disorders. For example, a study analyzing data from the National Health and Nutrition Examination Survey in USA reported that African American and Hispanic participants had significantly lower NLR values when compared to non-Hispanic white participants (59). Similar trends have been observed in other studies (60, 61).

Our systematic review and meta-analysis has a number of strengths, including the comprehensive assessment of three blood cell-derived inflammatory indices in patients with SSc with and without specific complications, the assessment of possible associations between the effect size and pre-defined study and patient characteristics, and a robust evaluation of the risk of bias and the certainty of evidence. Important limitations include the relatively small number of studies identified, which precluded the conduct of meta-regression and subgroup analyses for the PLR and MLR, the lack of studies investigating other important complications in patients with SSc, e.g., renal crisis, gastrointestinal and musculoskeletal involvement (), and, as previously discussed, the lack of relevant evidence in SSc patients from specific geographical locations, particularly Europe and North and South America. A further limitation is the lack of information provided on the pharmacological treatment for SSc in five studies (46, 52–55), with three additional studies investigating treatment naïve SSc patients (48, 49, 51). This prevented the assessment of the possible effect of immunosuppressive therapies on the NLR, PLR, and MLR values, a phenomenon reported in other studies in patients with and without autoimmune disorders (62, 63).

In conclusion, our systematic review and meta-analysis has shown the potential utility of blood cell-derived indices of inflammation, particularly the NLR and PLR, in the diagnosis of SSc and associated complications, monitoring of disease activity, and assessment of the effect of treatments. However, additional studies are required to confirm these observations in different ethnic groups and determine whether the NLR and PLR can enhance the diagnostic performance of clinical assessment and existing biomarkers in this patient group in routine practice.

Statements

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material. Further inquiries can be directed to the corresponding author.

Author contributions

AZ: Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – review & editing. AM: Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fimmu.2024.1395993/full#supplementary-material

References

  • 1

    AllanoreYSimmsRDistlerOTrojanowskaMPopeJDentonCPet al. Systemic sclerosis. Nat Rev Dis Primers. (2015) 1:15002. doi: 10.1038/nrdp.2015.2

  • 2

    VolkmannERAndreassonKSmithV. Systemic sclerosis. Lancet. (2023) 401:304–18. doi: 10.1016/S0140–6736(22)01692–0

  • 3

    IngegnoliFUghiNMihaiC. Update on the epidemiology, risk factors, and disease outcomes of systemic sclerosis. Best Pract Res Clin Rheumatol. (2018) 32:223–40. doi: 10.1016/j.berh.2018.08.005

  • 4

    van den HoogenFKhannaDFransenJJohnsonSRBaronMTyndallAet al. 2013 classification criteria for systemic sclerosis: an American College of Rheumatology/European League against Rheumatism collaborative initiative. Arthritis Rheum. (2013) 65:2737–47. doi: 10.1002/art.38098

  • 5

    KavanaughATomarRReveilleJSolomonDHHomburgerHA. Guidelines for clinical use of the antinuclear antibody test and tests for specific autoantibodies to nuclear antigens. American College of Pathologists. Arch Pathol Lab Med. (2000) 124:71–81. doi: 10.5858/2000-124-0071-GFCUOT

  • 6

    HuPQFertigNMedsgerTAJr.WrightTM. Correlation of serum anti-DNA topoisomerase I antibody levels with disease severity and activity in systemic sclerosis. Arthritis Rheumatol. (2003) 48:1363–73. doi: 10.1002/art.10977

  • 7

    van LeeuwenNMBoonstraMBakkerJAGrummelsAJordanSLiemSet al. Anticentromere antibody levels and isotypes and the development of systemic sclerosis. Arthritis Rheumatol. (2021) 73:2338–47. doi: 10.1002/art.41814

  • 8

    NikpourMHissariaPByronJSahharJMicallefMPaspaliarisWet al. Prevalence, correlates and clinical usefulness of antibodies to RNA polymerase III in systemic sclerosis: a cross-sectional analysis of data from an Australian cohort. Arthritis Res Ther. (2011) 13:R211. doi: 10.1186/ar3544

  • 9

    SureshSCharltonDSnellEKLaffoonMMedsgerTAJr.ZhuLet al. Development of pulmonary hypertension in over one-third of patients with th/to antibody-positive scleroderma in long-term follow-up. Arthritis Rheumatol. (2022) 74:1580–7. doi: 10.1002/art.42152

  • 10

    RabinovichCE. Challenges in the diagnosis and treatment of juvenile systemic sclerosis. Nat Rev Rheumatol. (2011) 7:676–80. doi: 10.1038/nrrheum.2011.148

  • 11

    McMahanZHHummersLK. Systemic sclerosis–challenges for clinical practice. Nat Rev Rheumatol. (2013) 9:90–100. doi: 10.1038/nrrheum.2012.191

  • 12

    LepriGBellando RandoneSMatucci CerinicMGuiducciS. Early diagnosis of systemic sclerosis, where do we stand today? Expert Rev Clin Immunol. (2022) 18:1–3. doi: 10.1080/1744666X.2022.2015327

  • 13

    AffandiAJRadstakeTRMarutW. Update on biomarkers in systemic sclerosis: tools for diagnosis and treatment. Semin Immunopathol. (2015) 37:475–87. doi: 10.1007/s00281–015-0506–4

  • 14

    WermuthPJPiera-VelazquezSRosenbloomJJimenezSA. Existing and novel biomarkers for precision medicine in systemic sclerosis. Nat Rev Rheumatol. (2018) 14:421–32. doi: 10.1038/s41584–018-0021–9

  • 15

    WermuthPJPiera-VelazquezSJimenezSA. Identification of novel systemic sclerosis biomarkers employing aptamer proteomic analysis. Rheumatol (Oxford). (2018) 57:1698–706. doi: 10.1093/rheumatology/kex404

  • 16

    SkaugBAssassiS. Biomarkers in systemic sclerosis. Curr Opin Rheumatol. (2019) 31:595–602. doi: 10.1097/BOR.0000000000000656

  • 17

    CavazzanaIVojinovicTAiroPFrediMCeribelliAPedrettiEet al. Systemic sclerosis-specific antibodies: novel and classical biomarkers. Clin Rev Allergy Immunol. (2023) 64:412–30. doi: 10.1007/s12016-022-08946-w

  • 18

    WareingNMohanVTaherianRVolkmannERLyonsMAWilhalmeHet al. Blood neutrophil count and neutrophil-to-lymphocyte ratio for prediction of disease progression and mortality in two independent systemic sclerosis cohorts. Arthritis Care Res (Hoboken). (2023) 75:648–56. doi: 10.1002/acr.24880

  • 19

    NtelisKBogdanosDDimitroulasTSakkasLDaoussisD. Platelets in systemic sclerosis: the missing link connecting vasculopathy, autoimmunity, and fibrosis? Curr Rheumatol Rep. (2019) 21:15. doi: 10.1007/s11926-019-0815-z

  • 20

    RamirezGAFranchiniSRovere-QueriniPSabbadiniMGManfrediAAMaugeriN. The role of platelets in the pathogenesis of systemic sclerosis. Front Immunol. (2012) 3:160. doi: 10.3389/fimmu.2012.00160

  • 21

    KoJNovianiMChellamuthuVRAlbaniSLowAHL. The pathogenesis of systemic sclerosis: the origin of fibrosis and interlink with vasculopathy and autoimmunity. Int J Mol Sci. (2023) 24:14287. doi: 10.3390/ijms241814287

  • 22

    WieloszEMajdanM. Haematological abnormalities in systemic sclerosis. Reumatologia. (2020) 58:162–6. doi: 10.5114/reum.2020.96655

  • 23

    LescoatALecureurVVargaJ. Contribution of monocytes and macrophages to the pathogenesis of systemic sclerosis: recent insights and therapeutic implications. Curr Opin Rheumatol. (2021) 33:463–70. doi: 10.1097/BOR.0000000000000835

  • 24

    CarvalheiroTHortaSvan RoonJAGSantiagoMSalvadorMJTrindadeHet al. Increased frequencies of circulating CXCL10-, CXCL8- and CCL4-producing monocytes and Siglec-3-expressing myeloid dendritic cells in systemic sclerosis patients. Inflammation Res. (2018) 67:169–77. doi: 10.1007/s00011–017-1106–7

  • 25

    LescoatALecureurVRousselMSunnaramBLBallerieACoiffierGet al. CD16-positive circulating monocytes and fibrotic manifestations of systemic sclerosis. Clin Rheumatol. (2017) 36:1649–54. doi: 10.1007/s10067–017-3597–6

  • 26

    SchneiderLMarcondesNAHaxVda Silva MoreiraIFUedaCYPiovesanRRet al. Flow cytometry evaluation of CD14/CD16 monocyte subpopulations in systemic sclerosis patients: a cross sectional controlled study. Adv Rheumatol. (2021) 61:27. doi: 10.1186/s42358–021-00182–8

  • 27

    WorrellJCO'ReillyS. Bi-directional communication: Conversations between fibroblasts and immune cells in systemic sclerosis. J Autoimmun. (2020) 113:102526. doi: 10.1016/j.jaut.2020.102526

  • 28

    ErreGLPaliogiannisPCastagnaFMangoniAACarruCPassiuGet al. Meta-analysis of neutrophil-to-lymphocyte and platelet-to-lymphocyte ratio in rheumatoid arthritis. Eur J Clin Invest. (2019) 49:e13037. doi: 10.1111/eci.13037

  • 29

    ZinelluAMangoniAA. Neutrophil-to-lymphocyte and platelet-to-lymphocyte ratio and disease activity in rheumatoid arthritis: A systematic review and meta-analysis. Eur J Clin Invest. (2023) 53:e13877. doi: 10.1111/eci.13877

  • 30

    SongBWKimARKimYKKimGTAhnEYSoMWet al. Diagnostic value of neutrophil-to-lymphocyte, platelet-to-lymphocyte, and monocyte-to-lymphocyte ratios for the assessment of rheumatoid arthritis in patients with undifferentiated inflammatory arthritis. Diagnostics (Basel). (2022) 12:1702. doi: 10.3390/diagnostics12071702

  • 31

    PaliogiannisPSattaRDeligiaGFarinaGBassuSMangoniAAet al. Associations between the neutrophil-to-lymphocyte and the platelet-to-lymphocyte ratios and the presence and severity of psoriasis: a systematic review and meta-analysis. Clin Exp Med. (2019) 19:37–45. doi: 10.1007/s10238-018-0538-x

  • 32

    SenerGInan YukselEGokdenizOKaramanKCanatHD. The relationship of hematological parameters and C-reactive protein (CRP) with disease presence, severity, and response to systemic therapy in patients with psoriasis. Cureus. (2023) 15:e43790. doi: 10.7759/cureus.43790

  • 33

    WangLWangCJiaXYangMYuJ. Relationship between neutrophil-to-lymphocyte ratio and systemic lupus erythematosus: A meta-analysis. Clinics (Sao Paulo). (2020) 75:e1450. doi: 10.6061/clinics/2020/e1450

  • 34

    EkiciHImamogluMOkmenFGencosmanGAkGErgenogluM. Evaluation of neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio in pregnant women with systemic lupus erythematosus. J Obstet Gynaecol. (2022) 42:872–6. doi: 10.1080/01443615.2021.1946022

  • 35

    Mercader-SalvansJGarcia-GonzalezMQuevedo-AbeledoJCQuevedo-RodriguezARomo-CorderoAOjeda-BrunoSet al. Blood composite scores in patients with systemic lupus erythematosus. Biomedicines. (2023) 11:2782. doi: 10.3390/biomedicines11102782

  • 36

    MoolaSMunnZTufanaruCAromatarisESearsKSfetcuRet al. Systematic reviews of etiology and risk. In: AromatarisEMunnZ, editors. Joanna Briggs Institute Reviewer’s Manual. Johanna Briggs Institute, Adelaide, Australia (2017).

  • 37

    BalshemHHelfandMSchunemannHJOxmanADKunzRBrozekJet al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. (2011) 64:401–6. doi: 10.1016/j.jclinepi.2010.07.015

  • 38

    BeggCBMazumdarM. Operating characteristics of a rank correlation test for publication bias. Biometrics. (1994) 50:1088–101. doi: 10.2307/2533446

  • 39

    SterneJAEggerM. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol. (2001) 54:1046–55. doi: 10.1016/s0895–4356(01)00377–8

  • 40

    DuvalSTweedieR. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. (2000) 56:455–63. doi: 10.1111/j.0006-341X.2000.00455.x

  • 41

    PageMJMcKenzieJEBossuytPMBoutronIHoffmannTCMulrowCDet al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. (2021) 372:n71. doi: 10.1136/bmj.n71

  • 42

    WanXWangWLiuJTongT. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. (2014) 14:135. doi: 10.1186/1471–2288-14–135

  • 43

    HigginsJPThompsonSG. Quantifying heterogeneity in a meta-analysis. Stat Med. (2002) 21:1539–58. doi: 10.1002/sim.1186

  • 44

    HigginsJPThompsonSGDeeksJJAltmanDG. Measuring inconsistency in meta-analyses. BMJ. (2003) 327:557–60. doi: 10.1136/bmj.327.7414.557

  • 45

    TobiasA. Assessing the influence of a single study in the meta-analysis estimate. Stata Tech Bulletin. (1999) 47:15–7.

  • 46

    EshebaNEShahbaA. Assessment of neutrophil lymphocyte ratio in systemic sclerosis patients in tanta university hospital: A promising marker in predicting disease severity. Egypt J Rheumatol Clin Immunol. (2016) 4:43–7. doi: 10.21608/EJRCI.2015.4475

  • 47

    YolbasSYildirimAGozelNUzBKocaSS. Hematological indices may be useful in the diagnosis of systemic lupus erythematosus and in determining disease activity in behcet's disease. Med Princ Pract. (2016) 25:510–6. doi: 10.1159/000447948

  • 48

    JungJ-HLeeY-MLeeE-GYooW-HLeeW-S. Neutrophil-to-lymphocyte ratio in diagnosis of systemic sclerosis for prediction of interstitial lung disease. J Rheumatic Dis. (2017) 24:138–42. doi: 10.4078/jrd.2017.24.3.138

  • 49

    YangZZhangZLinFRenYLiuDZhongRet al. Comparisons of neutrophil-, monocyte-, eosinophil-, and basophil- lymphocyte ratios among various systemic autoimmune rheumatic diseases. APMIS. (2017) 125:863–71. doi: 10.1111/apm.12722

  • 50

    KimAKimYKimGTAhnESoMWSohnDHet al. Platelet-to-lymphocyte ratio and neutrophil-to-lymphocyte ratio as potential makers for digital ulcers and interstitial lung disease in patients with systemic sclerosis: cross-sectional analysis of data from a prospective cohort study. Rheumatol Int. (2020) 40:1071–9. doi: 10.1007/s00296–020-04604–6

  • 51

    TezcanDTuranÇYılmazSSivrikayaAGülcemalSLimonMet al. What do simple hematological parameters tell us in patients with systemic sclerosis? Acta Dermatovenerologica Alpina Pannonica Adriatica. (2020) 29:101–7. doi: 10.15570/actaapa.2020.23

  • 52

    YaylaMEIlgenUOkatanIEUsluYurteriETorgutalpMKelesoglu DincerABet al. Association of simple hematological parameters with disease manifestations, activity, and severity in patients with systemic sclerosis. Clin Rheumatol. (2020) 39:77–83. doi: 10.1007/s10067–019-04685–0

  • 53

    SakrBRRabeaREElHamidSM. Value of hematological parameters as biomarkers of disease manifestations and severity in systemic sclerosis. Egyptian Rheumatologist. (2021) 43:159–65. doi: 10.1016/j.ejr.2020.06.008

  • 54

    LiHZhangXYuLShangJFanJFengXet al. Comparing clinical characteristics of systemic sclerosis with or without interstitial lung disease: A cross-sectional study from a single center of the Chinese Rheumatism Data Center. Front Med (Lausanne). (2022) 9:1061738. doi: 10.3389/fmed.2022.1061738

  • 55

    NejatifarFMirboloukNMasoolehISKazemnejadEGhavidel-ParsaBGhanbariAMet al. Association between neutrophil/lymphocyte ratio and disease severity in scleroderma patients. Heliyon. (2023) 9:e20576. doi: 10.1016/j.heliyon.2023.e20576

  • 56

    CohenJ. Statistical power analysis. Curr Dir Psychol Sci. (1992) 1:98–101. doi: 10.1111/1467-8721.ep10768783

  • 57

    RossLBaronMNikpourM. The challenges and controversies of measuring disease activity in systemic sclerosis. J Scleroderma Relat Disord. (2018) 3:115–21. doi: 10.1177/2397198318765061

  • 58

    ConnollyMK. Systemic sclerosis (scleroderma): remaining challenges. Ann Transl Med. (2021) 9:438. doi: 10.21037/atm-20–5449

  • 59

    AzabBCamacho-RiveraMTaioliE. Average values and racial differences of neutrophil lymphocyte ratio among a nationally representative sample of United States subjects. PloS One. (2014) 9:e112361. doi: 10.1371/journal.pone.0112361

  • 60

    MisumidaNKobayashiAKaneiY. Racial differences in the neutrophil-to-lymphocyte ratio in patients with non-ST-segment elevation myocardial infarction. Coron Artery Dis. (2015) 26:381–5. doi: 10.1097/MCA.0000000000000237

  • 61

    BrinnAStoneJ. Neutrophil-lymphocyte ratio across psychiatric diagnoses: a cross-sectional study using electronic health records. BMJ Open. (2020) 10:e036859. doi: 10.1136/bmjopen-2020–036859

  • 62

    MarteJLToneyNJCordesLSchlomJDonahueRNGulleyJL. Early changes in immune cell subsets with corticosteroids in patients with solid tumors: implications for COVID-19 management. J Immunother Cancer. (2020) 8:e001019. doi: 10.1136/jitc-2020–001019

  • 63

    Aktas KarabayEAksu CermanADemirDKivanc AltunayI. The effects of systemic psoriasis therapies on the C-reactive protein and the neutrophil-lymphocyte ratio. Ann Dermatol. (2019) 31:601–10. doi: 10.5021/ad.2019.31.6.601

Summary

Keywords

neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, monocyte-to-lymphocyte ratio, systemic sclerosis, disease complications, biomarkers, inflammation

Citation

Zinellu A and Mangoni AA (2024) The association between the neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, and monocyte-to-lymphocyte ratio and systemic sclerosis and its complications: a systematic review and meta-analysis. Front. Immunol. 15:1395993. doi: 10.3389/fimmu.2024.1395993

Received

05 March 2024

Accepted

26 April 2024

Published

10 May 2024

Volume

15 - 2024

Edited by

Katarzyna Romanowska-Próchnicka, National Institute of Geriatrics, Rheumatology and Rehabilitation, Poland

Reviewed by

Emanuele Bizzi, ASST Fatebenefratelli Sacco, Italy

Steven O’Reilly, STipe Therapeutics, Denmark

Updates

Copyright

*Correspondence: Arduino A. Mangoni,

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics