Screening for Distress in Oncological Patients: The Revised Version of the Psychological Distress Inventory (PDI-R)

Abstract

Background:

Psychological research in oncological settings is steadily increasing and the construct of psychological distress has rapidly gained popularity—leading to the development of questionnaires aimed at its measurement. The Psychological Distress Inventory (PDI) is one of the most used instruments, but its psychometric properties were not yet deeply evaluated. The present studies aimed at investigating the psychometric properties of the PDI (Study 1) and providing a revised version of the tool (Study 2).

Methods:

Oncological outpatients were enrolled at the Department of Medical Oncology of the Presidio Ospedaliero of Saronno, ASST Valle Olona, Italy. For the first study (N = 251), an Exploratory Graph Analysis was used to explore the item structure of the PDI. In the second study (N = 902), the psychometric properties of the revised PDI (PDI-R) were deeply assessed.

Results:

Study 1 showed that the PDI has a not clear structure and it should be reconsidered. On the opposite, Study 2 showed that the revised version (PDI-R) has a solid factorial structure, it is invariant across gender and age, and it has good psychometric properties.

Conclusion:

Results suggest that the PDI-R is a reliable measure of psychological distress in different samples of oncological patients, with stronger psychometric properties than the original version. Its use in the clinical and research field is therefore recommended to improve the quality of both assessment and treatment of psychological distress in patients with oncological problems.

Introduction

Cancer disease is one of the most common chronic diseases (Crocetti et al., 2006; Jemal et al., 2006) and, with about 9.6 million deaths per year (Ferlay et al., 2019; Teo et al., 2019), represents one of the major causes of death worldwide (WHO, 2014). In Italy, in 2019 cancers alone accounted for 29% of deaths, the most frequent types being breast (53,500), colorectal (49,000), lung (42,500), prostate (37,000), and bladder (29,700) (AIRTUM Working Group, 2021).

Despite the advances in the medical field, the diagnosis of cancer brings with it physical changes associated with the disease itself or to treatment side effects and certainly represents a stressful life event that may induce psychological distress in both patients and their loved ones (Norton et al., 2004; Sellick and Edwardson, 2007; Rossi et al., 2021)—also significantly altering the quality of their relationship (Mannarini and Boffo, 2014; Mannarini et al., 2017; Saritas et al., 2017; Catania et al., 2019; Granek et al., 2019; Rakici and Karaman, 2019).

Negative emotions might relate to the threat to life, and the uncertainty for both the treatment outcomes and the entailed suffering (Peck, 1972; Maguire, 1985; Holland, 1992; Amadori et al., 2002; Grassi and Riba, 2012; Wise et al., 2013; Clerici and Veneroni, 2014)—and studies reveal that at least one-third of the individuals diagnosed with cancer report high levels of psychological distress; which increase as the disease worsens (Carlson and Bultz, 2003; Sellick and Edwardson, 2007).

Psychological distress in the oncological setting corresponds to the cancer-related totality of emotions and feelings experienced by patients that may affect their ability to cope with cancer itself (Holland and Bultz, 2007; Compen et al., 2018). Indeed, evidence exists for the association between high levels of distress and decreased quality of life (Derogatis et al., 1983; Maguire, 1985; Fallowfield, 1988; Fallowfield et al., 2001; Riba et al., 2019), adherence to treatment, self-management (Newell et al., 1998; Partridge et al., 2002; Spiegel and Giese-Davis, 2003; Moore, 2010; Granek et al., 2019; Oliveri et al., 2019; Panzeri et al., 2021a), and survival in patients with oncological problems (Trask et al., 2002; Kwak et al., 2013; Ng et al., 2017). Further, Bultz and Carlson (2006) considered psychological distress a fundamental marker of wellbeing and—due to its reciprocal influence with the physical status of the sufferers (Zabora et al., 1997, 2001; Carlson et al., 2004; Louison et al., 2019; Panzeri et al., 2021b; Rossi Ferrario et al., 2021)—a key indicator of the patient’s global health (Bultz and Carlson, 2006) to the point of being listed as the sixth vital sign to be investigated in medicine (Grassi and Riba, 2012; Wise et al., 2013; Riba et al., 2019) along with blood pressure, temperature, heart rate, respiratory rate, and pain (Rose and Clarke, 2010; Cutillo et al., 2017).

However, this construct is still is often misinterpreted in literature and confused with other terms, such as «symptom distress» (Ridner, 2004)—which corresponds to patients’ discomfort related to the perceptions of their symptoms (McCorkle and Young, 1978). Moreover, it is also seriously underreported in the oncological field (Brain et al., 2006).

Physicians, oncologists, and psycho-oncologists should, therefore, properly screen psychological distress levels in patients with cancer at various (treatment) stages (Carlson and Bultz, 2003; Jacobsen, 2007). To facilitate its measurement, the National Comprehensive Cancer Network^® (NCCN^®) guidelines have indicated that psychological distress may be thought of in terms of (a conjunction of) anxiety and depression (Trask et al., 2002). Massé (2000) has attempted to identify the components of psychological distress and his results show that it consists of a change from a baseline emotional state to the experience of anxiety, depression, aggressiveness, self-depreciation, and demotivation (Massé, 2000). Indeed, when a clinician diagnoses cancer it is generally not possible to make an accurate prognosis; the patient has no guarantee that treatment will restore health. This situation increases the fear of worsening and death. The patient feels hopeless and unable to adopt behaviors useful for the improvement of health. These feelings of total helplessness contribute to the development of depressive symptoms (Rosselli et al., 2015). Furthermore, Pandey et al. (2007) pointed out that anxiety and depression are highly associated with distress in cancer patients to the point of supporting the idea of an overlap between these constructs (Pandey et al., 2007).

Accordingly, studies show that 23.4% of cancer patients report anxiety symptoms which worsen when the cancer symptoms appear, during examinations, and at diagnosis (Stark and House, 2000; Ng et al., 2017; Naser et al., 2021). Moreover, the presence in oncological patients of depressive symptoms ranges from 8 to 24% in non-palliative-care treatment during or after intervention (Krebber et al., 2014). Still, the presence of depressive symptoms might differ according to the cancer type, how these symptoms are evaluated, and the intervention phase—with a higher prevalence in more severe patients (Pirl, 2004; Krebber et al., 2014; Naser et al., 2021). Starting from this background, numerous researches have aimed to create and validate psychological tools for the assessment and measurement of distress in patients with organic pathologies (Carlson and Bultz, 2003; Herschbach et al., 2004; Mitchell, 2010).

However, the Psychological Distress Inventory (PDI, originally developed and validated in Italian context) (Morasso et al., 1996), one of the most worldwide used instruments for the screening and assessment of psychological distress in oncology (Vodermaier et al., 2009; Muzzatti and Annunziata, 2012), was validated without an in-depth evaluation of its psychometric properties, (e.g., factorial structure, measurement invariance, etc.). Indeed, it provides encouraging, albeit incomplete, information on reliability and criterion, concurrent, and discriminant validity but no data on construct validity (Muzzatti and Annunziata, 2012).

Notably, a screening tool that does not have strong and established psychometric properties may lead to misleading results. However, an even more serious problem might be represented by the measurement biases, which may, in turn, lead to underestimation of the measured problem or misdiagnosis—and therefore increase the patient’s suffering (Chad-Friedman et al., 2017). Given the strong impact of distress on health as well as on (also) medical care of cancer patients (DiMatteo et al., 2000), it is therefore essential to have well-established, well-structured, reliable, and that can be used in samples with different characteristics.

Consequently, the first goal (Study 1) of this study is to explore the psychometric properties (i.e., factor dimensionality) of the Psychological Distress Inventory (PDI) (Morasso et al., 1996). On the basis of results of the Study 1, the second aim (Study 2) is to develop and extensively assess the psychometric properties (i.e., structural validity, measurement invariance, screening ability, etc.) of a shortened version of the PDI: the Psychological Distress Inventory—Revised (PDI-R).

Study 1. Exploring the Dimensional Structure of the Psychological Distress Inventory

Materials and Methods

Procedure

An observational research design was used to investigate the psychological distress experienced by oncological outpatients during the first few weeks between the diagnosis of cancer and the first psychological clinical session—in line with the procedure provided by the HuCARE study protocol (Passalacqua et al., 2016; Marconi et al., 2020; Caminiti et al., 2021; Rossi et al., 2021).

This study was approved by the Ethics Committee of the Ospedale di Saronno (protocol n° 23247). All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Participants

Oncological outpatients were consecutively recruited from the Department of Medical Oncology, Presidio Ospedaliero di Saronno, ASST Valle Olona, in Saronno (Italy).

Inclusion criteria for participating in the study were: (A) having received a diagnosis of cancer within the last 6 months; (B) being an oncological outpatient (C) not being hospitalized for cancer-related problems within the last year, (D) following intravenous therapy for cancer; (E) being 18 years or older; (F) providing informed consent to participate in the study; and (G) being a native Italian speaker. Exclusion criteria were: (A) inability to understand the items of the questionnaire; (B) impossibility to be assessed due to speaking impairments and/or upcoming medical commitments.

A sample of 270 participants was initially assessed. However, 19 subjects were excluded due to missing data/answers. The final sample comprised, therefore, 251 participants: 120 males (47.80%) and 131 females (52.20%), aged from 20 to 86 years (mean = 63.24, SD = 12.56). Considering the type of cancer, 30.7% patients had breast cancer, 27.1% patients had lung cancer, 19.9% patients had gastrointestinal cancer, 10.6% patients had urogenital cancer, 4.2% patients had oncohematological cancer, and 7.5% patients had other type of cancer (e.g., head-and-neck cancer or skin cancer). Considering education level, 22.9% patients had an elementary school diploma, 35.4% patients had a middle school diploma, 34.3% patients had a high school diploma, and 7.4% patients had a bachelor/master’s degree. Considering civil status, 77.2% patients were either in a relationship or married, 11.4% patients were either separated or divorced, 8.3% patients were widowed, and 3.1% patients were single. Considering working status, 50.1% patients were retired, 27.6% patients were dependent workers, 12.4 patients were entrepreneurs, 3.7% patients were housewifes, 4.2 patients were unemployed, and 2% patients declared “other.”

Sample Size Determination

The sample size was calculated a priori considering the statistical analysis of this study. However, to date, within the framework of Exploratory Graph Analysis (EGA), no “gold standard” rule for determining the minimum sample size required to correctly estimate model parameters seems to exist (Golino and Epskamp, 2017). Therefore, a minimum sample size of 200 individuals was considered adequate.

Measures

Psychological Distress Inventory

The Psychological Distress Inventory (PDI) was used to measure the degree of psychophysical distress experienced by the sample (Morasso et al., 1996). It represents a self-report questionnaire aimed at detecting distress in individuals suffering from oncological problems by investigating the experience of discomfort related to both emotional and physical domains. The PDI comprises a total of 13 items—with good reliability properties (Morasso et al., 1996)—on a 5-point Likert-type response scale ranging from 1 (not at all) to 5 (very much). High scores correspond to a high degree of distress perceived by the subject. In this study, Cronbach’s alpha was equal to 0.804 and McDonald’s omega was equal to 0.846.

Statistical Analyses

The R software (R Core Team, 2017) was used with the following packages: bootnet (Epskamp et al., 2018); corrplot (Wei and Simko, 2017); EGAnet (Golino and Christensen, 2020), igraph (Csardi and Nepusz, 2006), networkTools (Jones, 2020), qgraph (Epskamp et al., 2012) psych (Revelle, 2018), and psychTools (Revelle, 2020).

Preliminary analyses were performed before carrying out the EGA (Christensen et al., 2020b). First, the normality of items, and the presence of excessive correlations (r > 0.70) between items, were inspected (Howell, 2013; Tabachnick and Fidell, 2014). Second, for each item, the level of informativeness was evaluated (Mullarkey et al., 2019; Bottesi et al., 2020). An item should be considered as badly informative as its SD is 2.5 SDs below the average of all the items (Mullarkey et al., 2018, 2019; Marchetti, 2019). Third, item redundancy was checked (Christensen et al., 2020a) by using a Unique Variable Analysis (UVA) approach with weighted topological overlap (wTO) method and adaptive alpha. However, as suggested by existing guidelines, possible item redundancies should be carefully evaluated—for example, considering the aims of the study and/or by inspecting the semantic content of the items (Christensen et al., 2020b).

Consequently, an EGA (Golino and Epskamp, 2017; Christensen et al., 2020b) was performed to assess the item clustering (i.e., dimensionality) of the PDI—given its several advantages over traditional exploratory factor-analytic techniques; it provides greater accuracy in identifying the correct number of factors/dimensions (Golino and Demetriou, 2017; Golino and Epskamp, 2017; Christensen and Golino, 2020; Golino et al., 2020a,b). The EGA produces a plot that might be considered as a “visual guide” (Golino and Epskamp, 2017; Golino et al., 2020a; Panzeri et al., 2021c). It displays the correct number of dimensions—by highlighting which items cluster together and their level of association: the thicker is an edge, the strongest is the relationship between item of a specific cluster (dimension/factor) (Mair, 2018; Christensen and Golino, 2020). The Exploratory Graph Analysis (EGA) was carried out by using a 5,000 parametric bootstrap procedure. Moreover, the GLASSO method with polychoric correlations was used to estimate model parameters (Costantini et al., 2015; Epskamp, 2017; Golino and Epskamp, 2017). In addition, the correct number of dimensions were detected by using the “Louvain community detection algorithm” (Blondel et al., 2008). It has demonstrated better performances than the Walktrap algorithm (Pons and Latapy, 2006) in recognizing clusters of items/dimensions (Christensen et al., 2020b).

Once the EGA revealed the number of dimensions composing the PDI, the questionnaire and item statistics were explored. First, item stability (IS) was computed to evaluate the proportion of times the original dimension is exactly replicated across bootstrap resamples—thus, assessesing the occurrence of each item within a certain specific dimension (Christensen et al., 2020b). IS ranging from 0 (“= perfect instability”) to 1 (“= perfect stability”) and values higher of 0.80 (IS ≥ 0.80) suggest that the item can be considered “stable” and consistently identified in the dimension (Christensen and Golino, 2019). The contribution of each node to the coherence of the dimensions was then assessed using the standardized node strength—namely, network loadings. It is important to note that they represent partial correlation loadings, and the magnitude of these loadings should therefore be interpreted according to the following benchmarks (Christensen and Golino, 2020; Golino et al., 2020c): small: λ_EGA ≥ 0.15; moderate: λ_EGA ≥ 0.25; large: λ_EGA ≥ 0.35.

The internal consistency of each factor was evaluated with Cronbach’s alpha and McDonald’s omega (McDonald, 1999; Howell, 2013).

Lastly, correlations between items were assessed using Pearson’s correlation coefficient and interpreted using Cohen’s (Cohen, 1988) classical benchmarks: r < 0.10, trivial; r from 0.10 to 0.30, small; r from 0.30 to 0.50, moderate; r > 0.50, large.

Results

Preliminary Analyses

First, as reported in Table 1, univariate normality was observed for the large majority of the PDI items. Considering non-normal distributed items, only three of them showed small-to-moderate deviations from normality (item#5, item#9, and item#13). Moreover, none of the bivariate correlations exceeded a critical level (r ≥ 0.70).

Second, the level of informativeness of each item was tested. Results showed that none of the 13 items of the PDI was badly informative (i.e., SD_item < 2.5 SD below the mean level of informativeness, M_SD = 1.16 ± 0.18)—suggesting that each item of the PDI provides adequate variability across the sample as well as a good level of informativeness.

Third, item redundancy was inspected. The UVA showed possible redundancy between some items of the PDI. Considering the aim of the study—exploring the structure of the PDI and redefining its psychometric properties—no item was removed. However, item redundancies were carefully considered and deeply studied before setting up Study 2.

Exploratory Graph Analysis

As reported in Figure 1 and Table 1, the EGA (5000 bootstrap) clearly identified three dimension/factor solution: median = 3; SE = 0.489; 95%CI [2.040, 3.960]. Moreover, the bootstrapped EGA showed that the probability of a three dimension/factor solution was = 0.682 (68.2%) and the probability of a four-dimension/factor solution was = 0.309 (30.9%).

FIGURE 1

Item Statistics

As reported in Table 1 and Figure 2, IS revealed that the PDI items were—on average—stable within and between their designated dimension/factor: Dim#1_{IS_replication_mean} = 0.936; Dim#2_{IS_replication_mean} = 0.708; Dim#3_{IS_replication_mean} = 0.813. More in detail, the IS analysis (Table 1) showed that most items had a good replication index. Indeed, the items in the first dimension (item#3, item#7, item#9, and item#10) displayed a replication index higher than 0.88; also the items in the second dimension (item#1, item#5, item#11, and item#13) displayed a replication index higher than 0.88. Lastly, the items in the third dimension (item#2, item#6) displayed a replication index higher than 0.99. It should be noted that item#4 (feel tired), item#8 (body image), and item#12 (sexual difficulties) did not achieve the recommended threshold of 0.80 in none of the three aforementioned dimensions.

FIGURE 2

Then, EGA-based network loadings (λ_EGA) were computed to assess the contribution of each node to the coherence of the dimensions. As reported in Table 1, each item showed a high association with its more stable dimension/factor. Considering the first dimension/factor, λ_EGA ranged from 0.257 (large) to 0.426 (large). For the second dimension/factor, λ_EGA ranged from 0.231 (large) to 0.380 (large). For the third dimension/factor, λ_EGA ranged from 0.302 (large) to 0.334 (large).

Then, an in-depth examination of the semantic content of the items—grouped according to EGA—led to the labeling of the three dimensions. The first dimension measures internal states of psychological distress: anxiety (item#3), depression (item#7), lack of self-esteem (item#9), and lack of motivation (item#10). The second dimension assesses distress related to relationships and the external world: lack of desire to talk to others (item#1), loneliness (item#5), lack of external interests (item#11), worsening of relationships (item#13). Lastly, the third dimension relates exclusively to reverse items and thus a situation of wellbeing/tranquility (item#2 and Item#6).

Study 2—Validation and an In-Depth Analysis of the Revised Version of the PDI (PDI-R)

Materials and Methods

Procedure

According to the HuCARE project protocol (Passalacqua et al., 2016; Marconi et al., 2020; Caminiti et al., 2021; Rossi et al., 2021), the same procedure and inclusion/exclusion criteria of Study 1 were applied. This study was approved by the Ethics Committee of the Ospedale di Saronno (protocol n° 23247). All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Also in this study, oncological outpatients of the study were still consecutively recruited at the Department of Medical Oncology, Presidio Ospedaliero di Saronno, ASST Valle Olona, in Saronno (Italy).

Participants

An initial sample of 936 oncological outpatients was assessed. However, 34 subjects were excluded from the sample due to missing data/answers. Therefore, the final sample comprised 902 oncological outpatients: 487 males (54%) and 415 females (46%), aged from 31 to 89 years (mean = 68.39, SD = 9.073). Considering the type of cancer, 34.8% patients had breast cancer, 24.9% patients had lung cancer, 18.8% patients had gastrointestinal cancer, 7.9% patients had urogenital cancer, 3.5% patients had pancreatic cancer, 2.7% patients had oncohematological cancer, and 7.4% patients had other type of cancer (e.g., head-and-neck cancer or skin cancer). Considering education level, 26.9% patients had an elementary school diploma, 32.8% patients had a middle school diploma, 30.8% patients had a high school diploma, and 9.5% patients had a bachelor/master’s degree. Considering civil status, 73% patients were either in a relationship or married, 6.8% patients were either separated or divorced, 10.3% patients were widowed, and 9.9% patients were single. Considering working status, 58.8% patients were retired, 20.7% patients were dependent workers, 9.6% patients were entrepreneurs, 5.6% patients were housewifes, 3.8 patients were unemployed, and 1.5% patients declared “other.”

Sample Size Calculation

Since the present study aimed to assess psychometric properties of a (“new”) questionnaire, the “n:q criterion” was used to determine the minimum sample size. In this formula, n is the number of subjects and q is the number of (free) model parameters to be estimated (Hu and Bentler, 1999; Muthén and Asparouhov, 2002; Yu, 2002). A ratio of five subjects per parameter (10:1; n_minimum = 480) was guaranteed (Bentler and Chou, 1987; Marsh et al., 1988; Hu and Bentler, 1999; Boomsma and Hoogland, 2001; Muthén and Asparouhov, 2002; Yu, 2002; Flora and Curran, 2004; Tomarken and Waller, 2005).

Measures

Psychological Distress Inventory—Revised

On the basis of the results of Study 1 and following consolidated methodological procedures (Pietrabissa et al., 2020a,b; Rossi et al., 2021), an in-depth analysis of items was performed. This procedure led to the removal of both the two reversed scored items of the original PDI (item#2 and item#6), and the three items that did not achieve IS threshold of 0.80 (item#4, item#8, and item#12). Consequently, on the basis of the results of Study 1, only two dimensions were retained: (A) “internal” and (B) “external” distress. The first dimension assessed internal states of psychological distress such as anxiety, depression, and lack of self-esteem. The second dimension assesses distress related to relationships and the external world: lack of desire to talk to others, loneliness, and worsening of relationships. Furthermore, considering the semantic content of the items, a hierarchical “general distress” dimension was also hypothesized. It is important to underline that the semantic content of the items has not been changed compared to the original version of the PDI provided by Morasso and colleagues (Morasso et al., 1996).

As for the original version of the PDI, the PDI-R relies on a 5-point Likert-type response scale (from 1 = “not at all” to 5 = “very much”) see Supplementary Appendix A (English version) and Supplementary Appendix B (Italian version). High scores correspond to a high degree of distress perceived by the subject.

The Hospitalized Anxiety and Depression Scale

The Hospitalized Anxiety and Depression Scale (HADS) was used to measure the degree of psychological suffering in oncological patients (Zigmond and Snaith, 1983; Annunziata et al., 2011; Iani et al., 2014). It is a self-report questionnaire—with a bi-factorial structure—aimed at detecting anxiety and depression in hospitalized individuals (as well as outpatients) affected by organic pathology, as well as their perceived distress as the sum of the scores obtained on the anxiety and depression dimensions (Iani et al., 2014). The HADS comprises a total of 14 items—with good reliability properties—scored on a 4-point Likert-type response scale. High scores correspond to a high degree of anxiety, depression, and distress perceived by the subject. Two cut-offs threshold for the HADS total score (HADS-T; ≥ 16 = moderate distress; ≥ 22 = severe distress) were deemed appropriate in general clinical health settings to distinguish between individuals with no-psychological suffering and those with psychological suffering (Olssøn et al., 2005; Singer et al., 2009; Vodermaier et al., 2009; Stern, 2014). In this study, Cronbach’s alpha was equal to: 0.836 for the anxiety subscale (HADS-A); 0.806 for the depression subscale (HADS-D); and 0.887 for the scale’s total score (HADS-T).

The Distress Thermometer

The Distress Thermometer (DT) (Bulli et al., 2008; O’Donnell, 2013; Cutillo et al., 2017) is one of the most worldwide used instruments for the measurement of emotional distress in different contexts including oncological settings. The DT is a self-report single-item measure scored on a 10-point Likert scale (from 0 = “no distress” to 10 = “extreme distress”) aimed at detecting psychological and emotive suffering. High scores correspond to a high degree of emotional distress perceived by the subject. Moreover, a cut-off threshold of ≥ 5 (Gil et al., 2005; Grassi et al., 2009; Vodermaier et al., 2009; Donovan et al., 2014) for the DT was deemed appropriate in general clinical health settings to distinguish between individuals with no-psychological distress and those who suffering from psychological distress (Grassi et al., 2009).

Statistical Analysis

R software (R Core Team, 2014, 2017) was used to perform statistical analyses with the following packages: cNORM (Lenhard W. et al., 2018); corrplot (Wei and Simko, 2017); irr (Gamer et al., 2019); lavaan (Rosseel, 2012; Rosseel et al., 2015), plotROC (Sachs, 2017); pROC (Robin et al., 2011), psych (Revelle, 2018), psychTools (Revelle, 2020), and semTools Contributors (2016).

A bi-factor model was specified: 4 items loaded onto the “internal distress” latent factor, while 4 items loaded onto the “external distress” latent factor, and each item also loaded onto a hierarchical “general distress” dimension.

The diagonal weighted least square (DWLS) estimator was used to assess the factorial structure of the PDI-R (Brown, 2015; Kline, 2016; Lionetti et al., 2016). Model fit was assessed by means of the Satorra-Bentler Chi-square statistics (S-Bχ²), the Root-Mean Square Error of Approximation (RMSEA), the Comparative Fit Index (CFI), and the Standardized Root Mean Residual (SRMR) (Muthén and Muthén, 1998–2017; Hoyle, 2012; van de Schoot et al., 2012; Brown, 2015; Kline, 2016). Moreover, the following cut-off criteria were chosen to evaluate the goodness of fit: (A) statistically non-significance of the χ², (B) an RMSEA lower than 0.08, (C) a CFI higher than 0.95, and (D) an SRMR lower than 0.08 (Muthén and Muthén, 1998–2017; Hoyle, 2012; van de Schoot et al., 2012; Brown, 2015; Kline, 2016).

In addition, for a comprehensive evaluation of the factorial structure of the PDI-R, two alternative models were further specified and compared. First, a single factor model was specified: all of the 8 items loaded onto a single distress latent factor. Second, a first-order two-factor model was specified: 4 items loaded onto the “internal distress” latent factor, while 4 items loaded onto the “external distress” latent factor; thus, the general distress dimension was not specified.

Model evaluations were performed by using the test differences in three fit indices, with the following criteria as cutoffs for model equality: DIFFTEST (equal to Δχ²; p-value > 0.050) and ΔCFI (<0.010) (Cheung and Rensvold, 2002; Millsap and Yun-Tein, 2004; Millsap, 2012). The crossing of the cutoff of two out of three of these indices is evidence of model inadequateness.

Moreover, since the PDI-R is a new questionnaire, items’ ability to discriminate subjects with low or high internal, external, and general distress was tested (Ebel, 1965; Chiorri, 2011). Item discriminant power (IDP) was computed. More in detail, the maximum total score and quartile rank for each subject were calculated. Subsequently, a series of independent sample t-tests—and their effect size (Cohen’s d) (Cohen, 1988)—were calculated to assess item discriminating power by using the total score of the scale as a dependent variable and its lowest and highest quartile as grouping variable (Ebel, 1965; Chiorri, 2011). Moreover, item-total correlation (adjusted; i.e., r_it–tot) was also computed (Howell, 2013; Pallant, 2013; Tabachnick and Fidell, 2014).

The internal consistency of each factor was evaluated with Cronbach’s alpha and McDonald’s omega (McDonald, 1999; Howell, 2013). Convergent validity was assessed with the Pearson correlation coefficient (Tabachnick and Fidell, 2014) and interpreted using the aforementioned Cohen’s benchmarks (Cohen, 1988).

Measurement invariance (MI) analyses were also performed to evaluate whether the factorial structure of the PDI-R was invariant between gender (male vs. female) and age (≤ 64 vs. ≥ 65) (Vandenberg and Lance, 2000). According to Meredith (1993) and Millsap (2012), the model structure was tested on each sample independently (Meredith, 1993). If the model fit was adequate in each sample, four nested models were sequentially specified and constrained to equality: the factorial structure (Model 1: Configural Invariance); the factorial structure and item factor loadings (Model 2: Metric Invariance); the factorial structure, item factor loadings, and item thresholds (Model 3: Scalar Invariance); the factorial structure, item factor loadings, item thresholds, and latent means (Model 4: Scalar Invariance); (Meredith, 1993; Vandenberg and Lance, 2000; Millsap, 2012). MI was assessed by using the above-mentioned test differences for model comparisons (Cheung and Rensvold, 2002; Millsap, 2012; van de Schoot et al., 2012).

Test-retest reliability of each scale was estimated on a subsample of 40 oncological patients by using the two-way mixed intraclass correlation coefficient (ICC_consistency) (de Vet et al., 2006; Berchtold, 2016; Koo and Li, 2016).

Considering that the PDI-R (as well as the PDI) was conceptualized as a screening tool, a Receiver Operating Characteristics (ROC) curve methodology was used to assess the PDI-R accuracy to discriminate between non-distressed and distressed patients (Zhou et al., 2002; Pepe, 2003; Ising et al., 2012; Savill et al., 2018). More in detail, the HADS-T score cut-offs (HADS-T ≤ 15 vs. HADS-T ≥ 16 and HADS-T ≤ 21 vs. HADS-T ≥ 22) and the DT cut-off (DT ≤ 4 vs. DT ≥ 5) were used as external criterion variable and the PDI-R total score was used as the dependent variable. The global accuracy-validity of the PDI-R was estimated with the area under the ROC curve (AUC; 5000 stratified bootstrap resamples)—interpreted using the Swets’ benchmarks: AUC = 0.50, null; AUC from 0.51 to 0.70, small; AUC from 0.71 to 0.90, moderate; AUC from 0.91 to 0.99, high; and AUC = 1.00, perfect accuracy (Zweig and Campbell, 1993; Swets, 1998). Moreover, sensibility (Se) and specificity (Sp) were computed (Zhou et al., 2002; Pepe, 2003).

Lastly, as a supplemental analysis, according to previous studies procedures (Dakanalis et al., 2013), a “general sample” of patients with cancer was created by merging the sample of Study 1 and the sample of Study 2 (N_total = 1153). Thus, according to the procedure described by Gary et al. (2021), normative scores (T-scores) of the PDI-R were computed as well as the distribution percentiles of its total score (Lenhard A. et al., 2018; Gary et al., 2021).

Results

Structural Validity

The PDI-R showed an excellent fit to the data. The Chi-square statistic resulted to be not statistically significant: S-Bχ² (12) = 17.913; p = 0.118. Moreover, all the other fit indices revealed a good fit to the data: the RMSEA = 0.023; 90%CI [0.000, 0.044]; p(RMSEA < 0.05) = 0.984, the CFI = 0.999, the SRMR = 0.022. As reported in Table 2 and Figure 2, all the items’ loadings were statistically significant and ranged from | 0.206| (item#3, internal distress) to | 0.791| (item#4, general distress).

Moreover, this factor solution was compared with different competing models (Muthén and Muthén, 1998–2017; Brown, 2015). As reported in Table 3, model comparisons revealed the superiority of the proposed solution. Consequently, the bi-factor model solution was maintained to perform the following analysis.

Psychometrics Properties

The IDP analysis showed that 8 items of the PDI-R discriminated well between subjects with low or high internal and external distress (Table 2). The discrimination parameter t_i ranged from |20.34| (item#6—external distress) to |42.33| (item#2—internal distress), with an associated effect size (Cohen’s d) ranging from 1.89 to 4.22, respectively. Also, the item-total correlation (adjusted) revealed a moderate-to-strong association between each item and the PDI-R scores.

Reliability analysis revealed satisfying results. Indeed, for the internal distress subscale, the Cronbach’s alpha was equal to 0.776 and the McDonald’s omega was equal to 0.842. The external distress subscale showed a Cronbach’s alpha equal to 0.754 and a McDonald’s omega was equal to 0.800. The general distress scale showed a Cronbach’s alpha was equal to 0.853 and a McDonald’s omega was equal to 0882.

Large correlations were found between the internal distress scale and the HADS-A scale (r = 0.724, p < 0.001); the HADS-D scale (r = 0.630; p < 0.001), the HADS-T (r = 0.741; p < 0.001), and the DT (r = 0.595, p < 0.001). Also, moderate-to-large correlations were found between the external distress scale and the HADS-A scale (r = 0.583, p < 0.001); the HADS-D scale (r = 0.665; p < 0.001), the HADS-T (r = 0.681; p < 0.001), and the DT (r = 0.490, p < 0.001). Lastly, large correlations were found between the general distress scale and the HADS-A scale (r = 0.713, p < 0.001); the HADS-D scale (r = 0.705; p < 0.001), the HADS-T (r = 0.775; p < 0.001), and the DT (r = 0.576, p < 0.001). In addition, the internal and external distress subscales revealed a large correlation: r = 0.685, p < 0.001.

Test-retest reliability showed satisfying results: the two-way mixed ICC was equal to 0.647, 95%CI [0.385, 0.797], for the internal distress scale, to 0.699, 95%CI [0.476, 0.827], for the external distress scale, and 0.685, 95%CI [0.452, 0.819], for the general distress scale.

Measurement Invariance

Gender (Male vs. Female)

Model Male

The Chi-square statistic resulted to be not statistically significant: S-Bχ² (12) = 13.490; p = 0.334. Moreover, all the other fit indices revealed a good fit to the data: the RMSEA = 0.016; 90%CI [0.000, 0.050]; p(RMSEA < 0.05) = 0.948, the CFI = 1.000, the SRMR = 0.025.

Model Female

The Chi-square statistic resulted to be not statistically significant: S-Bχ² (12) = 10.860; p = 0.541. Moreover, all the other fit indices revealed a good fit to the data: the RMSEA = 0.000; 90%CI [0.000, 0.046]; p(RMSEA < 0.05) = 0.968, the CFI = 1.000, the SRMR = 0.026.

Configural Invariance

The configural invariance model showed good model fit indices: S-Bχ² (24) = 24.350, p = 0.442; the RMSEA = 0.006; the CFI = 1.000; and the SRMR = 0.025; suggesting that the factor structure was similar between males and females.

Metric Invariance

The metric invariance model well-fitted the data: S-Bχ² (40) = 77.762, p < 0.001; the RMSEA = 0.046, the CFI = 0.996, and the SRMR = 0.043. A statistically significant decrease in chi-square was found: DIFTEST (16) = 53.412; p < 0.001. However, a non-statistically significant decreases in CFI was found: |ΔCFI| = 0.004, indicating that items were equivalently related to the latent factor independently from gender.

Scalar Invariance

The scalar invariance model showed good model fit indices: S-Bχ² (61) = 78.150, p < 0.001; the RMSEA = 0.025, the CFI = 0.998; and the SRMR = 0.030. A non-statistically significant decrease in chi-square was found: DIFTEST (21) = 0.387; p = 1. Moreover, a non-statistically significant decreases in CFI was found: |ΔCFI| = 0.002, suggesting that males and females had the same expected item response at the same absolute level of the trait.

Latent Means Invariance

The latent mean invariance model well-fitted the data: S-Bχ² (64) = 103.994; the RMSEA = 0.037, the CFI = 0.996; and the SRMR = 0.033. A statistically significant decrease in chi-square was found: DIFTEST (3) = 25.844; p < 0.001. Moreover, a non-statistically significant decreases in CFI was found: |ΔCFI| = 0.002, suggesting that males and females had the same expected latent mean of the traits.

Age (≤ 64 y.o. vs. ≥ 65 y.o.)

Model ≤ 64 y.o

The Chi-square statistic resulted to be not statistically significant: S-Bχ² (12) = 10.496; p = 0.573. Moreover, all the other fit indices revealed a good fit to the data: the RMSEA = 0.000; 90%CI [0.000, 0.051]; p(RMSEA < 0.05) = 0.947, the CFI = 1.000, the SRMR = 0.029.

Model ≥ 65 y.o

The Chi-square statistic resulted to be not statistically significant: S-Bχ² (12) = 11.080; p = 0.552. Moreover, all the other fit indices revealed a good fit to the data: the RMSEA = 0.000; 90%CI [0.000, 0.040]; p(RMSEA < 0.05) = 0.990, the CFI = 1.000, the SRMR = 0.021.

Configural Invariance

The configural invariance model showed good model fit indices: S-Bχ²(24) = 21.577; the RMSEA = 0.000; the CFI = 1; and the SRMR = 0.023; suggesting that the factor structure was similar across age.

Metric Invariance

The metric invariance model well-fitted the data: S-Bχ² (40) = 67.042; the RMSEA = 0.039, the CFI = 0.997, and the SRMR = 0.040. A statistically significant decrease in chi-square was found: DIFTEST (16) = 45.465; p < 0.001. However, a non-statistically significant decreases in CFI was found: |ΔCFI| = 0.003, indicating that items were equivalently related to the latent factor independently from age.

Scalar Invariance

The scalar invariance model showed good model fit indices: S-Bχ² (61) = 68.991; the RMSEA = 0.017, the CFI = 0.999; and the SRMR = 0.032. A non-statistically significant decrease in chi-square was found: DIFTEST (21) = 1.949; p = 1. Moreover, a non-statistically significant decreases in CFI was found: |ΔCFI| = 0.002, suggesting that younger (≤ 64 y.o.) and older (≥ 65 y.o.) patients had the same expected item response at the same absolute level of the trait.

Latent Means Invariance

The latent mean invariance model well-fitted the data: S-Bχ² (64) = 73.815; the RMSEA = 0.018, the CFI = 0.999; and the SRMR = 0.033. A non-statistically significant decrease in chi-square was found: DIFTEST (3) = 4.825; p = 0.185. Moreover, a non-statistically significant decreases in CFI was found: |ΔCFI| = 0.000, suggesting younger (≤64 y.o.) and older (≥65 y.o.) patients had the same expected latent mean of the traits.

Accuracy of the Psychological Distress Inventory—Revised as a Screening/Diagnostic Tool

Based on the HADS-T scale cut-off for moderate distress (HADS-T ≤ 15 vs. HADS-T ≥ 16), the “general distress” scale of the PDI-R obtained excellent accuracy in discriminating between patients without distress and patients with distress: AUC = 0.908, 95%CI [0.889, 0.928] (Table 4 and Figure 3A). Considering a cut-off point of 15 (i.e., PDI-R ≥ 15: risk of moderate distress), ROC curve revealed a SE equal to 0.881, 95%CI [0.884, 0.918], a SP equal to 0.786, 95%CI [0.753–0.818], and an ACC equal to 0.817, 95%CI [0.817, 0.817]. On the basis of the gold standard test (HADS-T; moderate distress), 67.3% of patients were classified as non-distressed and 32.4% of patients were classified as distressed. Thus, using the reported cut-off for the PDI-R, the ROC curve showed that 52.88% of individuals were correctly classified as “true negative” and 28.82% as “true positive” with a percentage of correct classification equal to 81.7%. On the contrary, 3.88% resulted to be “false negative” and 14.41% resulted to be “false positive” (18.9% of misclassification).

FIGURE 3

Based on the HADS-T scale cut-off for severe distress (HADS-T ≤ 21 vs. HADS-T ≥ 22), the “general distress” scale of the PDI-R obtained excellent accuracy in discriminating between patients without distress and patients with distress: AUC = 0.918, 95%CI [0.896, 0.939] (Table 4 and Figure 3B). Considering a cut-off point of 18 (i.e., PDI-R ≥ 18: risk of severe distress), ROC curve revealed a SE equal to 0.893, 95%CI [0.837, 0.948], a SP equal to 0.790, 95%CI [0.761, 0.819], and an ACC equal to 0.804, 95%CI [0.803, 0.804]. On the basis of the gold standard test (HADS-T; severe distress), 86.6% of patients were classified as non-distressed and 13.4% of patients were classified as distressed. Thus, using the reported cut-off for the PDI-R, the ROC curve showed that 68.4% of individuals were correctly classified as “true negative” and 11.97% as “true positive” with a percentage of correct classification equal to 80.37%. On the contrary, 1.44% resulted to be “false negative” and 18.18% resulted to be “false positive” (19.2% of misclassification).

Based on the DT scale cut-off for distress (DT ≤ 4 vs. DT ≥ 5), the “general distress” scale of the PDI-R obtained good accuracy in discriminating between patients without distress and patients with distress: AUC = 0.857, 95%CI [0.817, 0.897] (Table 4 and Figure 3C). Considering a cut-off point of 13 (i.e., PDI-R geq 13: risk of distress), ROC curve revealed a SE equal to 0.847, 95%CI [0.783, 0.912], a SP equal to 0.739, 95%CI [0.686, 0.792], and an ACC equal to 0.772, 95%CI [0.771, 0.773]. On the basis of the gold standard test (DT), 69.3% of patients were classified as non-distressed and 30.7% of patients were classified as distressed. Thus, using the reported cut-off for the PDI-R, the ROC curve showed that 51.05% of individuals were correctly classified as “true negative” and 26.18% as “true positive” with a percentage of correct classification equal to 77.23%. On the contrary, 4.71% resulted to be “false negative” and 18.06% resulted to be “false positive” (22.77% of misclassification).

Normative Scores

On the total sample of 1,153 patients with cancer (sample of Study 1 plus sample of Study 2), both the normative scores (T-score: mean = 50, SD = 10) and the percentile distribution were calculated. Results are reported in Tables 5, 6.

General Discussion

In the last decades, psychological research in medical and oncological settings constantly increased focusing on observed outcomes of psychological care (Mannarini et al., 2013; Caminiti et al., 2017, 2021; Rossi et al., 2021). In this context, the construct of psychological distress has rapidly gained popularity (Mitchell, 2010; Wise et al., 2013; Riba et al., 2019; Marconi et al., 2020). Indeed, due to its severe negative impacts on medical outcomes (DiMatteo et al., 2000), distress represents one of the most important indexes of suffering in oncological patients (Mitchell, 2010; Cutillo et al., 2017). This interest led to the development of several questionnaires aimed at measure this construct (Carlson and Bultz, 2003; Herschbach et al., 2004; Mitchell, 2010; Wise et al., 2013; Cutillo et al., 2017; Riba et al., 2019).

In particular, the PDI (Morasso et al., 1996) is one of the most used instruments worldwide (Wise et al., 2013; Riba et al., 2019). The PDI was originally developed in the Italian context: it is a thirteen-item self-report instrument (Morasso et al., 1996) and it is among the most recommended tools to screen for distress in oncology (Vodermaier et al., 2009; Muzzatti and Annunziata, 2012)—however, the original validation study showed that it lacks of a rigorous assessment of its (basic) psychometric properties such as its factorial structure. Indeed, The only identified validation study of PDI concerned Italian cancer patients (Morasso et al., 1996). In this study, criterion, concurrent and discriminant—but not construct—validity were reported, together with good internal consistency (Muzzatti and Annunziata, 2012). Despite it provides encouraging, albeit incomplete, information on reliability and validity, data on construct validity are strongly requited for a screening tool (Muzzatti and Annunziata, 2012).

Starting from this background, in line with previous researches (Panzeri et al., 2021c; Parola et al., 2022; Rossi et al., 2022), the aim of the first study here reported was to extensively examine the factorial structure of the PDI. To achieve this goal, EGA was used: an innovative technique, particularly precise and sensitive in identifying the correct number of factors/dimensions (Golino and Epskamp, 2017). Moreover, on the basis of the results of Study 1, the second study (Study 2) aimed at developing a (shortened) revised version of the PDI (PDI-R) and to provide an in-depth analysis of its psychometric properties including its factorial structure and measurement invariance, thus filling the gap in the literature. To achieve this, gold-standard statistical techniques were used to analyze the psychometric properties of the questionnaires (e.g., CFA).

The first study revealed that the (original) supposed single-factor structure of the PDI (Morasso et al., 1996)—one latent dimension/factor accounting for all of the 13 items—was not adequate. Indeed, the EGA revealed that the original pool of items grouped onto three different dimensions/factors—instead of a single one (Figure 1). These dimensions refer to: (1) internal states of psychological distress such as anxiety and depression; (2) relationships and the external world: lack of desire to talk to others and loneliness; and (3) wellbeing/tranquility. It should be noted that this result might be attributed to the original structuring procedure of the PDI (Morasso et al., 1996) and the statistical analysis performed. Indeed, Morasso et al. (1996) did not test the factorial structure of the PDI but only evaluated its internal consistency—and, since Cronbach’s alpha resulted good, a single-factor structure was inferred. In addition, despite the original validation article contained three studies attempting to validate the PDI, each of them was conducted relatively small sample sizes and thus, results may be biased.

In addition, scientific literature showed that modern techniques of statistical analysis—such as EGA (Golino and Epskamp, 2017)—are more strongly adequate and precise to extract the right number of factors of a questionnaire than classical exploratory techniques (e.g., exploratory factor analysis, EFA) (Golino and Demetriou, 2017; Christensen and Golino, 2019, 2020; Christensen, 2020; Christensen et al., 2020a; Golino et al., 2020b).

Results from Study 1 shed new light on the dimensional/factor structure of the PDI—inspiring the development of its revised (shortened) version: the PDI-R (Study 2).

For the validation and study of the psychometric properties of the PDI-R redundant items were removed: in particular, the two reversed scored items (PDI: item#2 and item#6). Moreover, also the three items that did not achieve the Item Stability threshold (PDI: item#4, item#8, and item#12) were removed. Consequently, only two first-order latent dimensions were retained: internal and external distress; and a second-order general distress dimension was hypothesized. Thus the factorial structure of a new, brief but solid questionnaire was tested on a large sample of oncological outpatients.

A bi-factor structure was specified, and the CFA successfully confirmed that all the 8 items of the PDI-R loaded onto the supposed first-order latent factor (internal and external distress) and all the items loaded on the general dimension of distress (Figure 2). CFA also revealed that the PDI-R had good structural validity with good fit indices (Muthén and Muthén, 1998–2017; Hoyle, 2012; van de Schoot et al., 2012; Brown, 2015).

Moreover, considering that the PDI-R is a new scale, the proposed factorial structure was compared with possible competing models: a single-factor model and a two-first order model. Results of model comparison demonstrated that the proposed bi-factor structure is the best factorial solution for the PDI-R.

Item discrimination power was also tested. Results showed that each of the first four items composing the PDI-R well discriminated between subjects with low or high internal distress. At the same time, for the external subscale, the item discrimination power indicated that each of the second four items comprising the PDI-R well discriminated between subjects with low or high external distress. These results suggest the goodness of the items to discriminate between different types of distress in the individuals, and the ability of each item to represent its latent construct.

Reliability analyses were also performed, providing satisfying results for the internal, external, and general distress scale. In addition, the 1-month test-retest reliability provided good results – as shown by the two-way mixed ICCs (Pursey et al., 2016).

Convergent validity analyses were also performed. In line with the scientific literature, significant correlations were found between the PDI-R total score and other well-consolidated measures of psychological suffering such as the HADS and the DT (Wise et al., 2013; Riba et al., 2019). Strong correlations were found between the PDI-R general distress scale, the HADS total score (r = 0.775), its subscales (HADS anxiety, r = 0.713; HADS depression, r = 0.705), and the DT (r = 0.576). These correlations suggest a strong association of distress (measured by the PDI-R) with psychological and emotive difficulties in oncological patients—due to the possible presence of people with severe diagnoses and related preoccupations and fears in the sample (Rossi et al., 2021).

Moreover, MI was tested to explore at which level (structural vs. loadings vs. intercepts/thresholds, means) there were differences across gender (males vs. females) and age (≤ 64 vs. ≥ 65). MI analysis showed that latent means invariance was achieved. This suggests that the eight items were equivalently related to the latent distress factors across each sample, and indicates that samples had the same expected item response at the same absolute level of the trait. Thus, males and females as well as patients with different age interpreted the PDI-R items in the same way (the factorial structure was equal across samples),with the same strength (items were related to the latent construct equally between samples), with the same “starting point” (item thresholds were similar among samples) and had the same latent mean of the construct (latent means were similar across samples). Consequently, experienced psychological distress measured by the PDI-R can be evaluated and compared between males and females as well as between patients with different age (≤ 64 vs. ≥ 65).

In addition, results from the ROC analyses showed that the PDI-R is an excellent screening/diagnostic tool for the detection of psychological distress. Indeed, the PDI-R presented an excellent accuracy in discriminating between distressed and non-distressed oncological outpatients. More in detail, considering the principal aim of a screening tool—i.e., to capture the majority of “positives” patients (Zou et al., 2012)—the PDI-R showed excellent sensitivity properties. At the same time, it is important to underline that—in contrast to many screening tools—specificity is not compromised and it showed very good values. These results suggest that—in spite of the small number of items (n = 8)—the PDI-R is an excellent instrument for the screening of distress in samples of cancer patients. In this regard, it should be emphasized that the proposed cut-offs are those with the best average ratio between sensitivity and specificity on a sample of 902 cancer patients and should therefore be considered as guidelines for patient classification—which, however, cannot replace the clinical interview. However, these cut-offs should not be used in an absolutely inflexible way: they should be evaluated according to the situation and what is to be obtained from the screening assessment (Cohen, 1988).

Consequently, in order to guide the interpretation of the score obtained on the “general distress scale” of the PDI-R, normative scores (T-scores) and the percentiles distribution were calculated—on a large sample of 1,153 patients with cancer. Using the cut-offs that emerged from Study 2 (i.e., PDI-R ≥ 15: moderate distress; PDI-R ≥ 18: severe distress) it is possible to observe that the thresholds for moderate and severe distress are, respectively, located in the presence of the 60th and 75th percentiles—underlining the adequacy of the previously proposed guidelines.

Despite these interesting findings, several limitations have to be highlighted. First, despite the sample size being adequate to perform a CFA, the provided bi-factor model might be expensive in terms of sample-to-parameter ratio. Consequently, cross-cultural adaptation and validation studies aimed at computing the factorial structure of the PDI-R should consider enrolling an appropriate number of outpatients. Moreover, the number of participants (n = 40) to which the PDI-R was re-administered was enough for the assessment of the test-retest reliability but far from being adequate for a longitudinal MI analysis. In addition, the sample of the present research was only composed by oncological outpatients: future studies should investigate the factorial structure of this inventory in other categories of (hospitalized) patients.

Despite these limitations, it is also important to emphasize some strengths of the present research work. First of all, this work is based on a two-step methodology that has already been consolidated in previous research. Second, both questionnaires (i.e., PDI and PDI-R) were extensively analyzed using—for example—innovative statistics (e.g., EGA). Third, both studies are based on large sample sizes, which makes it possible to use robust and reliable statistics. Fourth, the study of factorial invariance showed that the PDI-R is an instrument that is widely applicable to heterogeneous contexts with patients of both sexes or different age groups.

Still, this contribution shows that the PDI-R might be a good instrument for the assessment of psychological distress in oncological settings and that it might also be used for research purposes.

Last, the PDI-R might also represent a starting point for the assessment of psychological distress and the planning of (psycho-)oncological treatments aimed to reduce of the individuals’ psychological suffering and the general health status of patients in oncological settings.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  AIRTUM Working Group (2021). I numeri del Cancro in Italia 2021.Milan: AIOM.

  • Google Scholar

• 2

  AmadoriD.BellaniM. L.BruzziP.CasaliP. G.GrassiL.MorassoG.et al (2002). Psiconcologia.Milano: Masson.

  • Google Scholar

• 3

  AnnunziataM. A.MuzzattiB.AltoéG. (2011). Defining hospital anxiety and depression scale (HADS) structure by confirmatory factor analysis: a contribution to validation for oncological settings.Ann. Oncol.222330–2333. 10.1093/annonc/mdq750

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4

  BentlerP. M.ChouC. H. (1987). Practical issues in structural modeling.Sociol. Methods Res.1678–117. 10.1177/0049124187016001004

  • CrossRef
  • Google Scholar

• 5

  BerchtoldA. (2016). Test–retest: agreement or reliability?Methodol. Innov.9:2059799116672875. 10.1177/2059799116672875

  • CrossRef
  • Google Scholar

• 6

  BlondelV. D.GuillaumeJ.-L.LambiotteR.LefebvreE. (2008). Fast unfolding of communities in large networks.J. Stat. Mech. Theory Exp.2008:10008. 10.1088/1742-5468/2008/10/p10008

  • CrossRef
  • Google Scholar

• 7

  BoomsmaA.HooglandJ. J. (2001). “The robustness of LISREL modeling revisited. A Festschrift in honor of Karl Jöreskog,” in Structural Equation Models: Present and Future, edsCudeckR.du ToitS.SorbomD. (Lincolnwood, IL: Scientific Software International), 139–168.

  • Google Scholar

• 8

  BottesiG.MarchettiI.SicaC.GhisiM. (2020). What is the internal structure of intolerance of uncertainty? A network analysis approach.J. Anxiety Disord.75:102293. 10.1016/j.janxdis.2020.102293

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9

  BrainK.WilliamsB.IredaleR.FranceL.GrayJ. (2006). Psychological distress in men with breast cancer.J. Clin. Oncol.2495–101. 10.1200/jco.2006.10.064

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10

  BrownT. A. (2015). Confirmatory Factor Analysis for Applied Research.New York, NY: The Guilford Press.

  • Google Scholar

• 11

  BulliF.MiccinesiG.MaruelliA.KatzM.PaciE. (2008). The measure of psychological distress in cancer patients: the use of Distress Thermometer in the Oncological Rehabilitation Center of Florence.Support. Care Cancer17771. 10.1007/s00520-008-0543-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12

  BultzB. D.CarlsonL. E. (2006). Emotional distress: the sixth vital sign—future directions in cancer care.Psychooncology1593–95. 10.1002/pon.1022

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  CaminitiC.AnnunziataM. A.VerusioC.PintoC.AiroldiM.AragonaM.et al (2021). Effectiveness of a psychosocial care quality improvement strategy to address quality of life in patients with cancer: the HuCare2 stepped-wedge cluster randomized trial.JAMA Netw. Open4:e2128667. 10.1001/jamanetworkopen.2021.28667

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14

  CaminitiC.IezziE.PassalacquaR. (2017). Effectiveness of the HuCare Quality Improvement Strategy on health-related quality of life in patients with cancer: study protocol of a stepped-wedge cluster randomised controlled trial (HuCare2 study).BMJ Open7:e016347. 10.1136/bmjopen-2017-016347

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  CarlsonL. E.BultzB. D. (2003). Cancer distress screening: needs, models, and methods.J. Psychosom. Res.55403–409. 10.1016/S0022-3999(03)00514-2

  • CrossRef
  • Google Scholar

• 16

  CarlsonL. E.AngenM.CullumJ.GoodeyE.KoopmansJ.LamontL.et al (2004). High levels of untreated distress and fatigue in cancer patients.Br. J. Cancer902297–2304. 10.1038/sj.bjc.6601887

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17

  CataniaA. M.Sammut ScerriC.CataniaG. J. (2019). Men’s experience of their partners’ breast cancer diagnosis, breast surgery and oncological treatment.J. Clin. Nurs.281899–1910. 10.1111/jocn.14800

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18

  Chad-FriedmanE.ColemanS.TraegerL. N.PirlW. F.GoldmanR.AtlasS. J.et al (2017). Psychological distress associated with cancer screening: a systematic review.Cancer1233882–3894. 10.1002/cncr.30904

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19

  CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance.Struct. Equ. Modeling9233–255. 10.1207/S15328007SEM0902_5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20

  ChiorriC. (2011). Teoria e Tecnica Psicometrica. Costruire un test Psicologico.New York, NY: McGraw-Hill Companies.

  • Google Scholar

• 21

  ChristensenA. P. (2020). Towards a Network Psychometrics Approach to Assessment: Simulations for Redundancy, Dimensionality, and Loadings.Doctoral dissertation.Chapel Hill, NC: University of North Carolina.

  • Google Scholar

• 22

  ChristensenA. P.GolinoH. F. (2019). Estimating the stability of the number of factors via bootstrap exploratory graph analysis: a tutorial.PsyArXiv [Preprint]. 10.31234/osf.io/9deay

  • CrossRef
  • Google Scholar

• 23

  ChristensenA. P.GolinoH. F. (2020). On the equivalency of factor and network loadings.PsyArXiv [Preprint]. 10.31234/osf.io/xakez

  • CrossRef
  • Google Scholar

• 24

  ChristensenA. P.GarridoL. E.GolinoH. F. (2020a). Unique variable analysis: a novel approach for detecting redundant variables in multivariate data.PsyArXiv [Preprint]. 10.31234/osf.io/4kra2

  • CrossRef
  • Google Scholar

• 25

  ChristensenA. P.GolinoH.SilviaP. J. (2020b). A psychometric network perspective on the validity and validation of personality trait questionnaires.Eur. J. Pers.341095–1108. 10.1002/per.2265

  • CrossRef
  • Google Scholar

• 26

  ClericiC. A.VeneroniL. (2014). La Psicologia Clinica in Ospedale: Consulenza e Modelli di Intervento.Bologna: Il Mulino.

  • Google Scholar

• 27

  CohenJ. (1988). Statistical Power Analysis for the Behavioral Sciences.Hillsdale, NJ: Lawrence Erlbaum Associates.

  • Google Scholar

• 28

  CompenF. R.AdangE. M. M.BisselingE. M.Van der LeeM. L.SpeckensA. E. M. (2018). Exploring associations between psychiatric disorder, psychological distress, and health care utilization in cancer patients.Psychooncology27871–878. 10.1002/pon.4591

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29

  CostantiniG.EpskampS.BorsboomD.PeruginiM.MõttusR.WaldorpL. J.et al (2015). State of the aRt personality research: a tutorial on network analysis of personality data in R.J. Res. Pers.5413–29. 10.1016/j.jrp.2014.07.003

  • CrossRef
  • Google Scholar

• 30

  CrocettiE.CapocacciaR.CasellaC.De LisiV.FerrettiS.FocaF.et al (2006). I Tumori in Italia – Rapporto 2006: Incidenza, Mortalità e Stime.Milano: Edizioni Inferenze.

  • Google Scholar

• 31

  CsardiG.NepuszT. (2006). The igraph software package for complex network research.InterJournal Complex Syst.1695.

  • Google Scholar

• 32

  CutilloA.O’HeaE.PersonS.LessardD.HarralsonT.BoudreauxE. (2017). The distress thermometer: cutoff points and clinical use.Oncol. Nurs. Forum44329–336. 10.1188/17.onf.329-336

  • CrossRef
  • Google Scholar

• 33

  DakanalisA.ZanettiM. A.ClericiM.MadedduF.RivaG.CaccialanzaR. (2013). Italian version of the Dutch Eating Behavior Questionnaire. Psychometric proprieties and measurement invariance across sex, BMI-status and age.Appetite71187–195. 10.1016/j.appet.2013.08.010

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  de VetH. C.TerweeC. B.KnolD. L.BouterL. M. (2006). When to use agreement versus reliability measures.J. Clin. Epidemiol.591033–1039. 10.1016/j.jclinepi.2005.10.015

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  DerogatisL. R.MorrowG. R.FettingJ.PenmanD.PiasetskyS.SchmaleA. M.et al (1983). The prevalence of psychiatric disorders among cancer patients.JAMA249751–757. 10.1001/jama.1983.03330300035030

  • CrossRef
  • Google Scholar

• 36

  DiMatteoM. R.LepperH. S.CroghanT. W. (2000). Depression is a risk factor for noncompliance with medical treatment: meta-analysis of the effects of anxiety and depression on patient adherence.Arch. Intern. Med.1602101–2107. 10.1001/archinte.160.14.2101

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37

  DonovanK. A.GrassiL.McGintyH. L.JacobsenP. B. (2014). Validation of the distress thermometer worldwide: state of the science.Psychooncology23241–250. 10.1002/pon.3430

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38

  EbelR. L. (1965). Measuring Educational Achievement.Englewood Cliffs, NJ: Prentice-Hall.

  • Google Scholar

• 39

  EpskampS. (2017). Network Psychometrics.Unpublished doctoral dissertation.Amsterdam: University of Amsterdam.

  • Google Scholar

• 40

  EpskampS.BorsboomD.FriedE. I. (2018). Estimating psychological networks and their accuracy: a tutorial paper.Behav. Res. Methods50195–212. 10.3758/s13428-017-0862-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41

  EpskampS.CramerA. O. J.WaldorpL. J.SchmittmannV. D.BorsboomD. (2012). qgraph: network visualizations of relationships in psychometric data.J. Stat. Softw.48:18. 10.18637/jss.v048.i04

  • CrossRef
  • Google Scholar

• 42

  FallowfieldL. J. (1988). “The psychological complications of malignant disease,” in Medical Complications of Malignant Disease, edsKayeS. B.RankinE. M.Vol. 2, (London: Balliere Tindall), 461–478

  • Google Scholar

• 43

  FallowfieldL. J.RatcliffeD.JenkinsV.SaulJ. (2001). Psychiatric morbidity and its recognition by doctors in patients with cancer.Br. J. Cancer841011–1015. 10.1054/bjoc.2001.1724

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44

  FerlayJ.ColombetM.SoerjomataramI.MathersC.ParkinD. M.PiñerosM.et al (2019). Estimating the global cancer incidence and mortality in 2018: GLOBOCAN sources and methods.Int. J. Cancer1441941–1953. 10.1002/ijc.31937

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45

  FloraD. B.CurranP. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data.Psychol. Methods9466–491. 10.1037/1082-989X.9.4.466

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  GamerM.LemonJ.FellowsI.SinghP. (2019). irr: Various Coefficients of Interrater Reliability and Agreement. 0.84.1 ed.

  • Google Scholar

• 47

  GaryS.LenhardW.LenhardA. (2021). Modelling norm scores with the cNORM package in R.Psych3501–521. 10.3390/psych3030033

  • CrossRef
  • Google Scholar

• 48

  GilF.GrassiL.TravadoL.TomamichelM.GonzalezJ. R.TheS. G. (2005). Use of distress and depression thermometers to measure psychosocial morbidity among southern European cancer patients.Support. Care Cancer13600–606. 10.1007/s00520-005-0780-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49

  GolinoH. F.ChristensenA. P. (2020). EGAnet: Exploratory Graph Analysis – A Framework for Estimating the Number of Dimensions in Multivariate Data Using Network Psychometrics. R Package Version 0.9.7. ed.

  • Google Scholar

• 50

  GolinoH. F.DemetriouA. (2017). Estimating the dimensionality of intelligence like data using Exploratory Graph Analysis.Intelligence6254–70. 10.1016/j.intell.2017.02.007

  • CrossRef
  • Google Scholar

• 51

  GolinoH. F.EpskampS. (2017). Exploratory graph analysis: a new approach for estimating the number of dimensions in psychological research.PLoS One12:e0174035. 10.1371/journal.pone.0174035

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52

  GolinoH.MoulderR.ShiD.ChristensenA. P.GarridoL. E.NietoM. D.et al (2020a). Entropy fit indices: new fit measures for assessing the structure and dimensionality of multiple latent variables.Multivariate Behav. Res.56874–902. 10.1080/00273171.2020.1779642

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 53

  GolinoH.ShiD.ChristensenA. P.GarridoL. E.NietoM. D.SadanaR.et al (2020b). Investigating the performance of exploratory graph analysis and traditional techniques to identify the number of latent factors: a simulation and tutorial.Psychol. Methods25292–320. 10.1037/met0000255

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 54

  GolinoH. F.ThiyagarajanJ. A.SadanaR.TelesM.ChristensenA. P.BokerS. M. (2020c). Investigating the broad domains of intrinsic capacity, functional ability and environment: an exploratory graph analysis approach for improving analytical methodologies for measuring healthy aging.PsyArXiv [Preprint]. 10.31234/osf.io/hj5mc

  • CrossRef
  • Google Scholar

• 55

  GranekL.NakashO.AriadS.ShapiraS.Ben-DavidM. (2019). Cancer patients’ mental health distress and suicidality: impact on oncology healthcare workers and coping strategies.Crisis40429–436. 10.1027/0227-5910/a000591

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  GrassiL.RibaM. (2012). Clinical Psycho-Oncology: An International Perspective.Oxford: John Wiley & Sons.

  • Google Scholar

• 57

  GrassiL.SabatoS.RossiE.MarmaiL.BiancosinoB. (2009). Affective syndromes and their screening in cancer patients with early and stable disease: Italian ICD-10 data and performance of the Distress Thermometer from the Southern European Psycho-Oncology Study (SEPOS).J. Affect. Disord.114193–199. 10.1016/j.jad.2008.07.016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58

  HerschbachP.KellerM.KnightL.BrandlT.HuberB.HenrichG.et al (2004). Psychological problems of cancer patients: a cancer distress screening with a cancer-specific questionnaire.Br. J. Cancer91504–511. 10.1038/sj.bjc.6601986

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 59

  HollandJ. C. (1992). Psycho-oncology: overview, obstacles and opportunities.Psychooncology11–13. 10.1002/pon.2960010103

  • CrossRef
  • Google Scholar

• 60

  HollandJ. C.BultzB. D. (2007). The NCCN guideline for distress management: a case for making distress the sixth vital sign.J. Natl. Compr. Cancer Netw.53–7. 10.6004/jnccn.2007.0003

  • CrossRef
  • Google Scholar

• 61

  HowellD. C. (2013). Statistical Methods for Psychology.Belmont, CA: Wadsworth, Cengage Learning.

  • Google Scholar

• 62

  HoyleR. H. (2012). Handbook of Strucural Equation Modeling.New York, NY: The Guilford Press.

  • Google Scholar

• 63

  HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives.Struct. Equ. Modeling61–55. 10.1080/10705519909540118

  • CrossRef
  • Google Scholar

• 64

  IaniL.LauriolaM.CostantiniM. (2014). A confirmatory bifactor analysis of the hospital anxiety and depression scale in an Italian community sample.Health Qual. Life Outcomes1284. 10.1186/1477-7525-12-84

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65

  IsingH. K.VelingW.LoewyR. L.RietveldM. W.RietdijkJ.DragtS.et al (2012). The validity of the 16-item version of the Prodromal Questionnaire (PQ-16) to screen for Ultra High Risk of developing psychosis in the general help-seeking population.Schizophr. Bull.381288–1296. 10.1093/schbul/sbs068

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 66

  JacobsenP. B. (2007). Screening for psychological distress in cancer patients: challenges and opportunities.J. Clin. Oncol.254526–4527. 10.1200/jco.2007.13.1367

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 67

  JemalA.SiegelR.WardE.MurrayT.XuJ.SmigalC.et al (2006). Cancer Statistics, 2006.CA Cancer J. Clin.56106–130. 10.3322/canjclin.56.2.106

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 68

  JonesP. (2020). networktools: Tools for Identifying Important Nodes in Networks. 1.2.3 ed.

  • Google Scholar

• 69

  KlineR. B. (2016). Principles and Practice of Structural Equation Modeling.New York, NY: The Guilford Press.

  • Google Scholar

• 70

  KooT. K.LiM. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research.J. Chiropr. Med.15155–163. 10.1016/j.jcm.2016.02.012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 71

  KrebberA. M.BuffartL. M.KleijnG.RiepmaI. C.de BreeR.LeemansC. R.et al (2014). Prevalence of depression in cancer patients: a meta-analysis of diagnostic interviews and self-report instruments.Psychooncology23121–130. 10.1002/pon.3409

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72

  KwakM.ZebrackB. J.MeeskeK. A.EmbryL.AguilarC.BlockR.et al (2013). Trajectories of psychological distress in adolescent and young adult patients with cancer: a 1-year longitudinal study.J. Clin. Oncol.312160–2166. 10.1200/jco.2012.45.9222

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73

  LenhardA.LenhardW.SuggateS.SegererR. (2018). A continuous solution to the norming problem.Assessment25112–125. 10.1177/1073191116656437

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 74

  LenhardW.LenhardA.GaryS. (2018). cNorm—Generating Continuous Test Norms.

  • Google Scholar

• 75

  LionettiF.KeijsersL.DellagiuliaA.PastoreM. (2016). Evidence of factorial validity of parental knowledge, control and solicitation, and adolescent disclosure scales: when the ordered nature of Likert scales matters.Front. Psychol.7:941. 10.3389/fpsyg.2016.00941

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 76

  LouisonR.LeeJ.RoeV.GhoshC. (2019). Effects of a holistic, patient-centered approach on breast cancer relative dose intensity.Adv. Integr. Med.678–86. 10.1016/j.aimed.2018.06.005

  • CrossRef
  • Google Scholar

• 77

  MaguireP. (1985). The psychological impact of cancer.Br. J. Hosp. Med.34100–103.

  • Google Scholar

• 78

  MairP. (2018). Modern Psychometrics with R.Cham: Springer International Publishing.

  • Google Scholar

• 79

  MannariniS.BoffoM. (2014). The relevance of security: a latent domain of attachment relationships.Scand. J. Psychol.5553–59. 10.1111/sjop.12091

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 80

  MannariniS.BalottinL.MunariC.GattaM. (2017). Assessing conflict management in the couple: the definition of a latent dimension.Fam. J.2513–22. 10.1177/1066480716666066

  • CrossRef
  • Google Scholar

• 81

  MannariniS.BoffoM.BertucciV.AndrisaniA.AmbrosiniG. (2013). A Rasch-based dimension of delivery experience: spontaneous vs. medically assisted conception.J. Clin. Nurs.222404–2416. 10.1111/jocn.12264

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 82

  MarchettiI. (2019). Hopelessness: a network analysis.Cogn. Ther. Res.43611–619. 10.1007/s10608-018-9981-y

  • CrossRef
  • Google Scholar

• 83

  MarconiM.BianchiS.MinelliI.VerusioC. (2020). Il distress psicologico in un campione di pazienti oncologici: l’effetto di età e tipo di reparto.Ric. Psicol.41055–1074. 10.3280/RIP2020-004005

  • CrossRef
  • Google Scholar

• 84

  MarshH. W.BallaJ. R.McDonaldR. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: the effect of sample size.Psychol. Bull.103391–410. 10.1037/0033-2909.103.3.391

  • CrossRef
  • Google Scholar

• 85

  MasséR. (2000). Qualitative and quantitative analyses of psychological distress: methodological complementarity and ontological incommensurability.Qual. Health Res.10411–423. 10.1177/104973200129118426

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 86

  McCorkleR.YoungK. (1978). Development of a symptom distress scale.Cancer Nurs.1373–378.

  • Google Scholar

• 87

  McDonaldR. P. (1999). Test Theory: A Unified Treatment.Mahwah, NJ: Lawrence Erlbaum Associates.

  • Google Scholar

• 88

  MeredithW. (1993). Measurement invariance, factor analysis and factorial invariance.Psychometrika58525–543. 10.1007/bf02294825

  • CrossRef
  • Google Scholar

• 89

  MillsapR. E. (2012). Statistical Approaches to Measurement Invariance.New York, NY: Routledge.

  • Google Scholar

• 90

  MillsapR. E.Yun-TeinJ. (2004). Assessing factorial invariance in ordered-categorical measures.Multivariate Behav. Res.39479–515. 10.1207/s15327906mbr3903_4

  • CrossRef
  • Google Scholar

• 91

  MitchellA. J. (2010). Short screening tools for cancer-related distress: a review and diagnostic validity meta-analysis.J. Natl. Compr. Cancer Netw.8487–494. 10.6004/jnccn.2010.0035

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 92

  MooreS. (2010). Adherence to oral therapies for cancer: barriers and models for change.J. Adv. Pract. Oncol.1155–164. 10.6004/jadpro.2010.1.3.2

  • CrossRef
  • Google Scholar

• 93

  MorassoG.CostantiniM.BaraccoG.BorreaniC.CapelliM. (1996). Assessing psychological distress in cancer patients: validation of a self-administered questionnaire.Oncology53295–302. 10.1159/000227576

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 94

  MullarkeyM. C.MarchettiI.BeeversC. G. (2019). Using network analysis to identify central symptoms of adolescent depression.J. Clin. Child Adolesc. Psychol.48656–668. 10.1080/15374416.2018.1437735

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 95

  MullarkeyM. C.MarchettiI.BluthK.CarlsonC. L.ShumakeJ.BeeversC. G. (2018). Symptom centrality and infrequency of endorsement identify adolescent depression symptoms more strongly associated with life satisfaction.PsyArXiv[preprint]. 10.31234/osf.io/79hcj

  • CrossRef
  • Google Scholar

• 96

  MuthénB.AsparouhovT. (2002). Latent variable analysis with categorical outcomes: multiple-group and growth modeling in Mplus.Mplus Web Notes91–22. 10.1007/s10519-008-9237-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 97

  MuthénL. K.MuthénB. O. (1998-2017). Mplus User’s Guide.Los Angeles, CA: Muthén & Muthén.

  • Google Scholar

• 98

  MuzzattiB.AnnunziataM. A. (2012). Psychological distress screening in cancer patients: psychometric properties of tools available in Italy.Tumori J.98501–509. 10.1177/030089161209800416

  • CrossRef
  • Google Scholar

• 99

  NaserA. Y.HameedA. N.MustafaN.AlwafiH.DahmashE. Z.AlyamiH. S.et al (2021). Depression and anxiety in patients with cancer: a cross-sectional study.Front. Psychol.12:585534. 10.3389/fpsyg.2021.585534

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 100

  NewellS.Sanson-FisherR. W.GirgisA.BonaventuraA. (1998). How well do Medical oncologists’ perceptions reflect their patients’ reported physical and psychosocial problems?Cancer831640–1651. 10.1002/(SICI)1097-0142(19981015)83:8<1640::AID-CNCR21>3.0.CO;2-%23

  • CrossRef
  • Google Scholar

• 101

  NgC. G.MohamedS.KaurK.SulaimanA. H.ZainalN. Z.TaibN. A. (2017). Perceived distress and its association with depression and anxiety in breast cancer patients.PLoS One12:e0172975. 10.1371/journal.pone.0172975

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 102

  NortonT. R.ManneS. L.RubinS.CarlsonJ.HernandezE.EdelsonM. I.et al (2004). Prevalence and predictors of psychological distress among women with ovarian cancer.J. Clin. Oncol.22919–926. 10.1200/jco.2004.07.028

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 103

  O’DonnellE. (2013). The distress thermometer: a rapid and effective tool for the oncology social worker.Int. J. Health Care Qual. Assur.26353–359. 10.1108/09526861311319573

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 104

  OliveriS.ArnaboldiP.PizzoliS. F. M.FaccioF.GiudiceA. V.SangalliC.et al (2019). PTSD symptom clusters associated with short- and long-term adjustment in early diagnosed breast cancer patients.Ecancermedicalscience13917–917. 10.3332/ecancer.2019.917

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 105

  OlssønI.MykletunA.DahlA. A. (2005). The hospital anxiety and depression rating scale: a cross-sectional study of psychometrics and case finding abilities in general practice.BMC Psychiatry5:46. 10.1186/1471-244x-5-46

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 106

  PallantJ. (2013). SPSS Survival Manual.New York, NY: McGraw-Hill Education.

  • Google Scholar

• 107

  PandeyM.DeviN.ThomasB. C.Vinod KumarS.KrishnanR.RamdasK. (2007). Distress overlaps with anxiety and depression in patients with head and neck cancer.Psychooncology16582–586. 10.1002/pon.1123

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 108

  PanzeriA.BertaminiM.ButterS.LevitaL.Gibson-MillerJ.VidottoG.et al (2021a). Factors impacting resilience as a result of exposure to COVID-19: the ecological resilience model.PLoS One16:e0256041. 10.1371/journal.pone.0256041

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 109

  PanzeriA.KomiciK.CeruttiP.SaccoD.PistonoM.FerrarioS. R. (2021b). Gender differences and long-term outcome of over 75 elderlies in cardiac rehabilitation: highlighting the role of psychological and physical factors through a secondary analysis of a cohort study.Eur. J. Phys. Rehabil. Med.57288–297. 10.23736/s1973-9087.21.06484-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 110

  PanzeriA.Rossi FerrarioS.CeruttiP. (2021c). Psychological differences among healthcare workers of a rehabilitation institute during the COVID-19 pandemic: a two-step study.Front. Psychol.12:636129. 10.3389/fpsyg.2021.636129

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 111

  ParolaA.FuscoL.MarcionettiJ. (2022). The parental career-related behaviors questionnaire (PCB): psychometric properties in adolescents and young adults in the Italian context.Curr. Psychol.10.1007/s12144-022-02764-1

  • CrossRef
  • Google Scholar

• 112

  PartridgeA. H.AvornJ.WangP. S.WinerE. P. (2002). Adherence to therapy with oral antineoplastic agents.JNCI J. Natl. Cancer Inst.94652–661. 10.1093/jnci/94.9.652

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 113

  PassalacquaR.AnnunziataM. A.BorreaniC.DiodatiF.IsaL.SaleriJ.et al (2016). Feasibility of a quality improvement strategy integrating psychosocial care into 28 medical cancer centers (HuCare project).Support. Care Cancer24147–155. 10.1007/s00520-015-2756-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 114

  PeckA. (1972). Emotional reactions to having cancer.Am. J. Roentgenol.114591–599. 10.2214/ajr.114.3.591

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 115

  PepeM. S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction.New York, NY: Oxford Universtiy Press.

  • Google Scholar

• 116

  PietrabissaG.RossiA.BorrelloM.ManzoniG. M.MannariniS.CastelnuovoG.et al (2020a). Development and validation of a self-determination theory-based measure of motivation to exercise and diet in children.Front. Psychol.11:1299. 10.3389/fpsyg.2020.01299

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 117

  PietrabissaG.RossiA.SimpsonS.TagliagambeA.BertuzziV.VolpiC.et al (2020b). Evaluation of the reliability and validity of the Italian version of the schema mode inventory for eating disorders: short form for adults with dysfunctional eating behaviors.Eat. Weight Disord.25553–565. 10.1007/s40519-019-00644-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 118

  PirlW. F. (2004). Evidence report on the occurrence, assessment, and treatment of depression in cancer patients.J. Natl. Cancer Inst. Monogr.200432–39. 10.1093/jncimonographs/lgh026

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 119

  PonsP.LatapyM. (2006). Computing communities in large networks using random walks.J. Graph Algorithms Appl.10191–218. 10.7155/jgaa.00124

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 120

  PurseyK. M.CollinsC. E.StanwellP.BurrowsT. L. (2016). The stability of ‘food addiction’ as assessed by the Yale Food Addiction Scale in a non-clinical population over 18-months.Appetite96533–538. 10.1016/j.appet.2015.10.015

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 121

  R Core Team (2014). The R Project for Statistical Computing.Vienna: R Foundation for Statistical Computing.

  • Google Scholar

• 122

  R Core Team (2017). R: A Language and Environment for Statistical Computing.Vienna: R Foundation for Statistical Computing.

  • Google Scholar

• 123

  RakiciS. Y.KaramanE. (2019). Colorful screams of silent emotions: a study with oncological patients.Indian J. Palliat. Care25361–366. 10.4103/ijpc.ijpc_79_19

  • CrossRef
  • Google Scholar

• 124

  RevelleW. (2018). psych: Procedures for Personality and Psychological Research.Evanston, IL: Northwestern University.

  • Google Scholar

• 125

  RevelleW. (2020). psychTools:Tools to Accompany the ‘psych’ Package for Psychological Research”. 2.0.6 ed.Evanston, IL: Northwestern University.

  • Google Scholar

• 126

  RibaM. B.DonovanK. A.AndersenB.BraunI.BreitbartW. S.BrewerB. W.et al (2019). Distress management, version 3.2019, NCCN clinical practice guidelines in oncology.J. Natl. Compr. Canc. Netw.17:1229. 10.6004/jnccn.2019.0048

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 127

  RidnerS. H. (2004). Psychological distress: concept analysis.J. Adv. Nurs.45536–545. 10.1046/j.1365-2648.2003.02938.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 128

  RobinX.TurckN.HainardA.TibertiN.LisacekF.SanchezJ.et al (2011). pROC: an open-source package for R and S+ to analyze and compare ROC curves.BMC Bioinformatics12:77. 10.1186/1471-2105-12-77

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 129

  RoseL.ClarkeS. P. (2010). Vital signs.AJN Am. J. Nurs.110:11. 10.1097/01.NAJ.0000372049.58200.da

  • CrossRef
  • Google Scholar

• 130

  RosseelY. (2012). lavaan: an R package for structural equation modeling.J. Stat. Softw.481–36. 10.18637/jss.v048.i02

  • CrossRef
  • Google Scholar

• 131

  RosseelY.OberskiD.ByrnesJ.VanbrabantL.SavaleiV.MerkleE.et al (2015). Package ‘lavaan’: Latent Variable Analysis. R Package.

  • Google Scholar

• 132

  RosselliM.SalimbeniM. V.BessiC.NesiE.CarusoS.ArborettiD.et al (2015). Screening of distress among hospitalized patients in a department of internal medicine.Asian J. Psychiatry1891–96. 10.1016/j.ajp.2015.08.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 133

  Rossi FerrarioS.PanzeriA.CeruttiP.SaccoD. (2021). The psychological experience and intervention in post-acute COVID-19 inpatients.Neuropsychiatr. Dis. Treat.17413–422. 10.2147/NDT.S283558

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 134

  RossiA. A.ManzoniG. M.PietrabissaG.Di PauliD.MannariniS.CastelnuovoG. (2022). Weight stigma in patients with overweight and obesity: validation of the Italian Weight Self-Stigma Questionnaire (WSSQ).Eat. Weight Disord.10.1007/s40519-022-01385-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 135

  RossiA. A.MarconiM.TacciniF.VerusioC.MannariniS. (2021). From fear to hopelessness: the buffering effect of patient-centered communication in a sample of oncological patients during covid-19.Behav. Sci.1187. 10.3390/bs11060087

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 136

  SachsM. C. (2017). plotROC: a tool for plotting ROC curves.J. Stat. Softw.791–19. 10.18637/jss.v079.c02

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 137

  SaritasS. C.KavakF.AksoyA.SaritasS. (2017). Examination of the care burden of caregivers of oncology patients and the perceived social support from family.Int. J. Caring Sci.10448–455.

  • Google Scholar

• 138

  SavillM.D’AmbrosioJ.CannonT. D.LoewyR. L. (2018). Psychosis risk screening in different populations using the Prodromal Questionnaire: a systematic review.Early Interv. Psychiatry123–14. 10.1111/eip.12446

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 139

  SellickS. M.EdwardsonA. D. (2007). Screening new cancer patients for psychological distress using the hospital anxiety and depression scale.Psychooncology16534–542. 10.1002/pon.1085

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 140

  semTools Contributors (2016). semTools: Useful Tools for Structural Equation Modeling. R Package Version 0.4-14.

  • Google Scholar

• 141

  SingerS.KuhntS.GötzeH.HaussJ.HinzA.LiebmannA.et al (2009). Hospital anxiety and depression scale cutoff scores for cancer patients in acute care.Br. J. Cancer100908–912. 10.1038/sj.bjc.6604952

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 142

  SpiegelD.Giese-DavisJ. (2003). Depression and cancer: mechanisms and disease progression.Biol. Psychiatry54269–282. 10.1016/S0006-3223(03)00566-3

  • CrossRef
  • Google Scholar

• 143

  StarkD. P. H.HouseA. (2000). Anxiety in cancer patients.Br. J. Cancer831261–1267. 10.1054/bjoc.2000.1405

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 144

  SternA. F. (2014). The hospital anxiety and depression scale.Occup. Med.64393–394. 10.1093/occmed/kqu024

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 145

  SwetsJ. A. (1998). Measuring the accuracy of diagnostic systems.Science2401285–1293. 10.1126/science.3287615

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 146

  TabachnickB. G.FidellL. S. (2014). Using Multivariate Statistics.Harlow: Pearson.

  • Google Scholar

• 147

  TeoI.KrishnanA.LeeG. L. (2019). Psychosocial interventions for advanced cancer patients: a systematic review.Psychooncology281394–1407. 10.1002/pon.5103

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 148

  TomarkenA. J.WallerN. G. (2005). Structural equation modeling: strengths, limitations, and misconceptions.Annu. Rev. Clin. Psychol.131–65. 10.1146/annurev.clinpsy.1.102803.144239

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 149

  TraskP. C.PatersonA.RibaM.BrinesB.GriffithK.ParkerP.et al (2002). Assessment of psychological distress in prospective bone marrow transplant patients.Bone Marrow Transplant.29917–925. 10.1038/sj.bmt.1703557

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 150

  van de SchootR.LugtigP.HoxJ. (2012). A checklist for testing measurement invariance.Eur. J. Dev. Psychol.9486–492. 10.1080/17405629.2012.686740

  • CrossRef
  • Google Scholar

• 151

  VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research.Organ. Res. Methods34–70. 10.1177/109442810031002

  • CrossRef
  • Google Scholar

• 152

  VodermaierA.LindenW.SiuC. (2009). Screening for emotional distress in cancer patients: a systematic review of assessment instruments.JNCI J. Natl. Cancer Inst.1011464–1488. 10.1093/jnci/djp336

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 153

  WeiT.SimkoV. (2017). R package “corrplot”: Visualization of a Correlation Matrix. 0.84 ed.

  • Google Scholar

• 154

  WHO (2014). Global Status Report on Noncommunicable Diseases 2014.Geneva: World Health Organisation.

  • Google Scholar

• 155

  WiseT. N.BiondiM.CostantiniA. (2013). Psycho-oncology.Arlington, VA: American Psychiatric Pub.

  • Google Scholar

• 156

  YuC. Y. (2002). Evaluating Cutoff Criteria of Model Fit Indices for Latent Variable Models with Binary and Continuous Outcomes.Ph.D. thesis.Oakland, CA: University of California.

  • Google Scholar

• 157

  ZaboraJ.BlanchardC. G.SmithE. D.RobertsC. S.GlajchcnM.SharpJ. W.et al (1997). Prevalence of psychological distress among cancer patients across the disease continuum.J. Psychosoc. Oncol.1573–87. 10.1300/J077v15n02_05

  • CrossRef
  • Google Scholar

• 158

  ZaboraJ.BrintzenhofeSzocK.CurbowB.HookerC.PiantadosiS. (2001). The prevalence of psychological distress by cancer site.Psychooncology1019–28. 10.1002/1099-1611(200101/02)10:1<19::aid-pon501>3.0.co;2-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 159

  ZhouX. H.ObuchowskiN. A.McClishD. (2002). Statistical Methods in Diagnostic Medicine.New York, NY: Wiley.

  • Google Scholar

• 160

  ZigmondA. S.SnaithR. P. (1983). The hospital anxiety and depression scale.Acta Psychiatr. Scand.67361–370. 10.1111/j.1600-0447.1983.tb09716.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 161

  ZouK. H.LiuA.BandosA. I.Ohno-MachadoL.RocketteH. E. (2012). Statistical Evaluation of Diagnostic Performance: Topics in ROC Analysis.Boca Raton, FL: CRC press.

  • Google Scholar

• 162

  ZweigM. H.CampbellG. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.Clin. Chem.39561–577. 10.1093/clinchem/39.4.561

  • Pubmed Abstract
  • CrossRef
  • Google Scholar