Multicenter ethics approval was obtained from the National Research Ethics Service (NRES) South West Bristol, UK (ref 14/SW/0033) followed by individual “research and development” approvals from all the participating centers. A signed informed consent was obtained from all the study participants.

The study was a prospective longitudinal observational study. All study participants were assessed on two occasions, at baseline (t1—assessment 1) and after 7 days (t2—assessment 2). The time interval of 7 days between assessments was chosen to minimize the “learning effect” (Salek, 1992). The choice of 7 days period was further confirmed after discussing it with the patient research partner and the clinicians (hematologist) involved in the study design. The selection of the optimal time between the two assessments is very crucial as it will help in avoiding the overestimation (if it is shorter) and underestimation (if it is longer) of reliability. Patients who were receiving treatment or had completed their course of treatment may have a drastic change in their symptoms and HRQoL in a short period of time; hence, a slight difference in score was expected between the two assessments.

Patients with different hematological malignancies were recruited from seven secondary care hospitals in the UK. The inclusion criteria were: diagnosed with hematological malignancy as per the most recent WHO classification; at any stage of disease (defined as stable, progressing, and remission)—confirmed by clinical staff; at any stage of the treatment (due to start the treatment, on treatment, or finished treatment); able to read and write in English; ability to give written informed consent; and aged 17 or above. The exclusion criteria were: patients whose diagnosis was not confirmed; aged below 17; unable to read and write in English; patients with compromised mental capacity; and unavailable to complete the second assessment after 7 days.

The HM-PRO, a hematological malignancy specific patient-reported outcome measure, is a newly developed composite measure consisting of two scales: Part A (impact) and Part B (signs and symptoms) (Goswami et al., 2016; Goswami et al., 2018a; Goswami et al., 2018b; Goswami et al., 2019b; Goswami et al., 2020c). Part A measures the impact of hematological malignancy and its treatment on a patient’s HRQoL, and Part B captures the severity of different disease symptoms and treatment side effects. Part A has a total of 24 items in four domains: physical behavior (7); social well-being (3); emotional behavior (11); and eating and drinking habits (3) rated on a 3-point Likert scale (0 = not at all, 1 = A little, and 2 = A lot), and “not applicable” as a separate response option. Part B consists of 18 items in a single domain, with 3-point severity Likert scale (0 = not, 1 = Mild, and 2 = Severe). The third items of the “Eating and drinking habits” domain in Part A, i.e., “My drinking habits have changed”, and the ninth item of Part B “I have skin problems (e.g. itching, bruises, rashes, etc.)” are not included in the scoring system due to misfit in Rasch model but were collected for additional information (Goswami et al., 2017; Goswami et al., 2018b). The HM-PRO has demonstrated good construct validity of both convergent and divergent type comparing it with existing closely and distantly related PROs (Goswami et al., 2020a).

During the first assessment, patients were approached by the clinic/research nurse in an out-patient/day-care or in-patient setting, who explained in brief about the study and asked their willingness to participate. Those who agreed were asked to complete the HM-PRO after reading the patient information sheet and giving written informed consent. After completing the first assessment, all the patients were provided with a package containing the HM-PRO for the second assessment and a freepost self-addressed envelope to return the completed instrument after 7 days. Other relevant patient demographic information and preferred contact details were collected. On the 6^th day following the initial assessment, patients were reminded via text/calls/emails and asked to complete the instrument on the 7^th day and use the freepost envelope provided in the pack to send it back to the research team.

The data entry was performed in Microsoft Excel and 20% entries were randomly selected for cross-validation by a reviewer. Cleaning, coding, and analysis of the data were performed using SPSS Windows version 23. Cronbach’s alpha was calculated to measure the internal consistency of the scales and subscales of the HM-PRO, which is the average inter-item correlation (Cronbach, 1951; Frost et al., 2007; Tavakol and Dennick, 2011; Devellis, 2016). The alpha value reflects the extents to which the instrument measures the concept consistently. Cronbach’s alpha value greater than 0.7 was taken as reliable (Tavakol and Dennick, 2011). Spearman’s rank correlation was also calculated to assess the inter-item and item-total partial correlation. A moderate correlation (r_s = 0.2) is expected between items (Streiner and Norman, 2003). Intra-class correlation was calculated to assess the level of agreement between scores from the first assessment (test) and after 7 days (re-test). An ICC value of 1 means that 100% variability is because of difference between patients, and the ICC value of 0 means that all variability is due to within-patient variability and error (Studenic et al., 2016). A 2-way mixed-effects, absolute agreement, multiple raters measurement type of ICC was chosen as per McGraw and Wong convention (Mcgraw and Wong, 1996).

A total of 193 patients with different hematological malignancies (acute lymphoid leukemia = 12; acute myeloid leukemia = 28; chronic lymphoid leukemia = 17; chronic myeloid leukemia = 13; multiple myeloma = 33; indolent non-Hodgkin lymphoma = 16; aggressive non-Hodgkin lymphoma = 22; Hodgkin lymphoma = 14; myelodysplastic syndrome = 15; and myeloproliferative neoplasm = 23), at different stage of disease (stable = 71; remission = 64; and progressing = 58) participated in the study and completed the first assessment (t1). The mean age of patients was 63.3 (± 15.2, range = 18–89 years) and the mean time since diagnosis was 3.8 (± 4.9) years. However, a total of 150 patients completed the second assessment after 7 days (t2) and returned them by post, with a response rate of 77.7%. The data from these 150 patients (mean age 64.6 ± 14.4 years, range 17.9 to 89.2; mean time since diagnosis 4.0 ± 5.1 years, range 15 days to 25.8 years) were used for the test-retest reliability analysis (Table 1 and Figure 1). The highest number of patients (n = 41, 27.3%) were diagnosed less than 6 months prior to the study and the highest number of patients (n = 46, 30.7%) were in the age groups 70–80 years.

The Cronbach’s alpha estimates of the HM-PRO for assessments 1 and 2 (at t1 and t2) were above 0.9 for Part A, and above 0.8 for Part B, showing strong instrument stability. For individual domains of Part A, the lowest alpha value was observed for the social well-being domain for assessment 1 (alpha = 0.70) and assessment 2 (alpha = 0.68). Alpha value was greater than 0.8 (i.e., both assessments) for Physical Behavior, Emotional Behavior, and for Eating and drinking habits. Optimal homogeneity is reflected in moderate inter-item correlation and moderate-to-strong corrected item-total correlations (Streiner and Norman, 2003); hence, this was also examined. The corrected item-total correlation (CITC) of Part A items ranged from 0.42 to 0.75 for assessment 1 and 0.40 to 0.80 for assessment 2, reflecting strong internal consistency. For assessment 1, the lowest correlation was seen for Item “I am worried about dying” and the highest correlation was seen for the item “My eating habits have changed”. For Assessment 2 “I am worried about my appearance” (CITC = 0.40) was the lowest and “I have difficulty leaving the house” (CITC = 0.80) was the highest for Part A. The corrected item-total correlation for Part B ranged from 0.25 to 0.64 for assessment 1 and 0.24 to 0.65 for assessment 2, reflecting mostly moderate internal consistency. The lowest score was observed for item “I have hair loss” (CITC = 0.25) for assessment 1 and “I have night sweats” for assessment 2 (CITC = 0.24). This indicated that the HM-PRO is well-balanced in both Part A and Part B, as no item carried too much weight.

The reproducibility of the HM-PRO score in repeated administration was tested. Patients completed the HM-PRO on two different occasions, at baseline (assessment 1) and a follow-up assessment after 7 days (assessment 2). The level of agreement was very strong for overall Part A: ICC = 0.93 (95% CI = 0.90–0.95), and Part B: ICC = 0.91 (0.88–0.93) (Table 2). The ICC for four domains of Part A ranged from 0.85 to 0.91, with social well-being scoring the lowest (ICC = 0.85, 95% CI = 0.79–0.89) and physical behavior showing the highest agreement (ICC = 0.91, 95% CI = 0.88–0.94). The correlation between the mean score of test and re-test was calculated by Spearman’s correlation coefficient (r_s) (Table 4). The r_s ranged from 0.74 to 0.87, with Part A (r_s = 0.87) and Part B (r_s = 0.84) showing strong correlation. With respect to individual domains, physical behavior showed the highest correlation (r_s = 0.83), followed by emotional behavior (r_s = 0.82), eating and drinking (r_s = 0.75), but the lowest correlation for social well-being (r_s = 0.74).

With respect to agreement between individual items, ICC for all Part A items were greater than 0.7 and ranged from 0.73 to 0.94 (Table 3), with “My sleeping pattern has changed” scoring the lowest (ICC = 0.73, 95% CI = 0.62–0.80) and “I have difficulty with work (or studies)” showing the highest agreement (ICC = 0.94, 95% CI = 0.91–0.96). Further, the ICC for all the items in Part B ranged from 0.62–0.83. A total of four items had ICC lower than 0.7: “I have/had fever” (ICC = 0.62, 95% CI = 0.48–0.73); “I have headaches” (ICC = 0.66, 95% CI = 0.53–0.75); “I have stomach ache” (ICC = 0.69, 95% CI = 0.57–0.77); and “I have infections (e.g., chest, lung, urinary, etc.)” (ICC = 0.69, 95% CI = 0.57–0.77). The item with the highest agreement in Part B was “I feel tired” (ICC = 0.83, 95% CI = 0.77–0.88).

Furthermore, since HM-PRO is developed to be used across different hematological malignancies and stages of disease, it is therefore important to test the reliability of the instrument across these patient groups. The patient data were grouped as per the disease diagnosis and stage of disease, and ICC was calculated for the two assessments 1 and 2 (t1 and t2) to test the agreement. With respect to the stage of disease, the ICC was greater than 0.7 for all the four domains and overall score of Part A and Part B (Table 4) confirming the reliability of the HM-PRO across the three stages of disease. The highest agreement was observed for physical behavior domain for “stable” (ICC = 0.92, 0.86–0.95) and “remission” (ICC = 0.91, 0.83–0.95) stage of disease, whereas emotional behavior (ICC = 0.93, 0.87–0.96) had the highest ICC for “progressing” stage of disease. For the overall scores of Parts A and B, ICC was greater than 0.8 showing strong agreement between the two assessments across three stages of disease (Table 4).

With respect to the reliability testing across disease diagnosis, the ICC was greater than 0.80 for the overall score of Parts A and B, for all the 10 diagnoses, confirming strong reliability. Further, ICC was greater than 0.70 for all for domains across 10 diagnoses, but 0.53 for “eating and drinking” domain, demonstrating that the HM-PRO possesses acceptable reliability in each one of the 10 hematological malignancy (Table 5).

Following confirmation of the internal consistency and test-retest reliability, Spearman’s rank correlation coefficients were calculated for individual items in Part A and Part B, between the assessments 1 and 2 (t1 and t2). With respect to Part A, the correlation coefficient (r_s) ranged from 0.58 to 0.86, showing a moderate to a strong relationship. The lowest r_s value was observed for the item “My eating habits have changed” (r_s = 0.58). During treatment, patients experienced frequent changes due to chemotherapy, especially in the “sense of taste”. Because of this the lower correlation can be justified for “eating and drinking” habits in the recall period “today”. With respect to individual items in Part B, the correlation coefficient ranged from 0.49 to 0.72. The lower range is expected because of two reasons: first, the signs and symptoms change very frequently; second, the recall period of Part B is the last 3 days, and patients completed the second assessment after 7 days. In general, all the items in Part A and Part B showed a moderate to a strong correlation, demonstrating further confirmation of the reliability of HM-PRO.