A novel electronic patient-reported outcome delivery system to implement health-related quality of life measures in routine clinical care: An analysis of 5 years of experience

Objective: To develop a simple and secure technological solution to incorporate electronic patient-reported outcomes (ePROs) into routine clinical care.

Methods: A novel ePRO questionnaire delivery system was developed by Software for Specialists (S4S) in partnership with OPAL Rheumatology Australia. Validated questionnaires were sent from the electronic medical record (EMR) (Audit4) of patients with rheumatoid arthritis (RA), psoriatic arthritis (PsA), ankylosing spondylitis (AS), lupus or giant cell arteritis (GCA) and delivered to the patient's email address or completed in the clinic waiting room using a smart device (in-practice). Completed questionnaires were encrypted and returned to the patient's Audit4. Deidentified clinical data was extracted and aggregated across all sites. Data collected between April 2016-Dec 2020 were analysed descriptively.

Results: Between April 2016 to Dec 2020, 221,352 Functional Assessment of Chronic Illness Therapy Fatigue (FACIT-F), Patient Health Questionnaire-2 (PHQ-2) and/or HealthCare Resource Utilization (HCRU) questionnaires were sent from 39 of 42 contributing clinics (93%). 85% of questionnaires were delivered via email and 15% in-practice. Overall, 85% of patients completed at least one questionnaire, and of all questionnaires sent, 73% were completed. Females were more likely to engage with the questionnaires than males (87% vs. 81%), and older patients were slightly more likely to complete all questionnaires delivered.

Conclusions: The novel Audit4 ePRO delivery system is an effective tool for incorporating PROs into routine clinical care. The data generated provides a unique opportunity to understand the full burden of disease for patients in the real-world setting and the impact of interventions.

Introduction

Reducing clinical disease activity to very low levels or remission for inflammatory arthropathies such as rheumatoid arthritis (RA), psoriatic arthritis (PsA) and ankylosing spondylitis (AS) is now possible with the availability of highly effective biologic and targeted synthetic disease modifying anti-rheumatic drugs (b/tsDMARDs) (1–3). Despite clinical remission being an achievable treatment target, subjective symptoms experienced by the patient which can be difficult to detect and measure objectively, can remain, including pain, fatigue, sleep disruption, depression, anxiety, social role participation and employment limitations (4). Patient reported outcomes (PROs) allow the patient to document how the disease impacts their quality of life in a standardised manner without being filtered through the lens of the clinician, providing additional context to measures of disease activity used by clinicians to make treatment decisions.

Registry studies and clinical trials are increasingly incorporating PROs to measure the full burden of disease and better understand the effectiveness and value of medicines (5, 6). The American College of Rheumatology (ACR), European League Against Rheumatism (EULAR) and Outcome Measures in Rheumatology (OMERACT) all recommend and endorse sets of outcome measures which include PROs for rheumatic diseases (7–9). When incorporated into routine care, PROs can help the clinician to better understand the patient experience, enhance communication and encourage shared decision making, which can empower the patient in the management of their own condition (10–12). However, the realities of time pressures during routine clinical care can significantly hamper recording of PROs in a regular or systematic fashion. Paper-based surveys can be burdensome for administrative staff, and despite tremendous innovation in digital health technologies and an abundance of applications harvesting consumer health data, there remains a lack of standardised digital methods for integrating PROs into the clinical workflow and records which does not rely on third parties to shuttle sensitive information between patient and clinician (13).

Further compounding these challenges, 80% of rheumatologists in Australia are practising in small independent community clinics without an overarching administrative structure so there is limited capacity for clinicians to develop and implement innovative clinical tools to enhance care at a scale that is economical. In 2009, the Optimising Patient outcomes in Australian rheumatology (OPAL)-Quality Use of Medicines Initiative (QUMI) was established to collate clinical data from rheumatologists distributed around Australia to allow large scale outcomes research to be conducted within the unique Australian rheumatology setting. A bespoke electronic medical record (EMR, Audit4) was customised for rheumatologists contributing to OPAL in partnership with software company Software4Specialists Pty Ltd (S4S) to capture comprehensive clinical data at the point-of-care which could be regularly deidentified, extracted and aggregated across all sites to create the OPAL Dataset. This EMR is now used by 111 rheumatologists (approximately 1 in 3) around Australia, predominately in private clinics (14).

In 2015, OPAL and S4S developed technology to seamlessly integrate electronic PROs (ePRO) into the routine consultation by delivering questionnaires to the patient through the EMR, and completed questionnaires are returned securely to the patient's EMR for review at the next consultation. This paper briefly describes real-world patient engagement with this technology by assessing the demographics and completion rates of Functional assessment of Chronic illness Therapy Fatigue (FACIT-F), Patient Health Questionnaire-2 (PHQ-2), and Health Care Resource Utilization (HCRU) questionnaires delivered using this technology as part of routine clinical care.

Methods

Data source

A novel ePRO questionnaire delivery system was developed by Software4Specialists Pty Ltd (S4S) in partnership with OPAL Rheumatology. Validated PRO questionnaires were sent from the patient's EMR (Audit4, Software4Specialists Pty Ltd, Sydney, Australia) and delivered to the patient's email address at time intervals specified by the rheumatologist (defaults to quarterly) or completed in the clinic waiting room prior to the consultation using a tablet or the patient's smart phone (in-practice). Both options of delivery (email or in-practice) were available, and the rheumatologists could choose the delivery method most suitable for them and their patient. Completed questionnaires were encrypted and returned directly to the patient's Audit4 EMR held on the clinician's server for review at the next clinical consultation (Figure 1). The link to the PRO questionnaire expired within 28 days if the questionnaire was not completed, and the questionnaires were automatically cancelled if two consecutive links expired. This technology was made available to up to 111 rheumatologists located in 42 clinics in 6 states/territories in Australia (New South Wales, Queensland, Victoria, Western Australia, Tasmania and Australian Capitol Territory), and the use of this technology to furnish the clinical consultation was voluntary for clinicians and patients. Deidentified clinical data was extracted from the servers of participating rheumatologists and aggregated across all sites.

FIGURE 1

Figure 1. Electronic patient reported outcome delivery system workflow in Audit4.

Patient population

Sending of ePRO questionnaires was at the discretion of the treating rheumatologist in consultation with the patient and completion of the questionnaire by the patient was voluntary. All adult patients with a physician diagnosis of RA, AS, PsA lupus or GCA were included in the analysis if they were sent at least one FACIT-F, HCRU or PHQ-2 questionnaire between April 2016–December 2020. Patient demographics (age, gender) and diagnosis were analysed descriptively against the status (completed, expired, cancelled) of each questionnaire activated.

Patient-reported outcomes

This study focused on describing the demographics and completion rates of three questionnaires, the FACIT-F, PHQ-2 and the HCRU. The FACIT-fatigue is a validated, commonly used, 13-item measure of fatigue. Questions are scored on a 0–4 response scale, ranging from “Not at all” to “Very much so”. The over-all fatigue score can range between 0 and 52 with higher scores representing less fatigue (15, 16). The questions in the FACIT-F questionnaire can be viewed at https://www.facit.org/measures/FACIT-Fatigue.

The PHQ-2 is a brief, two-question questionnaire adapted from the PHQ-9. It is a validated measure, intended to be used as an initial screening for major depression. The PHQ-2 asks the patient about the frequency of depressed mood and anhedonia over the past two weeks and identifies individuals who require additional evaluation. It does not determine whether a patient meets the criteria for a depressive order (17–19). The questions included in the PHQ-2 can be seen in Supplementary Figure S1.

The HCRU questionnaire asked about medical events and treatments experienced by the patient in the 3 months prior. The questions focused on visits to doctors or nurses, emergency department visits, hospitalisations, outpatient procedures, visits to allied health practitioners and alternative health practitioners (questionnaire can be viewed in Supplementary Figure S2).

The activities of OPAL Rheumatology Ltd have received overarching ethics approval from the University of New South Wales (UNSW) Human Research Ethics Committee (HREC), based on a patient opt-out arrangement (HC17799). All analyses were performed using R version 4.0.2.

Results

Between April 2016-Dec 2020, 221,352 questionnaires (FACIT-F, PHQ-2 and/or HCRU) were sent from 39 of 42 contributing clinics (93%) to patients with a physician diagnosis of RA, AS, PsA, lupus or giant cell arteritis (GCA) (Table 1). The gender and age categories of patients receiving PROs were broadly representative of the OPAL cohort with females and patients 61–70 years old receiving the highest percentage of the total PROs sent. The percentage of PROs sent to patients with RA, AS and PsA was proportionally higher than the number of patients with these conditions in the OPAL dataset (Table 1).

TABLE 1

Table 1. Demographic and clinical features of patients in the total OPAL cohort and patients in each PRO dataset.

The majority (85%) of FACIT-F, HCRU and PHQ-2 questionnaires were delivered to the patient via email and 15% were completed in-practice. Overall, 85% of patients completed at least one FACIT-F, PHQ-2 or HCRU questionnaire (Table 2). 73% of all FACIT-F, PHQ-2 or HCRU questionnaires sent were completed. Although high completion rates were seen for both methods of delivery, worksheets done in practice had significantly higher completion rates than those delivered via email (Table 2). Importantly, these high completion rates were sustained over time with patients completing multiple PRO questionnaires. The median [inter-quartile range, (IQR)] number of questionnaires completed per patient was 3 (1,7) and the median (IQR) time since the first questionnaire was completed was 13 months (5,26) (Table 2).

TABLE 2

Table 2. Features of worksheets in each PRO dataset.

The longitudinal completion pattern for patients in each PRO dataset is shown in Table 3. Males were slightly more likely to not complete any of their FACIT-F, PHQ-2 or HCRU questionnaires (<20%) compared to females (<14%), while older individuals (over 80 years of age) were slightly more likely to complete all questionnaires (≥50%) compared to those 70 years and under.

TABLE 3

Table 3. Longitudinal completion pattern for patients in each PRO dataset.

When the completion method for patients in each PRO dataset was assessed longitudinally, most age groups (80 years and under) were found to be more likely to complete all FACIT-F, PHQ-2 or HCRU questionnaires that were sent by email, although individuals 80 years and older, were more likely to complete all in practice. No meaningful differences in the completion methods were seen between males and females (Table 4).

TABLE 4

Table 4. Longitudinal completion method for patients in each PRO dataset.

Discussion

The data presented here reflects the real-world use of ePROs in routine clinical practice. The results demonstrate high engagement and completion rates of ePROs by patients with common rheumatological diagnoses across all age categories. Importantly, this high completion rate was also sustained over time and many patients had completed multiple ePRO questionnaires. This is important as it allows the treating rheumatologist to track their patient's PRO measures longitudinally and monitor improvement. The high engagement rates reported here are not surprising considering the positive effect PROs reportedly have on the patient-clinician relationship. In a qualitative study investigating perspectives of patients with RA on PRO data collection, patients expressed their interest in PRO measures and desire for greater involvement in the management of their disease (20). In addition, in a recent study describing the impact of integrating PROs into routine clinic, patients reported that PROs improved communication and shared decision making with their rheumatologist and made the visit more patient-centred. This resulted in high satisfaction and confidence in their treatment (12).

The development of an ePRO system that can be emailed to the patient for them to complete at home in their own time is a major advantage of the Audit4 software. A study that investigated patients preference for PRO collection found that patients prefer entering ePRO data at home (21). This remote access also enabled continued collection of ePROs during the coronavirus (COVID-19) pandemic.

The rise of technology to facilitate healthcare holds great potential to improve patient outcomes in an efficient and patient-centered manner. Despite the shift towards digital health, ePROs have not been widely implemented in routine clinical care. In a recent study conducted in Germany, the main reasons cited by rheumatologists for not implementing ePROs was the absence of suitable software, and the difficulties and high costs associated with implementing innovative software solutions (22). As such very few healthcare systems have built the analytic capacity to fully leverage the potential of PROs. The Audit4 EMR ePRO system has overcome these hurdles and to the best of our knowledge is the first of its kind to allow seamless integration of PROs into the daily workflow of a large network of rheumatologists based nationwide across Australia. This solution not only supports real-time decisions made by clinicians and patients but also incorporates a successful mechanism for collecting quantitative data that opens the potential for these important data to contribute to EMR-based research, facilitating representation of the patient perspective.

Limitations

The decision to utilise the ePRO technology, and which patients they were sent to, was at the discretion of the rheumatologist resulting in possible selection bias which was not addressed in the current study. In addition, self -reported questionnaires are at risk of misinterpretation and bias due to inaccurate recall and social desirability which may impact the validity of results (23). For this reason, it is important to use validated PRO measures to better ensure the questionnaires are interpretable, reliable and measure what they claim to measure.

Future work

Due to the successful integration and real-world utilisation of the Audit4 ePRO system, important patient reported quality of life data is being captured within the OPAL dataset and available for future research questions. A follow-on study would be to describe the levels of fatigue, mood disturbance and healthcare resource utilisation in our cohort of RA, AS, PsA GCA and lupus patients. The relationship between PRO measures and the traditional clinical disease activity measures as well as investigating the impact ePROs have on patient outcomes are also questions needing further investigation.

Data availability statement

The data that support this study cannot be publicly shared and may be shared upon reasonable request to the corresponding author if appropriate. Requests to access the datasets should be directed to info@opalrheumatology.com.au.

Ethics statement

The activities of OPAL Rheumatology Ltd have received overarching ethics approval from the University of New South Wales Human Research Ethics Committee (HC17799), based on a patient opt-out arrangement. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

CD: conception of the work, design of the work, analysis of the data, interpretation of the data, and drafting of the work. COS and TS: conception of the work, design of the work, interpretation of the data, drafting of the work. KT, GL, HG, SC, PY and DM: conception of the work, interpretation of the data, acquisition of the data and critical revision of the work. TF, DH, DS: acquisition of the data and critical revision of the work. All authors contributed to the article and approved the submitted version.

Acknowledgments

The authors acknowledge the members of OPAL Rheumatology Ltd and their patients for providing clinical data for this study, and Software4Specialists Pty Ltd for providing the Audit4 platform.

Conflict of interest

Author DS, DH and TF are employees of Software4Specialists. Authors TS, COS and CD are employees of OPAL Rheumatology. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2022.1074931/full#supplementary-material.

References

1. Soriano ER. Defining remission in psoriatic arthritis: are we getting closer? J Rheumatol. (2015) 42(6):907–8. doi: 10.3899/jrheum.150331

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Littlejohn G, Roberts L, Bird P, De Jager J, Griffiths H, Nicholls D, et al. Patients with rheumatoid arthritis in the Australian OPAL cohort show significant improvement in disease activity over 5 years: a multicenter observational study. J Rheumatol. (2015) 42(9):1603–9. doi: 10.3899/jrheum.141575

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Zochling J, Braun J. Remission in ankylosing spondylitis. Clin Exp Rheumatol. (2006) 24(6 Suppl 43):S 88–92.PMID: 17083769

Google Scholar

4. Taylor PC. Adopting PROs in virtual and outpatient management of RA. Nat Rev Rheumatol. (2020) 16(9):477–8. doi: 10.1038/s41584-020-0449-6

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, et al. ISOQOL Recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. (2013) 22(8):1889–905. doi: 10.1007/s11136-012-0344-y

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Honomichl R, Katzan I, Thompson N, Abelson A, Deal C, Rose S, et al. The influence of collecting patient-reported outcome measures on visit satisfaction in rheumatology clinics. Rheumatol Adv Pract. (2020) 4(2):1–10. doi: 10.1093/rap/rkaa046

CrossRef Full Text | Google Scholar

7. Barber CEH, Zell J, Yazdany J, Davis AM, Cappelli L, Ehrlich-Jones L, et al. 2019 American college of rheumatology recommended patient-reported functional Status assessment measures in rheumatoid arthritis. Arthritis Care Res. (2019) 71(12):1531–9. doi: 10.1002/acr.24040

CrossRef Full Text | Google Scholar

8. Kirwan JR, Hewlett SE, Heiberg T, Hughes RA, Carr M, Hehir M, et al. Incorporating the patient perspective into outcome assessment in rheumatoid arthritis–progress at OMERACT 7. J Rheumatol. (2005) 32(11):2250–6. PMID: 16265712

PubMed Abstract | Google Scholar

9. Castrejón I, Gossec L, Carmona L. The EULAR outcome measures library: an evolutional database of validated patient-reported instruments. Ann Rheum Dis. (2015) 74(2):475–6. doi: 10.1136/annrheumdis-2014-206485

CrossRef Full Text | Google Scholar

10. Santana MJ, Feeny D. Framework to assess the effects of using patient-reported outcome measures in chronic care management. Qual Life Res. (2014) 23(5):1505–13. doi: 10.1007/s11136-013-0596-1

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Noonan VK, Lyddiatt A, Ware P, Jaglal SB, Riopelle RJ, Bingham CO, et al. Montreal Accord on patient-reported outcomes (PROs) use series—paper 3: patient-reported outcomes can facilitate shared decision-making and guide self-management. J Clin Epidemiol. (2017) 89:125–35. doi: 10.1016/j.jclinepi.2017.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bartlett SJ, De Leon E, Orbai A-M, Haque UJ, Manno RL, Ruffing V, et al. Patient-reported outcomes in RA care improve patient communication, decision-making, satisfaction and confidence: qualitative results. Rheumatology. (2020) 59(7):1662–70. doi: 10.1093/rheumatology/kez506

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Basch E, Snyder C. Overcoming barriers to integrating patient-reported outcomes in clinical practice and electronic health records. Ann Oncol Off J Eur Soc Med Oncol. (2017) 28(10):2332–3. doi: 10.1093/annonc/mdx506

CrossRef Full Text | Google Scholar

14. Littlejohn G, Tymms K, Smith T, Griffiths H. Using big data from real-world Australian rheumatology encounters to enhance clinical care and research. Clin Exp Rheumatol. (2020) 38(5):874–80. PMID: 31820719

PubMed Abstract | Google Scholar

15. Yellen SB, Cella DF, Webster K, Blendowski C, Kaplan E. Measuring fatigue and other anemia-related symptoms with the functional assessment of cancer therapy (FACT) measurement system. J Pain Symptom Manage. (1997) 13(2):63–74. doi: 10.1016/S0885-3924(96)00274-6

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Cella D, Yount S, Sorensen M, Chartash E, Sengupta N, Grober J. Validation of the functional assessment of chronic illness therapy fatigue scale relative to other instrumentation in patients with rheumatoid arthritis. J Rheumatol. (2005) 32(5):811–9. PMID: 15868614

PubMed Abstract | Google Scholar

17. Kroenke K, Spitzer RL, Williams JBW. The patient health questionnaire-2: validity of a two-item depression screener. Med Care. (2003) 41(11):1284–92. doi: 10.1097/01.MLR.0000093487.78664.3C

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gilbody S, Richards D, Brealey S, Hewitt C. Screening for depression in medical settings with the patient health questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern Med. (2007) 22(11):1596–602. doi: 10.1007/s11606-007-0333-y

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Carey M, Boyes A, Noble N, Waller A, Inder K. Validation of the PHQ-2 against the PHQ-9 for detecting depression in a large sample of Australian general practice patients. Aust J Prim Health. (2015) 22:262–6. doi: 10.1071/PY14149

CrossRef Full Text | Google Scholar

20. Navarro-Millán I, Zinski A, Shurbaji S, Johnson B, Fraenkel L, Willig J, et al. Perspectives of rheumatoid arthritis patients on electronic communication and patient-reported outcome data collection: a qualitative study. Arthritis Care Res. (2019) 71(1):80–7. doi: 10.1002/acr.23580

CrossRef Full Text | Google Scholar

21. Secher AE, Glintborg B, Gudbergsen H, Krogh NS, Sørensen IJ, Jensen DV, et al. Comparing patient-reported outcomes entered at home versus at hospital, and testing touch screens for initial recruitment to scientific trials in arthritis patients. Scand J Rheumatol. (2019) 48(3):178–84. doi: 10.1080/03009742.2018.1522666

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Krusche M, Klemm P, Grahammer M, Mucke J, Vossen D, Kleyer A, et al. Acceptance, usage, and barriers of electronic patient-reported outcomes among German rheumatologists: survey study. JMIR mHealth uHealth. (2020) 8(7):e18117. doi: 10.2196/18117

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Rosenman R, Tennekoon V, Hill LG. Measuring bias in self-reported data. Int J Behav Healthc Res. (2011) 2(4):320–32. doi: 10.1504/IJBHR.2011.043414

PubMed Abstract | CrossRef Full Text | Google Scholar