A decade of outcomes: The evolution of an australasian outcomes collaboration for chronic pain services

1. Introduction

The electronic Persistent Pain Outcomes Collaboration (ePPOC) was formed in 2013 to ensure benchmarking and outcomes were part of a growing agenda to improve the quality of life for people living with persistent pain in Australia (AU) and New Zealand (NZ) (1). The Faculty of Pain Medicine of the Australian & New Zealand College of Anaesthetists identified the need for a national outcomes data set to progress benchmarking and quality improvement. On development of an Australian National Pain Strategy (NPS), there were clear objectives on quality improvement through outcomes measurement and benchmarking and acknowledgement that this needed to occur in both the adult and paediatric populations. The achievement of this objective was realised in 2012, with the NSW Agency for Clinical Innovation Pain Management Network (part of the New South Wales Ministry of Health) funding the simultaneous implementation of both an adult dataset (ePPOC) and pediatric dataset (PaedePPOC) (2).

In the decade since the establishment of ePPOC and PaedePPOC, there have been ongoing improvements made in respect of data items captured and the processes supporting the submission of data and reporting of outcomes. These systemic improvements, combined with the active engagement of member services, allows for transparent discussions on individual service outcomes and the identification of quality improvement initiatives to bolster outcomes further. This paper will outline the core components of ePPOC, the use of patient-reported outcomes data for clinical and quality improvement, and the research outputs achieved.

2. Data collection approach

2.1. Assessment and feedback

ePPOC collects a standard set of information by specialist pain management services from across AU and NZ. This information is used to guide treatment for individual patients, measure outcomes following treatment and provide a benchmarking system across nine domains. The benchmarking system is designed to provide comparative outcomes and service process data to each pain service, identify best practice protocols and clinical variation, and drive quality improvement through setting aspirational targets for patient and service outcomes. ePPOC's ethics approval allows for the collection of patient de-identified outcome and specialist pain service data from participating services, for the purposes of reporting, quality assurance, and benchmarking. Specialist pain management services also seek advice from their relevant human research ethics committee where relevant (1).

Participating services receive twice-yearly standardized reports and stand-alone executive summaries and dashboards, summarizing their achievement against outcome measures with established benchmarks. The reports also allows services to anonymously compare their results against other participating services (see Figure 1 for an example of an executive summary). On receipt of each report, pain services can also access support from ePPOC's Improvement Facilitators (IFs). As will be discussed below, IFs work with services to analyze their results, identify areas of high performance, areas in need of improvement and relevant quality projects and strategies.

FIGURE 1

Figure 1. Example of one part of standardized report.

2.2. The ePPOC and PaedePPOC data collections

Central to ePPOC's cycle of assessment and feedback is the ePPOC data collection, which was designed to capture clinically meaningful information, with an emphasis on patient reported outcome measures (PROMs) in both the adult and pediatric programs, with additional carer-proxy outcomes measures in the pediatric program. The PROMs and carer proxy measures provide an assessment of the health status of a patient at a prescribed time during their treatment, with paired information pre- and post- treatment used to measure and benchmark patient outcomes.

Patients also provide information relating to the cause of their pain, how long it has been a problem, and where and how widespread it is (via a pain body map). Medication use and health service utilization in the three months prior to completing the referral questionnaire is also captured, as is information about the impact of pain on work productivity and employment. The pediatric data set includes carer reported data on the impact of the child's pain on school and carer work productivity and employment. Patient (and carer) reported information is supplemented with clinician reporting of the type and duration of treatments the patients received via treatment pathways and service events.

Table 1 summarizes the characteristics of adult patients (age greater than 18 years). The patient's work and productivity status showed that the majority of patients were not working due to pain at referral. The most commonly reported pain site was the back. Mental health condition was reported as the main comorbidity for chronic pain patients. Other commonly reported comorbidities included arthritis, “heart problems”, “digestive problems”, “respiratory problems” and high blood pressure.

TABLE 1

Table 1. Characteristics of adult patients included in the analysis (2014–2021).

Information collected in both the adult and pediatric data collections have been reported previously (1, 2), with a summary of the patient and carer-reported outcome measures used in both the adult and pediatric data collections provided in Table 2. Each measure is captured at referral, episode end and 3–6 months after completion of treatment where feasible.

TABLE 2

Table 2. Patient/carer reported measures for the adult and paediatric data collections.

An example of clinical outcomes across the full patient cohort is provided in Figure 2, where the level of improvement for pain interference, depression, anxiety and stress from referral to episode end is illustrated for the total adult cohort at end of 2021 reporting period.

FIGURE 2

Figure 2. Clinical outcomes by domain, total patient cohort 2021.

2.3. PROMs included in the adult data collection

The PROMs collected in the adult data collection align with the core domains recommended by the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) (3) and have been reported in detail previously (1). These domains and measures are:

Pain, measured by the pain severity elements of the Brief Pain Inventory (BPI) (4).

Physical functioning, measured by the BPI Interference items (4).

Emotional functioning, as measured by the Depression, Anxiety and Stress Scale (DASS) (5) and the Pain Catastrophizing Scale (PCS) (6).

The Pain Self-Efficacy Questionnaire (PSEQ) (7) is also included, to provide an assessment of how confident patients feel in performing activities despite their pain.

These outcomes are complimented by the ePPOC Patient Impression of Change (ePIC) assessment tool, which provides patients who have received treatment an opportunity to reflect on their perception of change, both overall and in respect to physical abilities.

2.4. PROMs and carer proxy measures: pediatric data collection

The PROMs and carer proxy measures included in the pediatric data collection were informed by the domains outlined in the PedIMMPACT Consensus statement (8). However, the pediatric data collection is more complicated than for adults, with patient and carer-reported tools catering to different age groups and used in combination in some cases. The pediatric tools cover the domains of pain severity, pain related anxiety, functional disability, and health related quality of life (physical, social, emotional and school functioning). The development of the current version of the data collection (version 2), including the rationales for tool selection has been previously documented (2).

2.5. Timing of assessment

The adult and pediatric data collections follow the same assessment protocol, with the patient (and carer) questionnaires administered at referral to the pain service and then at multiple time points once treatment has commenced, including the start and end of treatment pathways and the end of the overall episode of care. A follow-up questionnaire is administered 3–6 months after the episode has ended, to ascertain whether improvements from treatment have been maintained. Clinicians can also administer “clinical review” questionnaires at any time during treatment, if it is felt the additional information would be useful in planning care, or if a patient experiences a significant change in their condition.

The pediatric data collection also includes an assessment of the impact on the parent/carer of their child's pain. The timing of parent/carer-impact data collection varies slightly, with information collected at referral, 12-months following pathway start, pathway end, at the end of treatment and 3–6 months after the episode has ended.

2.6. Measuring patient outcomes

Patient outcomes are measured using a pre-/post- methodology, with the benchmarked patient outcome measures focused on the changes between referral to the pain service (pre-treatment) and at end of the episode of care (post-treatment). A patient is deemed to have achieved a positive outcome on a measure if they report a clinically significant improvement following treatment, or have reduced their average daily opioid intake by 50% or more. What constitutes a clinically significant improvement has been defined separately for each of the measures. For example, in relation to pain interference, a change of one point or more over the average of the seven interference items points to clinically significant change. These definitions are provided in a clinical reference manual provided to services and is also available publicly online (9).

ePPOC PROMS provides useful information to clinicians to inform care for a patient. The clinical utility of the data collection anecdotally increases both service staff and patient engagement, in turn leading to higher quality data provision by services as the value of the data is illustrated.

2.7. Epicentre computer application

As reported previously (1) ePPOC developed a software application called epiCentre (the ePPOC Patient Information Centre) which comprises a desktop application paired with a cloud-based REDCap server, used to deliver online questionnaires to patients/carers. In addition to supporting routine patient outcome reporting, epiCentre provides clinicians with the ability to generate progress reports for individual patients and to track improvement over time, with examples of all questionnaires provided by ePPOC online for researchers interested in examples (9). These individual-level reports are designed to enhance patient engagement with the pain service and support discussions between patients and their treatment providers about the care they are receiving and the progress they are making. Over the ten years of ePPOC's existence, the epiCentre application has been iteratively improved as feedback is received from services, further improving both quality of data and time for submission of data.

3. Australian and New Zealand pain service profile

Since establishment in 2013, there has been significant growth in the number of participating services. Pain management services who hold membership with ePPOC are dispersed across most states and territories of AU and throughout NZ. Figure 3 shows the accumulation of data-submitting services since ePPOC establishment in 2013. There are a small number of non-data submitting member services, which takes the total service membership above 100. The number of adult services increased from 21 in 2014 to 87 in 2022, while pediatric services increased from 3 in 2014 to 12 in 2021.

FIGURE 3

Figure 3. Number of data submitting services since ePPOC establishment.

ePPOC's dataset now includes 109625 patient-completed questionnaires collected by Australian adult pain services at time of referral, with a further 32,923 adult questionnaires collected in NZ. 2,920 questionnaires have been collected from AU pediatric patients and 65 from NZ (Table 1). Geographically, the majority of adult services are located in New South Wales (NSW) and Victoria, AU's two most populous states (10). Accordingly, 44,317 patients from NSW and 34,380 patients from Victoria services provided data at referral during the years 2014–2022. Most pediatric pain management services were located in NSW and Queensland, with only one NZ pediatric service. Table 3 outlines the number of services contributing patient data, by year.

TABLE 3

Table 3. Services location and received referral questionnaires since ePPOC establishment (January 2014–June 2022).

4. ePPOC governance and risk management

The involvement of key stakeholders in chronic pain management in the Australian and New Zealand context has been integral to the governance of ePPOC since establishment (1, 2). The format and structure of what is now known as the ePPOC Clinical Management Advisory Committee (CMAC) has evolved over the decade of ePPOC's operation. This evolution occurred as ePPOC's operations matured and grew—the initial national reference group overseeing the establishment of ePPOC divided into management advisory and clinical advisory groups, before being combined in 2018 into the CMAC in operation today (1). These changes reflected the need to adjust the scope of governance as ePPOC grew in size and robust processes were put in place for governance-related issues such as access to ePPOC data for research.

CMAC meets up to four times per year and incorporates key stakeholders from peak bodies, consumers and funders across Australia and New Zealand. CMAC provides approvals in relation to proposed research utilizing ePPOC data, provides input on strategic direction and is a valuable conduit more broadly in relation to chronic pain services and their own evolution. In respect to research, researchers provide a proposal requesting data access, which is initially reviewed by ePPOC staff and then submitted to CMAC for approval.

There remain significant risk factors for ePPOC, particularly in relation to recurrent funding. Current funders come from a range of state and government entities in Australia and New Zealand, with all funding on a fixed term basis. Each funder receives their own report with aggregated information on the performance of their funded service cohort—the degree of identifiability of services to funders is dependent on specific funding arrangements and policies of the funder. There are clear CMAC terms of reference outlining the need for disclosure of potential conflicts of interest and documentation of these in meeting minutes.

The short-term nature of funding arrangements creates challenges in respect to long-term planning and investment in resources for more substantial development of the ePPOC program. The current staffing numbers 6.4 full-time equivalent staff only—this creates its own challenges with resources to evolve the ePPOC program whilst maintaining a burgeoning membership and providing extensive reporting of data to each member service, their funders and to external researchers. Continuity of service provisions across the past ten years has been feasible due to growth in services participating and fortuitous timing of funding contracts.

5. Approach to quality improvement

5.1. Improvement facilitator role

As volume and diversity of chronic pain service membership grew, the need was identified for clinically trained staff who could work directly with member services to embed ePPOC protocols into existing clinical operations, provide ongoing support to ensure consistent high-quality practices, and facilitate use of the program to achieve continuous improvement. To address this, the role of Improvement Facilitator (IF) was created, and the first IF commenced in 2015.

The IF role requires health professionals with experience working in interdisciplinary, complex health settings. This experience affords knowledge of disciplines, interdisciplinary care, pain service operations, general health practices. In addition to an understanding of health care, IFs are also required to possess a firm understanding of continuous improvement principles and have intermediate competencies in data analysis and interpretation. Key Australian frameworks that guide ePPOC's approach to quality improvement are the National Safety and Quality Health Service (NSQHS) Standards (11) and the The National Strategic Action Plan for Pain Management (12). The IFs serve three key purposes: supporting services to embed ePPOC into their care delivery, providing ongoing support to services once they have embedded the approach, and finally acting as a conduit between the pain service and the broader ePPOC team. Each of these functions are described further below.

5.2. Embedding ePPOC into care delivery

When a new pain service joins ePPOC, the IF is responsible for introducing the principles and practices of ePPOC and aiding the integration of the epiCentre application within the pain service. The guidance required to effectively embed a new system into a dynamic clinic environment can be significant and includes training (face-to-face or virtual) on the key milestones for data collection, effective use of the epiCentre application, troubleshooting of any issues and liaison with the ePPOC team for resolution of those issues if required.

The initial training provided to a pain service is a useful quality improvement initiative in itself, providing the service an opportunity to review the patient journey and identify where there might be delays or gaps in progressing a patient from referral to commencement of treatment. Building rapport with all staff via meetings and other communication ensures a team understanding and engagement with ePPOC processes. Participation in ePPOC does involve extra administrative work for pain services, therefore reinforcing the benefits of robust data submission is an additional focus. A “whole team approach” ensures that data is collected at standard and agreed time points to provide good evidence on which services can progress quality improvement.

5.3. Ongoing support to services

IFs work with each service throughout their membership to ensure ongoing effective data collection and identification of quality improvement opportunities. This includes review of workflows and processes to ensure the collection of patient-reported outcome measures occurs at the correct time, and activities to increase user capability in respect of the epiCentre application.

IFs work directly with service staff to review their submitted data and identify records for revision/correction. Identification of data quality issues has seen the quality of service submitted data increase over time. Identification of data quality issues provides an opportunity for services to implement a quality approach to the data submission process. Over time, the quality of service submitted data increases. Obtaining sufficient outcome data is also important for the reliability of outcomes reporting and a necessary component for services to achieve a benchmark result. Benchmark results enable services to compare themselves to the aggregated data for all services and provide context for improvement of clinical practice. These outcomes provide an excellent starting point to identify clinical and service domains where results may be less than expected, prompting either review of workflows and processes or requests to ePPOC for quality improvement support and advice.

Report review meetings are offered to all participating pain services after they receive their six-monthly report. To assist, ePPOC provides a report review template to elicit information from the staff within each service. This prompts staff to reflect on their strengths and areas for improvement, concluding with an identified quality improvement goal. Offering routine service-specific support has been particularly important since the COVID pandemic, which forced changes to the way pain services delivered services, including a significant increase in online delivery of healthcare, leading to changes to group program content and service timeframes. The ongoing collection of data twice per year has allowed services to compare the outcomes of interventions delivered remotely with those delivered face to face.

ePPOC have found that services who regularly schedule a report review meeting are more likely to better understand their report data and use this to drive quality improvement. IFs also provide a valuable link between pain services for the purposes of sharing knowledge within the wider pain sector. Knowledge sharing and networking is also facilitated by ePPOC through the annual Australasian adult and pediatric benchmarking workshops, which are open to all member services. These workshops provide a further opportunity for services to gain insight into the wider sector and the outcomes being generated across Australia and New Zealand, promoting a community of practice.

5.4. Quality improvement as the “Engine Room” of outcomes collaboration

As discussed above, the IF role serves an important liaison role, facilitating bidirectional communication between the participating pain services and the wider ePPOC team, which is comprised of statisticians, a director, and administrative and information technology support staff. ePPOC is first and foremost a data collection designed to be collected and used at the point of care. Pain services are well-positioned to advise on the utility of ePPOC in their daily practice. As the designated point of first contact for services, the IFs receive valuable feedback from services on their experience with the program and can communicate this to the broader ePPOC team.

The IF role is now an established and essential component to the success of the ePPOC program, playing an important role in education, support, maintaining continued engagement, and facilitating communication between the member services and the broader ePPOC team. This role continues to evolve in line with changing internal and external environments. Historically, much of the focus has been on upskilling member services to maximize capability. As more services become proficient in their use of ePPOC, the IF role will need to pivot and provide adjusted offerings to accommodate the different needs of these services, which are likely to involve a deeper understanding of their data set, and individualized quality improvement initiatives.

6. Research outputs

6.1. External and internal research

Over the decade since ePPOC's establishment, there has been increasing interest in use of the ePPOC datasets for research purposes. ePPOC's binational dataset has provided a unique opportunity for studies to look at results across a large cohort of services from two countries, with a wide range of treatment modalities. The ability to study outcomes across a large dataset provides opportunities for customization of care and improvement of quality (13). The list of published articles using the ePPOC dataset can be found on the ePPOC website (9), and research to date illustrates the diversity of topics the dataset can engender. This has included opioid cessation impacts on pain and function, outcomes inequities by geographical location and patterns of patient outcomes in specialist pain management units. As occurs with the work undertaken by IFs with participating pain services, there is a quality improvement component of research, with the external applicants providing valuable insights into ways the ePPOC dataset can be itself improved.

ePPOC also publishes research findings as an information series on the ePPOC website (13), which are subject to the same governance and approval processes as external research using the ePPOC dataset. The publication of the information series helps to disseminate ePPOC research findings to pain services, who in turn can utilize the findings to improve pain management outcomes.

6.2. Future research

As discussed above, research is a core function of ePPOC and requests for access to the ePPOC dataset continue to increase. Planning is underway for the third iteration of the ePPOC dataset to reflect changing pain service and research needs in both the adult and pediatric spheres. A focus of research for ePPOC staff in the short to medium term will include investigation of the efficacy of a shortened dataset for some clinical applications, the impact of telehealth on pain service outcomes and more detailed study on waiting times and outcomes. The ePPOC research strategy will continue to be informed by member services, with guidance from ePPOC's governance committee (CMAC).

7. Discussion

The first ten years of ePPOC's existence has seen its dataset used to undertake research and quality improvement initiatives that are improving understanding of chronic pain and effective pain management. This ongoing maturity of approach has a substantial impact, both in respect of human factors (improved understanding of chronic pain, the lived experience of pain amongst diverse populations and geographies) and in respect of system factors (increased knowledge of treatment modalities and enhancing service approaches to improve outcomes). This evolution is continuous, with a strategic review of ePPOC identifying the need for further improvements in information technology and customization of the dataset and reports for different service needs.

Since the report on the establishment of ePPOC (1), the number of participating services has doubled to more than 100, further strengthening the dataset and the potential impact of quality improvement initiatives.

8. Conclusion

After a decade of operation, ePPOC has matured to be a well-regarded outcomes collaboration/registry. Like the wider chronic pain sector, there is constant change and evolution. The provision of robust outcomes data framed within a quality improvement context, allows pain services to have reliable guidance on their impact and the potential for further impact as quality improves further. If sustainability of funding and continued close liaison with the chronic pain sector occurs, there remains the opportunity for the ePPOC dataset to be an ongoing source of both outcomes and registry information that can inform practice at the local, funder and governmental policy levels.

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: Access to the dataset are bound by the Australian Health Services Research Institute Data Access policy, requiring application to the corresponding author. Requests to access these datasets should be directed to David Holloway, eppoc-uow@uow.edu.au.

Author contributions

DH: First author who coordinated all aspects of compilation and review in addition to equal contribution of content. SA, MB, KC, MC, DS: equal contribution after first author, all provided content and provided revisions at request of first author. All authors contributed to the article and approved the submitted version.

Acknowledgments

We would like to thank the following entities for supporting their pain services to participate in the ePPOC program: New Zealand Accident Compensation Corporation, New South Wales Agency for Clinical Innovation, Queensland Health, South Australia Health, Transport Accident Commission Victoria, Victorian Agency for Health Information, WorkSafe Victoria, Western Australia Primary Health Alliance. Thanks to those pain services who fund their own membership to ePPOC. Thanks also to the Australian Pain Society, the Faculty of Pain Medicine and Painaustralia for their ongoing support. Finally, we'd also like to thank the clinical and administrative staff who collect, collate, and submit the data that allows ePPOC to exist.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Tardif H, Arnold C, Hayes C, Eagar K. Establishment of the australasian electronic persistent pain outcomes collaboration. Pain Med. (2017) 18(6):1007–18. doi: 10.1093/pm/pnw201

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Lord SM, Tardif HP, Kepreotes EA, Blanchard M, Eagar K. The paediatric electronic persistent pain outcomes collaboration (PaedePPOC): establishment of a binational system for benchmarking children’s persistent pain services. PAIN. (2019) 160:1572–85. doi: 10.1097/j.pain.0000000000001548

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Turk DC, Dworkin RH, Allen RR, Bellamy N, Brandenburg N, Carr DB, et al. Core outcome domains for chronic pain clinical trials: iMMPACT recommendations. PAIN. (2003) 106(3):337–45. doi: 10.1016/j.pain.2003.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Cleeland CS, Ryan KM. Pain assessment: global use of the brief pain inventory. Ann. Acad. Med. Singap. (1994) 23(2):129–38.8080219

PubMed Abstract | Google Scholar

5. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the beck depression and anxiety inventories. Behav Res Ther. (1995) 33(3):335–43. doi: 10.1016/0005-7967(94)00075-u

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Sullivan MJ, Bishop SR, Pivik J. The pain catastrophizing scale: development and validation. Psychol Assess. (1995) 7:524–32. doi: 10.1037/1040-3590.7.4.524

CrossRef Full Text | Google Scholar

7. Nicholas MK. The pain self-efficacy questionnaire: taking pain into account. Eur J Pain. (2007) 11(2):153–63. doi: 10.1016/j.ejpain.2005.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

8. McGrath PJ, Walco GA, Turk DC, Dworkin RH, Brown MT, Davidson K, et al. PedIMMPACT. Core outcome domains and measures for pediatric acute and chronic/recurrent pain clinical trials: PedIMMPACT recommendations. J Pain. (2008) 9(9):771–83. doi: 10.1016/j.jpain.2008.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Electronic Persistent Pain Outcomes Collaboration. Electronic persistent pain outcomes collaboration. Wollongong, NSW: Australian Health Services Research Institute (2023). (cited 2023 Jan 15). Available at: https://www.uow.edu.au/ahsri/eppoc/.

10. Australian Bureau of Statistics. National, state and territory population: Statistics about the population and components of change (births, deaths, migration) for Australia and its states and territories, https://www.abs.gov.au/statistics/people/population/national-state-and-territory-population/latest-release (Accessed January 5, 2023) (2022).

11. Australian Commission on Safety and Quality in Health Care. National quality and safety health service standards—second edition. Sydney, NSW: Australian Commission on Safety and Quality in Health Care (2021). (cited 2023 Jan 15). Available at: https://www.safetyandquality.gov.au/publications-and-resources/resource-library/national-safety-and-quality-health-service-standards-second-edition.

12. Australian Government Department of Health and Aged Care. The national strategic action plan for pain management. Canberra, ACT: Australian Government Department of Health and Aged Care (2021). (cited 2023 Jan 15). Available at: https://www.health.gov.au/resources/publications/the-national-strategic-action-plan-for-pain-management.

13. Zaslansky R, Meissner W, Stamer UM. Insights from studying a large cohort of patients. PAIN. (2022); Oct 17. doi: 10.1097/j.pain.0000000000002812. [Epub ahead of print]

CrossRef Full Text | Google Scholar