# CONTINUOUS QUALITY IMPROVEMENT—ADVANCING UNDERSTANDING OF DESIGN, APPLICATION, IMPACT AND EVALUATION OF CQI APPROACHES

EDITED BY: Ross Bailie, Sarah Larkins and Edward Broughton PUBLISHED IN: Frontiers in Public Health

#### *Frontiers Copyright Statement*

*© Copyright 2007-2017 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.*

*The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.*

*Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.*

*Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.*

*As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.*

*All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use.*

ISSN 1664-8714 ISBN 978-2-88945-377-1 DOI 10.3389/978-2-88945-377-1

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# **CONTINUOUS QUALITY IMPROVEMENT—ADVANCING UNDERSTANDING OF DESIGN, APPLICATION, IMPACT AND EVALUATION OF CQI APPROACHES**

Topic Editors: **Ross Bailie,** The University of Sydney, Australia **Sarah Larkins,** James Cook University, Australia **Edward Broughton,** University Research Co, United States

**Citation:** Bailie, R., Larkins, S., Broughton, E., eds. (2017). Continuous Quality Improvement— Advancing Understanding of Design, Application, Impact and Evaluation of CQI Approaches. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-377-1

# Table of Contents

*05 Editorial: Continuous Quality Improvement (CQI)—Advancing Understanding of Design, Application, Impact, and Evaluation of CQI Approaches*

Ross Bailie, Jodie Bailie, Sarah Larkins and Edward Broughton

#### **Variation and Priority Gaps in Care; Barriers and Enablers for Improvement**

*08 Determinants and Gaps in Preventive Care Delivery for Indigenous Australians: A Cross-sectional Analysis*

Christopher Bailie, Veronica Matthews, Jodie Bailie, Paul Burgess, Kerry Copley, Catherine Kennedy, Liz Moore, Sarah Larkins, Sandra Thompson and Ross Stewart Bailie

*19 Delivery of Eye and Vision Services in Aboriginal and Torres Strait Islander Primary Healthcare Centers*

Anthea M. Burnett, Anna Morse, Thomas Naduvilath, Andrea Boudville, Hugh R. Taylor and Ross Bailie

*27 Integrated Clinical Decision Support Systems Promote Absolute Cardiovascular Risk Assessment: An Important Primary Prevention Measure in Aboriginal and Torres Strait Islander Primary Health Care*

Veronica Matthews, Christopher P. Burgess, Christine Connors, Elizabeth Moore, David Peiris, David Scrimgeour, Sandra C. Thompson, Sarah Larkins and Ross Bailie

*36 System-Level Action Required for Wide-Scale Improvement in Quality of Primary Health Care: Synthesis of Feedback from an Interactive Process to Promote Dissemination and Use of Aggregated Quality of Care Data*

Jodie Bailie, Alison Laycock, Veronica Matthews and Ross Bailie

*48 Wide Variation in Absolute Cardiovascular Risk Assessment in Aboriginal and Torres Strait Islander People with Type 2 Diabetes*

Bhakti R. Vasant, Veronica Matthews, Christopher P. Burgess, Christine M. Connors and Ross S. Bailie

#### **Primary Healthcare Centre Systems**

*56 Exploring Systems That Support Good Clinical Care in Indigenous Primary Health-care Services: A Retrospective Analysis of Longitudinal Systems Assessment Tool Data from High-Improving Services*

Cindy Woods, Karen Carlisle, Sarah Larkins, Sandra Claire Thompson, Komla Tsey, Veronica Matthews and Ross Bailie

#### **Engaging at Multiple Levels of the System**

*73 Interactive Dissemination: Engaging Stakeholders in the Use of Aggregated Quality Improvement Data for System-Wide Change in Australian Indigenous Primary Health Care*

Alison Laycock, Jodie Bailie, Veronica Matthews and Ross Bailie

#### **Facilitation**

*81 Enabling Continuous Quality Improvement in Practice: The Role and Contribution of Facilitation*

Gillian Harvey and Elizabeth Lynch

#### **CQI in Health Professional Education**

*86 Improving Community Health Using an Outcome-Oriented CQI Approach to Community-Engaged Health Professions Education*

Amy Clithero, Simone Jacquelyn Ross, Lyn Middleton, Carole Reeve and Andre-Jacques Neusy

#### **CQI in Health Promotion and Beyond**

*92 Improving Health Promotion Using Quality Improvement Techniques in Australian Indigenous Primary Health Care*

Nikki Percival, Lynette O'Donoghue, Vivian Lin, Komla Tsey and Ross Stewart Bailie

# Editorial: Continuous Quality Improvement (CQI)—Advancing Understanding of Design, Application, Impact, and Evaluation of CQI Approaches

*Ross Bailie1 \*, Jodie Bailie1 , Sarah Larkins <sup>2</sup> and Edward Broughton3*

*<sup>1</sup> The University of Sydney, The University Centre for Rural Health, Lismore, NSW, Australia, 2 James Cook University, College of Medicine and Dentistry, Townsville, QLD, Australia, 3University Research Co., LLC, Chevy Chase, MD, United States*

Keywords: primary health care, health systems research, continuous quality improvement, Aboriginal and Torres Strait Islander health, building block

#### **Editorial on the Research Topic**

#### **Continuous Quality Improvement (CQI)—Advancing Understanding of Design, Application, Impact, and Evaluation of CQI Approaches**

#### *Edited and Reviewed by:*

*Kai Ruggeri, University of Cambridge, United Kingdom*

*\*Correspondence: Ross Bailie ross.bailie@sydney.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 20 October 2017 Accepted: 03 November 2017 Published: 23 November 2017*

#### *Citation:*

*Bailie R, Bailie J, Larkins S and Broughton E (2017) Editorial: Continuous Quality Improvement (CQI)—Advancing Understanding of Design, Application, Impact, and Evaluation of CQI Approaches. Front. Public Health 5:306. doi: 10.3389/fpubh.2017.00306*

Continuous quality improvement (CQI) approaches are increasingly used to bridge gaps between the evidence base for best practice, what actually happens in practice, and achievement of better population health outcomes. Among a range of quality improvement strategies, CQI is characterized by iterative use of processes to identify quality problems, develop solutions, and implement and evaluate changes. Application of CQI in health care is evolving and evidence of their success continues to emerge (1–3).

Through the Research Topic, "Continuous Quality Improvement (CQI)—Advancing Understanding of Design, Application, Impact, and Evaluation of CQI approaches," we aimed to aggregate knowledge of useful approaches to tailoring CQI approaches for different contexts, and for implementation, scale-up and evaluation of CQI interventions/programs. This Research Topic has attracted seven original research reports and three "perspectives" papers. Thirty-six authors have contributed from eighteen research organizations, universities, and policy and service delivery organizations. All original research articles and one perspective paper come from the Australian Audit and Best Practice for Chronic Disease (ABCD) National Research Partnership ("ABCD Partnership") in Indigenous primary healthcare settings (4–6). To some extent, this reflects the interests and connections of two of the Topic Editors, who were lead investigators on the ABCD Partnership. This Partnership has made a prominent contribution to original research on CQI in primary healthcare internationally, with over 50 papers published in the peer-reviewed literature over the past 10 years.

As most articles in this Research Topic arise from the ABCD Partnership, a brief overview of the program provides a useful backdrop. The program originated in 2002 in the Top End of the Northern Territory in Australia, and built on substantial prior research and evaluation of CQI methods in Indigenous primary healthcare. With substantial growth and enthusiastic support from service providers and researchers around Australia, the ABCD Partnership has focused since 2010

**Abbreviations:** ABCD, audit and best practice for chronic disease; CQI, continuous quality improvement.

on exploring clinical performance variation, examining strategies for improving primary care, and working with health service staff, management and policy makers to enhance effective implementation of successful strategies (4). By the end of 2014, the ABCD Partnership had generated the largest and most comprehensive dataset on quality of care in Australian Indigenous primary healthcare settings. The Partnership's work is being extended through the Centre of Research Excellence in Integrated Quality Improvement (6).

Several research papers included in this Research Topic illustrate consistent findings of wide variation in adherence to clinical best-practice guidelines between health centers (Bailie et al.; Burnett et al.; Matthews et al.). The papers also show variation among different aspects of care, with relatively good delivery of some modes of care [Bailie et al.; (7)] and poor delivery of others—such as follow-up of abnormal clinical or laboratory findings. These findings are evident in eye care (Burnett et al.), general preventive clinical care (Bailie et al.), and in absolute cardiovascular risk assessment (Matthews et al.; Vasant et al.). The findings are consistent with other ABCD-related publications on diabetes care (8), preventive health (9), maternal care (10), child health (11), rheumatic heart disease (12), and sexual health (13).

Systems to support good clinical care are explored by Woods et al. in five primary healthcare centers that were identified through ABCD data as achieving substantially greater improvement than others over successive CQI cycles. Attention to understanding and improving systems was shown to be vital to the improvements in clinical care achieved by these health centers. Improved staffing and commitment to working in the community were standout aspects of health center systems that underpinned improvements in clinical care.

On a wider scale, engagement by primary healthcare services in the ABCD Partnership has enabled assessment of system functioning at district, regional, state, and national levels, as reflected in stakeholders' perceptions of barriers and enablers to addressing gaps in chronic illness care and child health, and identifying drivers for improvement (Bailie et al.). Primary drivers included staff capability, availability and use of information systems and decision support tools, embedding of CQI processes, and community engagement. We have also shown how consistent and sustained policy and infrastructure support for CQI enables large-scale and ongoing improvements in quality of care (3).

Commitment of the ABCD team to promoting effective use of CQI data is reflected in one "perspective" paper, which describes a theory-informed cyclical interactive dissemination strategy (Laycock et al.). Concurrent developmental evaluation provides a mechanism for learning and refinement over successive cycles (14).

The other two perspective articles (not specifically from the ABCD program) highlight the role of facilitation in CQI and the potential for application of CQI in health professional education. The emerging evidence on facilitation as a vital tool for effective CQI should guide resourcing and approaches to CQI (Harvey and Lynch). The approach builds on the humanistic principles of modern CQI methods—participation, engagement, shared decision-making, enabling others, and tailoring to context. The framework for CQI approaches to health professional education described by Clithero et al. directly addresses a critical need for innovative approaches to health workforce development that will strengthen community engagement and embed CQI principles into health system functioning. The scale and scope of need in workforce development is strongly evident in findings of the ABCD program.

Importantly, CQI methods are proving useful in assessing and potentially improving delivery of evidence-based health promotion practices (Percival et al.). Percival's experience in this field highlights the health facility and wider system challenges facing effective implementation of CQI methods. In health promotion these barriers include low priority given to health promotion in the face of heavy demands for acute clinical care. This work in health promotion complements other research on applying CQI to social determinants of health more broadly (15), including community food supply (16), housing (17), and education (18).

The publications in this special issue address many of the "building blocks" of high performing primary care described by Bodenheimer and colleagues in the US; namely, four foundational components (engaged leadership, data-driven improvement, empanelment, and team-based care) that are vital to facilitate the implementation of the other six elements (patient-team partnership, population management, continuity of care, prompt access to care, comprehensiveness, and care coordination) (19). They are also relevant to Australian based work on clinical microsystems and development of CQI tools for mainstream general practice, such as the Primary Care-Practice Improvement Tool (with similar components to the ABCD systems assessment tool) (20).

Continuous quality improvement is vital to improving health outcomes through system strengthening. We anticipate substantial future development of CQI methods. By late 2017, there had been over 20,000 views of this Research Topic, and many articles have already been cited in peer-review manuscripts. Further research on CQI in primary healthcare would be well guided by a systematic scoping review of literature summarizing empirical research on current knowledge in the field, and identifying key knowledge gaps.

### AUTHOR CONTRIBUTIONS

RB wrote the first draft. JB has revised content and structure. SL and EB reviewed and edited subsequent drafts. All authors have approved the final version of the manuscript for publication.

### ACKNOWLEDGMENTS

We would like to thank all of those who contributed to this Research Topic as authors, review editors, and colleagues.

## FUNDING

The National Health and Medical Research Council funded the ABCD National Research Partnership Project (grant number 545267) and the Centre for Research Excellence in Integrated Quality Improvement (grant number 1078927). In-kind and financial support was provided by the Lowitja Institute and a range of Community-Controlled and Government agencies.

## REFERENCES


improvement approach. *BMC Health Serv Res* (2013) 13(1):525. doi:10.1186/ 1472-6963-13-525


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. RB was the chief investigator on the ABCD National Research Partnership and is the chief investigator on the Centre of Research Excellence in Integrated Quality Improvement. All papers published in the Research Topic received peer review from members of the Frontiers in Public Health Policy panel of reviewers who were independent of named authors on any given article published in this volume, consistent with the journal policy on conflict-of-interest.

*Copyright © 2017 Bailie, Bailie, Larkins and Broughton. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Determinants and Gaps in Preventive Care Delivery for Indigenous Australians: A Cross-sectional Analysis

*Christopher Bailie1 , Veronica Matthews2 , Jodie Bailie2 , Paul Burgess2,3 , Kerry Copley4 , Catherine Kennedy5 , Liz Moore4 , Sarah Larkins6,7 , Sandra Thompson8 and Ross Stewart Bailie2,9 \**

*1School of Medicine, University of Queensland, Brisbane, QLD, Australia, 2Menzies School of Health Research, Charles Darwin University, Darwin, NT, Australia, 3Primary Health Care Branch, Top End Health Service, Northern Territory Government, Darwin, NT, Australia, 4Aboriginal Medical Services Alliance NT (AMSANT), Darwin, NT, Australia, 5Maari Ma Health Aboriginal Corporation, Broken Hill, NSW, Australia, 6College of Medicine and Dentistry, James Cook University, Townsville, QLD, Australia, 7Anton Breinl Research Centre for Health Systems Strengthening, Australian Institute of Tropical Health and Medicine, James Cook University, Townsville, QLD, Australia, 8Western Australian Centre for Rural Health, School of Primary, Aboriginal and Rural Health Care, University of Western Australia, Geraldton, WA, Australia, 9School of Population Health, University of Queensland, Brisbane, QLD, Australia*

#### *Edited by:*

*Edward Broughton, University Research Co., LLC, USA*

#### *Reviewed by:*

*Elizabeth Querna Cliff, University of Michigan, USA Hugh J. S. Dawkins, Department of Health of Western Australia, Australia*

> *\*Correspondence: Ross Stewart Bailie ross.bailie@menzies.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 22 November 2015 Accepted: 15 February 2016 Published: 10 March 2016*

#### *Citation:*

*Bailie C, Matthews V, Bailie J, Burgess P, Copley K, Kennedy C, Moore L, Larkins S, Thompson S and Bailie RS (2016) Determinants and Gaps in Preventive Care Delivery for Indigenous Australians: A Cross-sectional Analysis. Front. Public Health 4:34. doi: 10.3389/fpubh.2016.00034*

Background: Potentially preventable chronic diseases are the greatest contributor to the health gap between Aboriginal and Torres Strait Islander peoples and non-Indigenous Australians. Preventive care is important for earlier detection and control of chronic disease, and a number of recent policy initiatives have aimed to enhance delivery of preventive care. We examined documented delivery of recommended preventive services for Indigenous peoples across Australia and investigated the influence of health center and client level factors on adherence to best practice guidelines.

Methods: Clinical audit data from 2012 to 2014 for 3,623 well adult clients (aged 15–54) of 101 health centers from four Australian states and territories were analyzed to determine adherence to delivery of 26 recommended preventive services classified into five different modes of care on the basis of the way in which they are delivered (e.g., basic measurement; laboratory tests and imaging; assessment and brief interventions, eye, ear, and oral checks; follow-up of abnormal findings). Summary statistics were used to describe the delivery of each service item across jurisdictions. Multilevel regression models were used to quantify the variation in service delivery attributable to health center and client level factors and to identify factors associated with higher quality care.

Results: Delivery of recommended preventive care varied widely between service items, with good delivery of most basic measurements but poor follow-up of abnormal findings. Health center characteristics were associated with most variation. Higher quality care was associated with Northern Territory location, urban services, and smaller service population size. Client factors associated with higher quality care included age between 25 and 34 years, female sex, and more regular attendance.

Conclusion: Wide variation in documented preventive care delivery, poor follow-up of abnormal findings, and system factors that influence quality of care should be addressed through continuous quality improvement approaches that engage stakeholders at multiple levels (including, for example, access to care in the community, appropriate decision support for practitioners, and financial incentives and context appropriate guidelines).

Keywords: preventive healthcare, Aboriginal and Torres Strait Islander, variation, indigenous, quality of care, adherence to best practice guidelines

#### INTRODUCTION

Worldwide, Indigenous populations experience poorer health than their non-Indigenous counterparts (1). The greatest contributor to the gap in health outcomes between Aboriginal and Torres Strait Islander1 peoples and the general Australian population is potentially preventable chronic disease (2). The critical role of preventive healthcare in curbing the impact of chronic disease is now widely recognized (3, 4). However, reported levels of delivery of recommended preventive care to Indigenous Australians still have much room for improvement (5–9).

Australians have access to Medicare, a universal health insurance scheme. Indigenous Australians access primary health care (PHC) through both private general practice and primary health services designed to meet the needs of Indigenous peoples. These include Aboriginal community-controlled health services (ACCHS) and government operated Indigenousspecific services (10).

Recently, a number of Australian Government policy initiatives have attempted to improve prevention for Indigenous Australians, including the Indigenous Chronic Disease Package (ICDP) from 2009 to 2013 (11). Preventive care priorities within the ICDP were: (a) smoking cessation; (b) increasing uptake of preventive health assessments and follow-up of abnormal findings; (c) workforce training to improve access to care and preventive health care delivery.

Preventive care for Indigenous Australians is incentivized by the Medicare Benefit Schedule (MBS) 715 annual "Health Assessment for Aboriginal and Torres Strait Islander People" that has achieved better identification of chronic disease and its risk factors as well as reduction in cardiovascular disease risk (12–14). Continuous quality improvement (CQI) in Indigenous PHC services has proved effective in improving preventive care delivery through enabling PHC services to identify and address barriers to preventive care (15).

The Audit and Best Practice for Chronic Disease National Research Partnership (ABCD NRP) and One21seventy have supported the use of CQI in Indigenous PHC by providing evidence-based clinical audit and systems assessment tools and through training and assistance with their use (16, 17). More than 270 health centers across Australia have used ABCD/ One21seventy tools and processes to improve their quality of care (18, 19). Data collected from over 170 health centers are available for research purposes through the ABCD NRP.

Obtaining reliable data on delivery of care in relation to best practice guidelines are critical in targeting strategies to improve performance (6). Data to date for individual preventive service items have shown wide variation in delivery between clients and health centers (5–8, 14, 20).

Overall measurement of service delivery may lack the detail needed to identify specific opportunities for improvement in quality of care (21). Understanding performance in relation to the mechanisms by which care processes are delivered (referred to as "modes of care"), for example: basic measurements, lifestyle interventions, laboratory tests, and so on, may allow better insight into higher level system changes needed to improve care quality (21).

The aim of this paper is to assess variation in delivery of preventive health care to Indigenous people using data collected through the ABCD NRP. Specifically, our objectives are to: (1) assess differences in delivery of recommended preventive services; (2) examine health center and client level factors associated with quality care; (3) examine how these factors vary across different modes of care.

### MATERIALS AND METHODS

#### Setting

ACCHS and government operated Indigenous health centers participating in the ABCD NRP were included in this study. These health centers vary in size from small services with 1–2 nursing staff to large services with a range of medical, nursing, and allied health professionals. They predominantly but not exclusively serve Indigenous clients.

#### Data Sources

As part of their routine CQI activities, participating health centers performed annual audits of client medical records to determine whether recommended preventive service items were documented as delivered in the previous 24 months. Audit inclusion criteria were: (1) age between 15 and 55 years; (2) resident in the community for at least six of the last 12 months; (3) no diagnosis of diabetes, hypertension, coronary heart disease, chronic heart failure, rheumatic heart disease or chronic kidney disease; (4) not pregnant or not less than 6 weeks postpartum at the time of audit; (5) attendance at the health center in the previous 24 months.

Health centers were encouraged to audit all client records if their eligible client population was less than 30. For eligible client populations greater than 30, health centers audited a random sample of records of at least 30 eligible clients.

Samples were stratified by age and gender. Data were available from 101 health centers spread across four Australian states and

<sup>1</sup> In the interest of brevity Aboriginal and Torres Strait Islander people are hereafter respectfully referred to as "Indigenous."

territories over the period 2012–2014. Data from the most recent preventive services clinical audit from each health center were included in our analysis.

#### Measures

The One21seventy/ABCD NRP preventive services audit tool contains 26 items recommended for delivery at least every 24 months. We classified items into five modes of care: basic measurements; laboratory and imaging; eye, ear and oral checks; assessment and counseling for lifestyle risk factors; and follow-up of abnormal findings (**Table 1**). A service was recorded as delivered for each eligible client if there was clear record of delivery at least once within the previous 24 months.

Four follow-up items were included in the audit. These were follow-up for high blood pressure (BP), for protein on urinalysis, for high blood glucose level (BGL), and for abnormal lipid

#### TABLE 1 | Percentage delivery of preventive service items by jurisdiction.


*Number of client records and health centers are overall for each jurisdiction. The actual number of client records is lower for some service items that are recommended for restricted populations.*

*To calculate delivery we assumed that: a all adults had been sexually active; bthat adults had no family history of premature CHD or CKD; c that females were not at above average risk of breast cancer; for dthat all remote locations were trachoma endemic areas.*

*NAAT, nucleic acid amplification test; BP, blood pressure; CHD, chronic heart disease; CKD, chronic kidney disease.*

*\*National guide to a preventive health assessment for Aboriginal and Torres Strait Islander people (22).*

† *Standard treatment manual (23).*

‡ *Chronic disease guidelines (24).* profile. Definitions of adequate follow-up, respectively, were a documented plan for: (1) repeat BP measurement in 2–4 weeks; (2) testing albumin-creatinine ratio; (3) re-testing BGL; (4) re-testing lipid profile. Denominators for calculating indicators of follow-up were based on numbers of clients who had an abnormal result for the relevant clinical/laboratory investigation within the past 2 years.

Some guidelines restrict recommended delivery of some services to special populations, for example, based upon a family history or when a person is sexually active. In cases where audit data were insufficient to define these special populations we made the following assumptions in order to estimate delivery: (a) for items related to sexual and reproductive health, we assumed that all young adults had been sexually active [noting the high rate of sexual activity among young Australians (25)]; (b) for serum lipid testing, we assumed that no clients had a family history of premature coronary heart disease or chronic kidney disease; (c) for mammography, we assumed that no female clients were at above average risk of breast cancer; (d) for trachoma screening, we assumed that all remote health centers were located in trachoma endemic areas, and therefore that screening was recommended [noting the higher rates of trachoma in remote and very remote areas (26)].

Health center characteristics included, size of population served, governance (ACCHS or government operated), and location (urban, regional, or remote). Duration of CQI participation has been shown to be associated with improved quality of care (27) and was therefore included in the analysis to correct for potential bias associated with different health centers participating in CQI for different lengths of time. The clinical audit data included information on the following client characteristics; age, gender, Indigenous status, and whether or not the client had attended the health center in the previous 6 months.

#### Statistical Analysis

We used STATA version 13 software for statistical analysis. Using summary statistics, we described the mean delivery of each service item for each jurisdiction (**Table 1**).

For the purposes of the analysis, the data from South Australia (SA) and Western Australia (WA) health centers were pooled because (a) there were relatively small number of participating health centers (two from WA, six from SA); (b) service delivery levels were relatively similar for SA and WA compared to other jurisdictions; and (c) SA and WA had relatively less developed support structures for CQI processes compared to the Northern Territory (NT) and Queensland (QLD) over the period covered by this study.

Adherence to delivery of recommended service items was calculated for each mode of care by dividing the documented service items for that client by the total number of recommended service items in that mode of care. We also calculated adherence to delivery for an overall composite indicator that included all service items except those related to follow-up of abnormal findings (**Table 1**).

Aggregate scores for the composite indicator and each mode of care were converted into binary outcome indicators that categorized "higher" performance as being above the median (top 50%) of delivery for all clients across all health centers. "Lower performance" was categorized as being below or equal to the median.

We used multi-level mixed effects logistic regression models to quantify the variation in service delivery separately for the overall indicator and each mode of care. These models allowed for the hierarchical structure of the data (clients nested within health centers).

For each indicator, we calculated unadjusted odds ratios to measure the association between "higher" levels of delivery and each health center and client characteristic.

For the adjusted analysis, we used a stepwise modeling strategy starting with an "empty" model with no explanatory variables (Model A). Health center (Model B) and then client level variables (Model C) were then introduced into the empty model. Potential interactions were introduced into the final model (Model C) in a stepwise manner and their significance was tested. No interactions were found to make a meaningful difference, and none were included in the final analysis. The model was tested for sensitivity to alternate specifications (including alternative cut points for the outcome variable) during the model building process and was found to be generally robust.

The reduction in variance due to the stepwise introduction of the client and health center level variables in the models was determined by the proportional change in variance (PCV). The PCV provides an estimate of the extent to which these factors may explain differences in propensity for better delivery of health care (28).

We calculated median odds ratios (MORs) to interpret variance in the odds ratio scale. In the odds ratio scale, the MOR describes the increase in median probability of better delivery if a client was to move from one randomly picked health center to another (29). For a MOR equal to 1, there is no difference between health centers in their probability of adhering to the recommended service delivery. The greater the MOR, the greater the unexplained variability between health centers.

### Ethics Approvals

Ethics approval has been obtained for the ABCD NRP project research ethics committees in relevant jurisdictions of Australia. These include the Human Research Ethics Committee (HREC) of the Northern Territory Department of Health and Menzies School of Health Research (HREC-EC00153); Central Australian HREC (HREC-12-53); Queensland HREC Darling Downs Health Services District (HREC/11/QTDD/47); South Australian Aboriginal Health Research Ethics Committee (04-10-319); Curtin University HREC (HR140/2008); Western Australian Country Health Services Research Ethics Committee (2011/27); Western Australia Aboriginal Health Information and Ethics Committee (111-8/05); and University of Western Australia HREC (RA/4/1/5051).

## RESULTS

Of the 101 participating health centers, 93 were located in the NT and QLD. More than 90% from QLD and the NT were located in a regional or remote area compared to 50% for SA and WA (**Table 2**). Fifty percent of health centers had a population of fewer than 500 people, 84% were government operated.

Records of 3,623 clients from these health centers were audited from February 2012 to December 2014. The sample size for each health center varied between 7 and 68 client records. Of these, 50% were females and 90% were Indigenous (**Table 2**).

The proportion of eligible clients receiving recommended care ranged from 18 to 85% for individual preventive services (**Table 1**). The mean delivery of the composite indicator including all service items except follow-up was 58%. Most basic measurements were delivered at levels of approximately 70–80% (**Table 1**); although recording of body mass index (BMI), waist circumference, and urinalysis were relatively low (37–47%). Most recommended laboratory and imaging investigations were delivered to approximately 50% of clients, apart from mammography which was delivered to 18% of eligible females. Eyes, ears, and oral checks were recorded as delivered to 18–45% of eligible clients. Delivery of service items to do with assessment and brief intervention for lifestyle risk factors ranged from 46 to 72%. Follow-up of abnormal findings was relatively low (17–28%) except for follow-up of positive protein on urinalysis (61%).

The unadjusted logistic regression analysis for the composite overall indicator showed significant effects for all factors except governance (**Table 3**). The health center MOR for the empty model for the overall indicator was 4.02 (**Table 4**; Model A), meaning if a client were to move from one randomly picked health center to another with higher delivery, they would have a 4.02 times higher chance (in median) of higher delivery. For the adjusted analysis for the overall composite indicator, the reduction in health center level variance for the addition of health center factors (PCV) was 60% (**Table 4**; Model B). Health center factors associated with higher levels of delivery included urban location, smaller service population, and location in the NT. Client level factors associated with higher delivery included being aged 25–34 years compared to other age groups, female gender, and more recent health center attendance (**Table 4**; Model C).

The PCV in Model B for each mode of care (additional files 1–5) ranged between 14 and 67% for different models of care with follow-up being the lowest and basic measurements being the highest. Health center factors associated with higher delivery were similar across modes of care (**Table 5**). There was some variation across jurisdictions with NT health centers significantly associated with higher delivery for all modes of care except for lifestyle risk factors and follow-up. The pattern of client level effects was similar across modes of care except for gender, where females were more likely to receive basic measurements and assessment and intervention for lifestyle risk factors but less likely to have laboratory and imaging investigations. There was some variation in the effect of age group.

### DISCUSSION

Australia has lacked high quality systematically collected information on the quality of preventive care delivered in general practice and other primary health care settings. This study examining the most comprehensive dataset of its kind currently available in Australia provides the most extensive snapshot of delivery of



 **<** 0.01).


TABLE 4 | Adjusted multilevel logistic regression analysis of health center and client level factors on a composite indicator of guideline-scheduled preventive service items recommended for well adults (*N* **=** 3,623 clients, 101 health centers) (\**p* **<** 0.05; \*\**p* **<** 0.01) (see Table 1 for service items included in this indicator).

*Median odds ratio (MOR): odds of receiving above median service delivery if client was to change health center or jurisdiction; proportional change in variance (PCV): per cent variation explained in odds for better health care delivery by introduction of health center or client level factors.*

preventive care to Indigenous people across Australia to date. We discuss the key findings around three themes: (a) Variation between aspects of care and key opportunities for improvement; (b) follow-up of identified risk factors and abnormal clinical and laboratory findings; and (c) variation between health centers. Key considerations for policy, practice, and research arising from these findings are outlined in **Table 6**.

### Variation between Aspects of Care and Key Opportunities for Improvement

There is substantial variation in the delivery of recommended preventive services; from about 70 to 80% for most basic measurements to less than 20% for mammogram screening for breast cancer. While comparison with previously published research is limited by the comparability of data, the patterns of service delivery evident in our study are generally similar to those previously described, with generally good delivery of basic measurements and lower documented delivery of recommended laboratory investigations (5, 8, 20). While our analysis is consistent with previous findings of reasonable levels of assessment of lifestyle risk factors (5, 8, 20), we found higher levels of brief intervention to address these risk factors than previously reported, especially for alcohol and tobacco use.

Several service items including measurement of BMI and waist circumference stand out for being simple and quick to complete yet having low levels of delivery. This might reflect a lack of decision support within the clinic records, patient refusal or reluctance among some clinicians to engage clients in discussion of their weight. As the vast majority of clients present for reasons other than a preventive health assessment (8), the context of an acute presentation may detract from providing preventive care to people with no diagnosed chronic disease and isolated and non-urgent risk factors.

Low levels of documented delivery of some laboratory tests may relate to practical barriers to laboratory services in the

#### TABLE 5 | Adjusted multilevel logistic regression analyses of health center and client level factors on delivery of guideline-scheduled service items by mode of care showing only significant associations (see additional files for full model outputs) (\**p* **<** 0.05; \*\**p* **<** 0.01).


remote locations in which most services included in this study are operating. It may also reflect poor integration of pathology results into electronic health records.

The low level of mammogram screening may be due to limited access to medical imaging services in remote areas and reflects a wider problem of access to specialized services for Indigenous Australians (30). Mammography screening for remote communities relies on either mobile screening units with a limited number of visits, or patient travel to larger centers which is not currently funded in the NT (33).

Our findings that female gender, being aged 25–34 years and recent health center attendance are associated with higher delivery of preventive care are consistent with previous studies (5, 7). That better preventive care delivery was significantly higher for females compared to males is likely to reflect a number of factors, including differences in health-seeking behaviors, proactive women's health staff and that health centers tend to be predominantly female staffed so may be seen as "women's places." The relatively lower level of preventive care delivered to males represents a key opportunity for improvement.

### Follow-up of Risk Factors and Abnormal Findings

Documented evidence of follow-up of various identified risk factors and of abnormal clinical and laboratory findings was poor (notably follow-up for abnormal BP, BGL, and serum lipids) and arguably represents the most significant opportunity for improving early intervention for chronic diseases. Reasons for lack of follow-up are varied and are present at the patient, health


TABLE 6 | Key findings and considerations for policy, practice, and research.

center, and policy levels (31). Lack of knowledge of the reasons for follow-up among some patients and therefore low demand, as well as concerns regarding transport and the cost of follow-up services may contribute to low levels of follow-up (31). Follow-up is likely to be relatively more difficult for clients who attend the health center infrequently; more than one quarter of clients in our study had not attended in the previous 6 months and many Indigenous people have a high level of mobility.

### Variation between Health Centers

In contrast with previous findings that client level factors were responsible for the majority of inter-client variability in delivery (5), we found measured health center factors explained the majority of variation in delivery of preventive care. Urban location, smaller service population, and location in the NT were associated with higher quality care.

Our findings support a previous study showing that a smaller health center service population is associated with greater adherence to best practice guidelines. However, in contrast to another finding of that study that committee or board operated health centers performed better than their government run counterparts we found no difference in outcomes between the two (5). The association of health center location in the NT may reflect greater investment in PHC, a longer and stronger history of engagement in CQI, partnership between ACCHS and government operated services, support for guideline implementation, and possibly greater commitment to information management compared to other jurisdictions.

In contrast to previous research, we found that urban location was associated with better delivery compared to regional or remote (5). This finding in particular should be interpreted with caution given the relatively small number and selected nature of urban health centers included in our study. Excluding one especially high performing urban health center from our analysis caused the effect of remote location compared to urban location to be insignificant for overall delivery, although regional location was still significantly associated with poorer delivery. If this effect does hold for the wider population of health centers, it may in part reflect greater access to referral services in urban locations as remote practitioners may be reasonably reluctant to carry out services such as visual acuity testing and oral health checks where there are more limited options for referral for treatment. It may also reflect resourcing and funding advantages in urban areas or a strategic decision among larger urban health centers to invest in Medicare funded health assessments as an opportunity to increase service remuneration. There has been wide variation in uptake of adult health assessments between health centers (6, 34), with substantial increases in urban and regional locations in the last few years and relatively little change in remote locations (34).

There are various factors that influence the effectiveness of PHC centers, which can be complex and difficult to measure, and may account for the unexplained variation between health centers (35). These include community linkages, organizational culture, effectiveness of team structure and function, degree of staff turnover, availability of Aboriginal Health Practitioners, allied health professionals and other resources, and use of information technology systems for recall and reminders (27, 35, 36).

Limitations of this study mostly relate to the generalizability of findings. Health centers participated on a voluntary basis were not randomly selected, therefore, our data may not be representative of all health centers in each jurisdiction. Furthermore, data refer only to those people who have attended participating health centers within the last 24 months and therefore do not provide reliable population estimates. The age and gender stratified samples are designed to facilitate comparison between communities and over time. Estimates based on this sampling approach may differ from those based on unstratified samples.

It must be emphasized that our data are based on recorded delivery, which generally underestimate actual service delivery (37). However, under-recording is problematic in team based care and in areas of workforce turnover as well as potentially resulting in unnecessary servicing, so accurate recording is important for improving quality of care (5). Assumptions made regarding

#### REFERENCES


prevalence of sexual activity and trachoma are likely to have led to underestimation of actual levels of delivery in relation to best practice guidelines while those concerning lipid testing and mammography may have led to some over- or underestimation.

### CONCLUSION

We assessed the delivery of recommended preventive health care for clients of Indigenous health centers. Wide variation in delivery between service items, low levels of documented follow-up for abnormal clinical findings, and the importance of health center factors in determining adherence to best practice guidelines provide valuable insights for improving quality of preventive care for Indigenous Australians. Improvement may be achieved by addressing physical, social, and cultural barriers to accessing preventive care, and by strengthening systems for follow-up and completion of preventive health assessments. Further clarification of the impact of health center factors such as resourcing, team structure and function, and use of clinical information systems will give better insight into possible improvements. Addressing identified gaps in preventive screening according to best practice and prioritizing interventions to address these can be implemented through CQI approaches that engage stakeholders at multiple levels (15, 38).

### AUTHOR CONTRIBUTIONS

CB designed and conducted the analysis, and drafted the manuscript under the supervision of VM who has lead responsibility for ABCD NRP data management and analysis. All other authors contributed to design of audit tools and facilitation of quality improvement processes from which the data used in this study arise, and contributed to data interpretation and refinement of the manuscript.

### ACKNOWLEDGMENTS

The development of this manuscript would not have been possible without the active support, enthusiasm, and commitment of members of the ABCD National Research Partnership, especially the participating primary health care centers that have provided de-identified data.

### FUNDING

The ABCD National Research Partnership Project was supported by funding from the National Health and Medical Research Council (545267) and the Lowitja Institute, and by in-kind and financial support from a range of Community Controlled and Government agencies. RB's work is supported by an Australian Research Council Future Fellowship (100100087).


community health centres in Australia. *BMC Health Serv Res* (2011) **11**:139. doi:10.1186/1472-6963-11-139


**Conflict of Interest Statement:** The authors declare that the above research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Bailie, Matthews, Bailie, Burgess, Copley, Kennedy, Moore, Larkins, Thompson and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Delivery of Eye and Vision Services in Aboriginal and Torres Strait Islander Primary Healthcare Centers

*Anthea M. Burnett1,2,3 \*, Anna Morse1,2, Thomas Naduvilath1,2 , Andrea Boudville2,4 , Hugh R. Taylor2,4 and Ross Bailie5*

*1Brien Holden Vision Institute, Public Health, Sydney, NSW, Australia, 2Vision CRC, Sydney, NSW, Australia, 3School of Optometry and Vision Science, University of New South Wales, Kensington, NSW, Australia, 4 Indigenous Eye Health, Melbourne School of Population and Global Health, The University of Melbourne, Parkville, VIC, Australia, 5Menzies School of Health Research, Charles Darwin University, Brisbane, QLD, Australia*

Background: Routine eye and vision assessments are vital for the detection and subsequent management of vision loss, which is particularly important for Aboriginal and Torres Strait Islander people who face higher rates of vision loss than other Australians. In order to guide improvements, this paper will describe patterns, variations, and gaps in these eye and vision assessments for Aboriginal and Torres Strait Islander people.

#### *Edited by:*

*Diane Cooper, University of the Western Cape, South Africa*

#### *Reviewed by:*

*Anu Mary Oommen, Christian Medical College, India Ginny Marisa Sargent, Australian National University, Australia*

*\*Correspondence: Anthea M. Burnett a.burnett@brienholdenvision.org*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 05 August 2016 Accepted: 05 December 2016 Published: 19 December 2016*

#### *Citation:*

*Burnett AM, Morse A, Naduvilath T, Boudville A, Taylor HR and Bailie R (2016) Delivery of Eye and Vision Services in Aboriginal and Torres Strait Islander Primary Healthcare Centers. Front. Public Health 4:276. doi: 10.3389/fpubh.2016.00276*

Methods: Clinical audits from 124 primary healthcare centers (sample size 15,175) from five Australian states and territories were conducted during 2005–2012. Main outcome measure was adherence to current guidelines for delivery of eye and vision assessments to adults with diabetes, those without a diagnosed major chronic disease and children attending primary healthcare centers.

Results: Overall delivery of recommended eye and vision assessments varied widely between health centers. Of the adults with diabetes, 46% had a visual acuity assessment recorded within the previous 12 months (health center range 0–88%) and 33% had a retinal examination recorded (health center range 0–73%). Of the adults with no diagnosed major chronic disease, 31% had a visual acuity assessment recorded within the previous 2 years (health center range 0–86%) and 13% had received an examination for trichiasis (health center range 0–40%). In children, 49% had a record of a vision assessment (health center range 0–97%) and 25% had a record of an examination for trachoma within the previous 12 months (health center range 0–100%).

Conclusion: There was considerable range and variation in the recorded delivery of scheduled eye and vision assessments across health centers. Sharing the successful strategies of the better-performing health centers to support focused improvements in key areas of need may increase overall rates of eye examinations, which is important for the timely detection, referral, and treatment of eye conditions affecting Aboriginal and Torres Strait Islander people, especially for those with diabetes.

Keywords: Aboriginal and Torres Strait Islander people, primary healthcare centers, delivery of health care, eye care, diabetes, quality of health care, quality indicators

## INTRODUCTION

Aboriginal and Torres Strait Islander Australians experience significantly higher rates of vision impairment than other Australians (1, 2). The largely avoidable, preventable, or treatable nature of the majority (94%) of these cases (2) indicates the need for improved early detection pathways, timely referral, and appropriate and accessible treatment. Additionally, as Aboriginal and Torres Strait Islander adults with diabetes, older than 40 years, form 72% of those requiring an eye examination in any year (3), understanding ways to further improve access and uptake of eye assessments for patients with diabetes is important, given their higher risk of preventable vision loss.

Eye health services for Aboriginal and Torres Strait Islander communities are typically provided by visiting practitioners; even more so in remote or very remote locations (4). However, there is still a shortage of optometric and ophthalmic services in many rural and remote areas (4, 5) and significantly lower rates of eye examinations (by optometrists or ophthalmologists) in areas with higher proportions of Aboriginal and Torres Strait Islander people (4, 6). Current policy recommendations for better Aboriginal and Torres Strait Islander eye care in the "Roadmap to Close the Gap for Vision" place emphasis on primary eye care as part of comprehensive primary health care to address barriers to eye care (7). Similarly, international eye care strategies highlight the key role of primary health care in preventing vision loss and blindness (8).

As primary healthcare (PHC) centers are the frontline of health service delivery, they can often be the first point of contact for Aboriginal and Torres Strait Islander adults with diabetes or vision/eye problems. Hence, PHC centers play a crucial role in eye care (9), especially for patients with diabetes (7). Basic eye and vision screening assessments are conducted during routine health assessments such as the Aboriginal and Torres Strait Islander Health Assessment (10). When linked with distinct eye care referral processes, regular screening can help to identify and refer eye problems earlier, preventing vision loss (11). Primary healthcare practitioners also play a case management role, supporting and coordinating patients' timely access to comprehensive eye examinations and specialist eye care, particularly for patients with chronic conditions such as diabetes (12). This process can help improve efficiency of eye care service delivery systems by identifying and referring cases needing comprehensive eye care, targeting visiting eye care services to patients who most need them, and detecting vision problems at earlier stages (particularly important in the case of diabetic retinopathy) (13).

To gain insight into primary eye care coverage in Aboriginal and Torres Strait Islander PHC centers and to establish a baseline for comparison with future studies, we undertook an exploratory analysis of datasets from clinical file audits of PHC centers participating in the quality improvement action research project – the Audit and Best practice for Chronic Disease (ABCD) project (14). This study describes patterns, variations, and gaps in eye and vision assessments and associations with geographic location of health center, patient age, gender, and health center attendance. We discuss the implications of the findings with a focus on identifying approaches that will drive improvements in primary eye care services for Aboriginal and Torres Strait Islander communities.

### MATERIALS AND METHODS

### Study Population and Data Collection

The data presented here were collected as part of a national quality improvement project – the ABCD project (15), between 2005 and 2012. One hundred and twenty-four Aboriginal and Torres Strait Islander PHC centers in five states/territories voluntarily performed annual audits of client medical records and provided de-identified audit data to the ABCD National Research Partnership to investigate variations in quality of care. The audits were conducted by trained members of the project team in conjunction with local PHC center staff using three standardized audit tools and protocols developed by the Menzies School of Health Research. These tools assess: (1) delivery of services to clients with Type 2 diabetes, (2) delivery of preventative health care, and (3) delivery of child health care. For each of these client cohort datasets, the delivery of eye and vision services according to existing best practice guidelines (**Table 1**) was assessed.

For the three client cohorts, the records of Aboriginal and Torres Strait Islander clients who met the following criteria were eligible for audit: (1) Aboriginal and Torres Strait Islander patients with a definite diagnosis of Type 2 diabetes aged 15 years and over, (2) Aboriginal and Torres Strait Islander adults with no diagnosed major chronic disease attending the PHC center, in the prior 24 months from the date of the audit, for an annual well-person's check, acute care, or a preventative service, and aged between 15 and 64 years, and (3) Aboriginal and Torres Strait Islander children aged 15 years and under. For each of these client cohorts, clients were required to be residents of the community for at least 6 months of 12 months prior to the audit (or in the case of an infant, half the infant's life) in order to be eligible. A random sample of 30 clinical records for each cohort was audited from participating centers (**Table 2**), where there were fewer than 30 eligible records identified, all eligible records were included. Eye and vision services were assessed as "delivered" if there was a record of the service being delivered within specific periods in line with best practice eye care guidelines.

### Statistical Analysis

Treating individual clients as the unit of analysis, our data had inherent multilevel, dependency structure as eye and vision care data collected at the individual level were clustered within health centers. Multistage logistic regression models were used to examine associations of specific factors (location, age, gender, and attendance), with delivery of eye and vision care services (**Tables 3**–**5**). The outcomes included eye examination, vision assessment, and examination for trichiasis and trachoma. The year of audit was added as a factor in the model to account for the variation over time. Association with outcomes was described using odds ratio and 95% confidence intervals. Variations in eye or vision assessments between health centers were described using violin plots. Level of statistical significance was set at 5%.


TABLE 1 | Recommended eye and vision assessments for Aboriginal and Torres Strait Islander Australians.

*VA, visual acuity; NT, the Northern Territory; QLD, Queensland.*

*NACCHO/RACGP – National guide to a preventive health assessment for Aboriginal and Torres Strait Islander people (16, 17).*

*NH&MRC – guidelines for the management of diabetic retinopathy (18, 19).*

*Medicare benefit schedule item 708 – Aboriginal and Torres Strait Islander Child Health Check (MBS item 708) (20).*

*Medicare benefit schedule item 715 – Medicare Health Assessment for Aboriginal and Torres Strait Islander people (MBS ITEM 715) (21).*

*a Communicable Diseases Network Australia (CDNA) guideline recommends screening by jurisdictional teams.*

Analysis was performed using STATA and SPSS (Version 22.0., IBM Corp., Armonk, NY, USA).

Ethics approval was obtained from research ethics committees in each jurisdiction [Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research (HREC-EC00153); Central Australian Human Research Ethics Committee (HREC-12-53)]; New South Wales Greater Western Area Health Service Human Research Committee (HREC/11/GWAHS/23); Queensland Human Research Ethics Committee Darling Downs Health Services District (HREC/11/ QTDD/47); South Australian Aboriginal Health Research Ethics Committee (04-10-319); Curtin University Human Research Ethics Committee (HR140/2008); Western Australian Country Health Services Research Ethics Committee (2011/27); Western Australia Aboriginal Health Information and Ethics Committee (111-8/05); University of Western Australia Human Research Ethics Committee (RA/4/1/5051).

#### RESULTS

The records of 15,175 Aboriginal and Torres Strait Islander clients were audited from 124 participating health centers across various locations (comprising city, regional, and remote jurisdictions) from five Australian states and territories (**Table 2**). Among the records of adults with diabetes, 46% (3,320/7,320) had a VA assessment recorded and 33% (2,381/7,300) had records of receiving a retinal examination within the previous 12 months. Of the records of the adults with no diagnosed major chronic disease, 31% had a VA assessment recorded within the last 2 years (236/759), while only 13% (380/2,829) of the audited records had received an examination for trichiasis. From the records of children, 49% (2,415/4,909) had a vision assessment recorded within the past 12 months (guidelines recommend an annual assessment for children aged ≥4 years in NT or ≥3 months in all other areas), 45% (2,085/4,632) had a record of an eye examination and 25% (223/893) had a record of an examination for trachoma (guidelines recommend an annual examination for children aged ≥4 years old in the NT, or if indicated in other states and territories).

There was significant variability in documented delivery of these assessments by state/territory and location (**Tables 3**–**5**). Participating PHC centers in New South Wales (NSW), the Northern Territory (NT), and Queensland (QLD) had relatively high recorded VA assessments and retinal examinations for adults with diabetes, while Western Australian (WA) health centers recorded higher rates of trichiasis examinations to adults with no diagnosed major chronic disease. Similarly, variation was observed in delivery between states among the audited child records, with participating PHC centers in NSW and South Australia (SA) delivering the most eye assessments and those in the NT recording the most trachoma examinations for children.

There was considerable variation in delivery of scheduled services to Aboriginal and Torres Strait Islander clients across health centers (**Figure 1**). The range between health centers for delivery of VA assessments to adults with type 2 diabetes was from 0 to 88%, while the delivery of retinal examinations ranged from 0 to 73%. For adults with no diagnosed major chronic disease, the range of documented delivery of visual acuity assessments was between 0 and 86%, with the range of trichiasis examinations delivered between 0 and 40%. Some centers provided vision and

#### TABLE 2 | Characteristics of the three cohorts audited.


*a Not all health centers participated in all three audits.*

*bOne participant was aged 68.5 years and was retained in analysis.* 

TABLE 3 | Adjusted multilevel logistic regression analysis of patient characteristics and documented delivery of vision and eye health services for Aboriginal and Torres Strait Islander adults with diabetes in participating health centers between June 2005 and August 2012.


*VA, visual acuity, NSW, New South Wales, NT, Northern Territory, QLD, Queensland, WA, Western Australia, SA, South Australia.*

*Odds ratios significant at 0.05 level are shown in bold.*

*a Within the previous 12 months.* TABLE 4 | Adjusted multilevel logistic regression analysis of patient characteristics and documented delivery of vision and eye health services for Aboriginal and Torres Strait Islander adults in participating health centers between June 2005 and August 2012.


*VA, visual acuity; NSW, New South Wales; NT, Northern Territory; QLD, Queensland; WA, Western Australia; SA, South Australia.*

*Odds ratios significant at 0.05 level are shown in bold.*

*a Within the previous 2 years.*

TABLE 5 | Adjusted multilevel logistic regression analysis of patient characteristics and documented delivery of vision and eye health services for Aboriginal and Torres Strait Islander children in participating health centers between June 2005 and August 2012.


*Odds ratios significant at 0.05 level are shown in bold.*

*a Within the previous 12 months.*

eye assessments to 100% of eligible children – the delivery of trachoma and eye examinations to eligible children ranged from 0 to 100%, while the delivery of the vision assessment ranged from 0 to 97%.

Documented delivery of VA and retinal examinations increased with age for adults with diabetes, but not for documented delivery of VA and trichiasis assessments to adults with no major chronic disease (**Table 4**). There was no variability between genders for adults (**Tables 3** and **4**) or children (**Table 5**). Documented eye assessments were significantly more likely for patients who had visited the health center within the previous 12 months.

## DISCUSSION

Our investigation revealed significant variation in the documented delivery of eye and vision assessments to Aboriginal and Torres Strait Islander clients attending PHC centers. While some centers provided excellent levels of assessments, others provided low levels. This variation in performance presents an opportunity for improvement in the documented delivery of these assessments by examining the factors that underlie variation in delivering services, as has previously been conducted in Aboriginal and Torres Strait Islander communities for preventative and diabetes care (22, 23). The successful strategies of the better-performing health centers could then be shared with the under-performing, to inform and support focused improvements in key areas of need.

On average, clients with diabetes were more likely to have a VA assessment recorded (46%) than adults with no diagnosed major chronic disease (31%). This may reflect the stricter adherence to routine screening and regular monitoring for patients with diabetes encouraged by chronic disease management plans. It may also reflect practitioners' knowledge of the risk of increased vision loss for patients with diabetes or patients' reports of issues with their vision. Additionally, patients with greater engagement in their health care may be more likely to engage with eye care services.

In this study, only 33% of Aboriginal and Torres Strait Islander adults with diabetes had a documented retinal examination within the previous 12 months. This is consistent with a previous audit of patient files in the NT, which reported that 34% had a documented fundus examination in the prior 12 months (24). The National Indigenous Eye Health Survey conducted in 2008 also reported low frequencies (20%) of eye examinations for patients with diabetes (self-reported) (2). Understanding ways to further improve access and uptake of eye assessments for patients with diabetes is important, given their higher risk of preventable vision loss. As retinal examinations largely rely on services being provided by visiting or off-site eye practitioners, utilizing retinal photo-screening integrated with primary care, image grading, and reporting systems may increase rates of examinations. This strategy has been shown to improve screening outcomes for other Australians with diabetes (25, 26). As we were unable to determine which centers used retinal imaging in this dataset, we could not investigate whether retinal imaging influences examinations rates. This may be useful in future audits.

Australia is the only high-income country in the world where trachoma is endemic – trachoma currently occurs in remote and very remote Aboriginal communities in the NT, SA, and WA (27), with pockets of trachoma in Far West NSW and Far North QLD (2). Although trachoma examinations were recorded more frequently for NT children, 60% of children from the NT did not have a trachoma examination recorded, despite guidelines stating that all children aged 4 years or older should be examined for trachoma annually. In the other participating states, less than 10% of children in areas where trachoma was indicated had a recorded trachoma examination. The National Trachoma Surveillance and Reporting Unit (NTSRU) jurisdiction covers communities designated as being at-risk or potentially at-risk of trachoma and has reported trachoma screening rates ranging from of 63 to 92% (27). Our results are significantly less than this, and this would seem to reflect the failure to record in the clinical records the trachoma examinations conducted by the jurisdictional trachoma screening programs. Improved coordination between external trachoma screening services and PHC centers will enable PHC centers to continue to make a significant contribution toward closing the gap for vision in Australia (9).

Several limitations should be acknowledged when interpreting these data. First, the data only indicate recorded services provided. Given that some aspects of eye care may be accessed in other off-site settings (e.g., optometry and ophthalmology services), it is probable that the rates of retinal examinations may be higher than recorded in PHC records. Second, as these data are only from the Aboriginal and Torres Strait Islander health centers that agree to have their audit data included in the aggregate for the ABCD project, findings are not necessarily representative of all health centers across Australia. Third, the relatively small numbers, self-selection, and uneven distribution of participating health centers in some states/territories means that the data cannot be regarded as broadly representative for these jurisdictions and hence any comparison between state and territory data should only be considered as representative of the cluster of health services participating from each jurisdiction. The inclusion of the state/territory variables in the multivariate models was primarily to enhance model fit and adjust for state/territory level confounders. Finally, as the timeframe in which the adult audit data was collected (2005–2012) precedes the date (2013) from when adult vision and eye assessments became mandatory within Aboriginal and Torres Strait Islander health assessments (MBS715) (28), it is possible that reporting has since improved. A useful area for future analysis would be to determine whether mandatory inclusion of vision tests in adult health assessments has had a significant impact on rates of recorded eye assessments – these results provide an important baseline from which future improvements could be demonstrated. Furthermore, given that the ABCD program and associated audits exist for the purpose of supporting continuous quality improvement (CQI), another useful area for future study may be to track changes in recorded rates of eye and vision assessments over time.

This study has identified opportunities for PHC centers to increase the documented delivery of eye and vision assessments to Aboriginal and Torres Strait Islander clients. Establishing or strengthening systems for external eye practitioners to report back to PHC practitioners' results from retinal examinations or retinal photograph for people with diabetes may lead to notable improvements and represents a potential "quick win" to increase the rates of recorded retinal examinations. More importantly, it would also offer better patient care and coordination by informing PHC practitioners' of eye care history for the patients they oversee. Similarly, strengthening coordination with external trachoma screening programs may also allow PHC practitioners to better monitor trachoma endemicity in their community.

### CONCLUSION

Routine eye and vision assessments for Aboriginal and Torres Strait Islander adults and children attending PHC centers are currently not being recorded at the recommended levels with considerable variation between health centers. These results can represent a baseline, from which improvements in primary eye and vision assessments for Aboriginal and Torres Strait Islander Australians could be made and monitored.

These results also highlight the value of performing clinical audits to identify aspects of eye care and health centers that are being conducted relatively well, or need improvement. The successful strategies of the better-performing health centers could then be shared with the under-performing, to inform and support focused improvements in key areas of need.

### AUTHOR CONTRIBUTIONS

AB interpreted the data and drafted the manuscript. AM contributed to interpretation of the data, drafting, and critically revised the manuscript. TN analyzed and the data and contributed to interpretation. AB, HT, and RB critically reviewed and commented on the project design and on the manuscript. All authors read and approved the final manuscript.

### ACKNOWLEDGMENTS

This project was only possible with the contributions and partnership with the ABCD National Partnership Project team. We thank Veronica Matthews for her expertise in data analysis and reviewing the manuscript.

### FUNDING

The ABCD National Research Partnership Project is supported by funding from the National Health and Medical Research Council (ID No.545267) and the Lowitja Institute, and by in-kind and financial support from a range of Aboriginal Community Controlled and Government agencies. This study was funded by the Brien Holden Vision Institute and the Australian Government's CRC scheme through the Vision CRC.

### REFERENCES


*National Guide to a Preventative Health Assessment for Aboriginal and Torres Strait Islander People*. South Melbourne: The RACGP (2012).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Burnett, Morse, Naduvilath, Boudville, Taylor and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Integrated Clinical Decision Support Systems Promote Absolute Cardiovascular Risk Assessment: An Important Primary Prevention Measure in Aboriginal and Torres Strait Islander Primary Health Care

*Veronica Matthews <sup>1</sup> \*, Christopher P. Burgess <sup>2</sup> , Christine Connors <sup>2</sup> , Elizabeth Moore3 , David Peiris <sup>4</sup> , David Scrimgeour <sup>5</sup> , Sandra C. Thompson6 , Sarah Larkins <sup>7</sup> and Ross Bailie1*

*<sup>1</sup> The University of Sydney, University Centre for Rural Health – North Coast, Lismore, NSW, Australia, 2 Top End Health Service, Northern Territory Government, Darwin, NT, Australia, 3Aboriginal Medical Services Alliance Northern Territory, Alice Springs, NT, Australia, 4 The George Institute for Global Health, Sydney, NSW, Australia, 5Spinifex Health Service, Tjuntjuntjara, WA, Australia, 6Western Australian Centre for Rural Health, University of Western Australia, Geraldton, WA, Australia, 7College of Medicine and Dentistry, James Cook University, Townsville, QLD, Australia*

#### *Edited by:*

*Amelia Kekeletso Ranotsi, Maluti Adventist College, Lesotho*

#### *Reviewed by:*

*Jan Fizzell, NSW Health, Australia Colin MacDougall, Flinders University, Australia*

*\*Correspondence:*

*Veronica Matthews veronica.matthews@ sydney.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 09 March 2017 Accepted: 18 August 2017 Published: 04 September 2017*

#### *Citation:*

*Matthews V, Burgess CP, Connors C, Moore E, Peiris D, Scrimgeour D, Thompson SC, Larkins S and Bailie R (2017) Integrated Clinical Decision Support Systems Promote Absolute Cardiovascular Risk Assessment: An Important Primary Prevention Measure in Aboriginal and Torres Strait Islander Primary Health Care. Front. Public Health 5:233. doi: 10.3389/fpubh.2017.00233*

Background: Aboriginal and Torres Strait Islander Australians experience a greater burden of disease compared to non-Indigenous Australians. Around one-fifth of the health disparity is caused by cardiovascular disease (CVD). Despite the importance of absolute cardiovascular risk assessment (CVRA) as a screening and early intervention tool, few studies have reported its use within the Australian Indigenous primary health care (PHC) sector. This study utilizes data from a large-scale quality improvement program to examine variation in documented CVRA as a primary prevention strategy for individuals without prior CVD across four Australian jurisdictions. We also examine the proportion with elevated risk and follow-up actions recorded.

Methods: We undertook cross-sectional analysis of 2,052 client records from 97 PHC centers to assess CVRA in Indigenous adults aged ≥20 years with no recorded chronic disease diagnosis (2012–2014). Multilevel regression was used to quantify the variation in CVRA attributable to health center and client level factors. The main outcome measure was the proportion of eligible adults who had CVRA recorded. Secondary outcomes were the proportion of clients with elevated risk that had follow-up actions recorded.

Results: Approximately 23% (*n* = 478) of eligible clients had documented CVRA. Almost all assessments (99%) were conducted in the Northern Territory. Within this jurisdiction, there was wide variation between centers in the proportion of clients with documented CVRA (median 38%; range 0–86%). Regression analysis showed health center factors accounted for 48% of the variation. Centers with integrated clinical decision support systems were more likely to document CVRA (OR 21.1; 95% CI 5.4–82.4; *p* < 0.001). Eleven percent (*n* = 53) of clients were found with moderate/ high CVD risk, of whom almost one-third were under 35 years (*n* = 16). Documentation of follow-up varied with respect to the targeted risk factor. Fewer than 30% with abnormal blood lipid or glucose levels had follow-up management plans recorded.

Conclusion: There was wide variation in CVRA between jurisdictions and between PHC centers. Learnings from successful interventions to educate and support centers in CVRA provision should be shared with stakeholders more widely. Where risk has been identified, further improvement in follow-up management is required to prevent CVD onset and reduce future burden in Australia's Indigenous population.

Keywords: cardiovascular disease, risk assessment, Indigenous health, prevention, primary health care

#### INTRODUCTION

Health inequities between Aboriginal and Torres Strait Islander (respectfully referred to as Indigenous) and non-Indigenous Australians are well documented (1, 2) and are a legacy of colonization, disempowerment and ongoing racial, social and economic inequality (3). It has been estimated that continued inequality accounts for between one-third and one-half of the 10-year life expectancy gap between Indigenous and non-Indigenous people (2, 4), highlighting the importance of addressing the social determinants of health and ensuring equity of access to quality health care.

Highly preventable chronic diseases contribute most to the higher rate of poor health and premature death experienced by Aboriginal and Torres Strait Islander people. Cardiovascular disease (CVD), largely driven by the combined effect of several modifiable risk factors such as smoking and obesity, is the leading contributor accounting for one-fifth of the health gap (1). In addition to improving social and economic determinants of health, effective CVD prevention, through regular screening and early intervention, would make a significant contribution to reducing the health gap and disease burden within the Indigenous population (5).

Health promotion, prevention, and early treatment services are a key component of Australia's primary health care (PHC) system. Access to PHC for Aboriginal and Torres Strait Islander people is through community-controlled health centers, government-operated community health centers, and private general practitioners (GPs), with some variation across diverse geographies. Aboriginal and Torres Strait Islander community-controlled centers and some government centers operating in predominantly Indigenous communities offer models of comprehensive PHC providing access to doctors, nurses, allied health, social and emotional wellbeing professionals, and medical specialists. Service size, however, varies depending on remoteness, with visiting services a feature of remote locations.

A recent national initiative, "*Better cardiac care for Aboriginal and Torres Strait Islander people*," outlines priority action areas to address inequities in cardiovascular health service delivery between Indigenous and non-Indigenous people (6). Priority actions are staged across the disease continuum and include cardiovascular risk assessment (CVRA) as a key aspect of primary prevention, along with practitioner follow-up and intervention for those identified at risk, such as pharmacotherapy and ongoing culturally appropriate support to facilitate lifestyle modification (5).

Absolute CVRA is a screening and management process intended for use by PHC practitioners to calculate the probability of a cardiovascular event within 5 years, taking into account the synergistic effect of multiple risk factors that may be present (7). The risk calculator takes account of age, sex, systolic blood pressure, smoking status, levels of total and high-density lipoprotein cholesterol, and presence of diabetes (7).

Despite the importance of CVRA as a screening and early intervention tool, few studies have reported its use within the Indigenous PHC sector or in the broader Australian PHC setting. This reflects the lack of national and jurisdictional data on CVRA and PHC services in general (6). In 2012, the Northern Territory (NT) government implemented a large-scale strategy to strengthen chronic disease prevention in Indigenous communities that included regular CVRA data collection and reporting and the roll-out of an automated CVRA calculator within the electronic medical record system used by government health centers (8). In 2015, a similar calculator was introduced into the *Communicare* electronic medical record system used by many Aboriginal and Torres Strait Islander community-controlled health centers in the NT and other jurisdictions.

This study examines variation in documented CVRA for adults with no prior diagnosis of chronic disease as a primary prevention strategy in Indigenous PHC centers across four Australian jurisdictions (2012–2014). We also report on the proportion of Indigenous people found with elevated risk and the proportion that had subsequent follow-up actions documented.

#### MATERIALS AND METHODS

This is a retrospective cross-sectional study of preventive care clinical audits undertaken by 97 Indigenous PHC

**Abbreviations:** ABCD, Audit and Best Practice for Chronic Disease; CQI, continuous quality improvement; CVD, cardiovascular disease; CVRA, cardiovascular risk assessment; GP, general practitioner; nKPI, national Key Performance Indicator; NT, Northern Territory; PCV, proportional change in variance; PHC, primary health care; QLD, Queensland; SA, South Australia; WA, Western Australia.

centers participating in the Audit and Best Practice for Chronic Disease (ABCD) project. The ABCD project is a researchbased continuous quality improvement (CQI) initiative that has operated on a national scale since 2005, co-designing best practice clinical audit tools (covering different aspects of comprehensive PHC delivery) and processes with relevant stakeholders (9). The majority of participating services within the ABCD program are community-controlled or governmentoperated centers predominantly serving Indigenous communities. The preventive care CQI process was designed to enable participating health centers assess the level of adherence to best practice guidelines and assess organizational systems to support prevention and early detection of chronic disease. The audits are conducted by local staff trained in the use of ABCD tools and processes.

Clients included in a preventive care audit are those aged ≥15 years with no recorded diagnosis of diabetes, hypertension, coronary heart disease, rheumatic heart disease, or chronic kidney disease, who have been resident in the community for ≥6 months within the last year (10). Where the eligible population numbers 30 or less, the audit protocol recommends inclusion of all records. For 30 or more eligible clients, the protocol provides guidance on a sufficient number of randomly selected records to achieve 90–95% confidence of the sample representing the service population. Samples were stratified by age and gender.

Given the value of risk assessment as a primary prevention measure and high rates of premature CVD in Indigenous people, the audit tool incorporates assessment against best practice guidelines related to CVRA delivery. Current national guidelines recommend CVRA be provided to Indigenous adults aged between 35 and 74 years who are not known to have CVD or to be at clinically determined high risk (7). For NT Indigenous residents, the CVRA age criterion has been lowered to 20 years due to local prevalence of early onset CVD (11). Low, moderate, and high risks correspond to <10, 10–15, and >15% probability of a cardiovascular event within the next 5 years as determined by the type of calculator used in the assessment. As no CVRA algorithm has been validated in the Indigenous Australian population, current calculators underestimate the risk, failing to consider historical context and the consequential socioeconomic disadvantage and premature CVD prevalent within the population (12). Following the precedent set within New Zealand's CVRA guidelines for the Maori population, the NT PHC standard treatment (CARPA) manual included an upward risk adjustment of 5% on the Framingham algorithm for the NT Indigenous population (11, 13).

A subset of the preventive care audit data was used for this study to examine variation in documented CVRA as a primary prevention strategy for Indigenous adults with no previous documentation of chronic disease (**Figure 1**). We used the most recent preventive care audit from each of the 97 centers conducted between 2012 and 2014. While the audit tool includes clients with a diagnosis of dyslipidaemia, we excluded these records (*n* = 22) given it is a prominent risk factor for CVD and may influence clinical judgments with respect to a patient's

\*A preventive care audit excludes clients with a record of diabetes, hypertension, coronary heart disease, rheumatic heart disease, and chronic kidney disease.

background and calculating absolute risk (14). Almost 90% of clients with a record of dyslipidaemia did not have a record of CVRA within the last 24 months. Our "healthy cohort" criteria have therefore excluded individuals eligible for CVRA according to national guidelines, such as people with diabetes under the age of 60 years without microalbuminuria or people with stages 1 and 2 chronic kidney disease. De-identified clinic records of over 2,000 healthy Indigenous adults were included in the analysis. In addition to demographic information (age and sex), the audit recorded whether individuals received an adult health check and a CVRA within the last 24 months and the calculated CVD risk level. Other information collected included relevant risk factor documentation [yes/no for smoking status, body mass index, waist circumference, urinalysis, blood pressure, and blood glucose and lipid levels] and follow-up actions for abnormal findings. Health center factors such as location, population size, governance, and length of participation in the ABCD program were also recorded.

Our main outcome measure was the proportion of eligible clients who had documented CVRA. We used client and health center level information to determine independent factors associated with CVRA. Secondary outcomes for those identified at moderate/high risk were the proportion that had documented follow-up management plans and brief interventions. It was not possible to assess the level of pharmacotherapy intervention as this information was not captured within the audit tool.

Summary statistics was used to describe variation in CVRA across health centers and jurisdictions, the number of adults with elevated risk and documented level of follow-up. Cross-jurisdictional information has been aggregated where there were counts less than five. Given the hierarchical nature of the data (clients within health centers), multilevel mixed effects logistic regression models were used (health center variable was treated as a random effect with random intercept) to quantify the variation in CVRA attributable to health center and client level factors. Because a large majority of CVRA was recorded in NT centers, we restricted the regression analysis to this jurisdiction. As most centers were located in remote areas, we excluded location as a predictor variable. We calculated odds ratios to measure the unadjusted and adjusted associations between independent factors and CVRA (adjusting for year of audit). In a step-wise fashion, we included significant health center (Model A) then client variables (Model B) from the unadjusted analyses and measured the proportional change in variance to determine the amount of variation attributable to the different levels. Potential interactions were checked for significance. Statistical associations were considered significant if the *p*-value was <0.007 (Bonferroni correction). Analysis was completed using STATA software, version 14.

### RESULTS

The majority (92%) of PHC centers were located in remote or very remote areas, and 84% were government operated (**Table 1**). There were 2,052 eligible Indigenous clients aged between 20 and 75 years with almost equal numbers of males and females. Acute care was the primary reason for attendance for 49% of clients. There was wide variation across jurisdictions and health centers in the documentation of risk factors used to


*a Location based on the Australian Standard Geographical Classification system.*

*NT, Northern Territory; QLD, Queensland; SA, South Australia; WA, Western Australia.*

TABLE 2 | Primary health care (PHC) center documentation of risk factors used for cardiovascular risk assessment (median % and range).


*a A lipid profile includes total cholesterol, high-density lipoprotein, low-density lipoprotein, and triglycerides.*

*bMinimum level of risk factor documentation required for estimates using the Framingham algorithm.*

assess CVD risk (**Table 2**). NT centers had higher documentation of client smoking status, blood pressure, and lipid profile (essential measures for the Framingham algorithm) compared to other jurisdictions. Overall, 23% of eligible clients had CVRA documented and almost all of the assessments occurred in the NT (**Table 3**).

Due to the audit exclusion criteria and local CVRA guidelines, the NT cohort (*n* = 1,388) had a larger proportion (65%) of young adults (<35 years). Focusing on this jurisdiction, there was a clear trend of improvement in the mean proportion of PHC center clients documented as receiving CVRA; however, wide variation persisted across years (**Figure 2**). Health center factors accounted for 48% of the variation (**Table 4**, Model A). Clients were more likely to have documented CVRA if they attended government-operated centers (distinguished by the availability of automated CVRA calculators within their electronic patient information systems, Model A: OR 21.1; 95% CI 5.4–82.4; *p* < 0.001), and if they had an adult health check (Model A: OR 3.9; 95% CI 2.8–5.4; *p* < 0.001). There was no significant interaction between governance and provision of health checks. Client factors did not appreciably explain any further variation, although adults aged ≥45 years were more likely to have CVRA compared to the youngest age group (Model B: OR 2.0; 95% CI 1.3–3.2; *p* = 0.003).

Of the NT clients who received CVRA, 11% (*n* = 53; 95% CI: 8–14%) were found to be at moderate/high risk of a cardiovascular event in the next 5 years (**Table 5**). Thirty percent of adults with elevated risk were under 35 years (*n* = 16; **Table 5**) and 64% (*n* = 34) had ≥3 modifiable risk factors documented (**Table 6**). The main risk factors recorded were abnormal lipid levels (*n* = 43 at risk clients), being overweight (*n* = 38), and smoking (*n* = 33; **Table 6**). In terms of follow-up action documented, less than one-third of clients with abnormal blood lipid or glucose levels had a management plan recorded (with a scheduled repeat measurement), 82% (*n* = 27) of smokers and 71% (*n* = 27) of overweight clients had documentation of a brief intervention or referral (**Table 6**).

TABLE 3 | Documented delivery of cardiovascular risk assessment (CVRA) by primary health care (PHC) centers (mean % and total number), by jurisdiction (2012–2014).


#### DISCUSSION

This study provides original data on CVRA as an important CVD primary prevention activity for adults with no prior chronic disease diagnosis in the Aboriginal and Torres Strait Islander PHC sector. CVRA is particularly important in the generally healthy population as adults with known major chronic illnesses such as diabetes are at higher risk of CVD, regardless of the other risk measurements included in CVRA. Our finding that 23% of eligible clients had documented CVRA is lower than in similar studies reporting inadequate levels of screening (15, 16) due to our exclusion of clients with chronic disease, regardless of severity. The NT had substantially higher level documentation of CVRA than other jurisdictions and of those assessed at moderate/high risk, 30% were under the age of 35 years.

There have been similar findings within the Australian general practice sector where it has been reported that GPs: may not routinely calculate absolute risk for the general population; focus treatment on individual risk factors; and lack the data necessary for calculation of absolute risk (especially lipid screening) (17, 18). Lack of risk factor recording is also an issue in the current study, where in some jurisdictions, the minimal level of risk factor documentation in client records required for CVRA calculation (using the Framingham algorithm: smoking status, blood pressure, and lipid profile) was under 35%. Despite significant government TABLE 4 | Multilevel regression analysis—cardiovascular risk assessment (CVRA) in Northern Territory primary health care centers, 2012–2014 (*n* = 48 centers; 1,388 clients).


*PCV, proportional change in variance; significance level p* < *0.007.*

TABLE 5 | Number (%) of Northern Territory clients with documented cardiovascular risk assessment (CVRA) and risk level, by age group.


*a 95% confidence intervals calculated for population proportion of risk level estimates.*

funding for tackling smoking initiatives within the Indigenous community, there remains a high level of variation in recording of smoking status, particularly in Queensland. Participating health centers in Queensland were predominantly government-operated and not representative of the community-control sector. The latest national Key Performance Indicator (nKPI) report (December 2014) shows Aboriginal and Torres Strait Islander communitycontrolled health centers in Queensland documenting smoking status for 83% of their clientele (19).

TABLE 6 | Documentation of risk factors and follow-up interventions in Northern Territory clients with elevated risk (*n* = 53).


*a High BMI* ≥ *25; abnormal blood pressure: systolic pressure* ≥*140 mmHg and/or the diastolic pressure* ≥*90 mmHg; abnormal blood glucose:* ≥*5.5 mmol; abnormal blood lipids: low-density lipoprotein* >*2.0 mmol/L or high-density lipoprotein* <*1 mmol/L or triglycerides* >*1.5 mmol/L.*

*bFollow-up action—brief intervention or referral for relevant lifestyle modification (smoking cessation/weight management).*

*c Follow-up action—documented management plan including repeat test schedule to monitor levels and for high blood pressure and lipid readings, referral to doctor for assessment and potential medication control.*

Despite the recording of individual risk factors in the health records of many clients attending health centers in the NT, almost all CVRA (*n* = 439) occurred within the government sector, which had automated calculators available within their electronic patient information record systems. As manual calculation of CVRA can be cumbersome, this highlights the value of the automated calculator that was not available within the systems used by the community-control sector during the study period. Automation of CVRA also allowed the NT government to implement three monthly rapid CQI cycles on key performance indicators to continually identify screening and treatment gaps (8). There have also been dedicated educators and CQI facilitators coaching local teams in CVRA and assisting with assessments and identification of clients for recall while located off-site from the PHC centers. Sustained implementation of these higher level system supports differentiates the NT from other jurisdictions where large-scale CQI auditing for preventive care has taken place without improvement in CVRA. Other targeted interventions incorporating electronic decision support tools and CQI processes have also led to increased rates of CVRA coverage across PHC centre service populations in urban, rural and remote locations in Australia and New Zealand (20, 21). Learnings from these successful interventions have the potential to improve screening rates in other areas, and it is likely that coverage rates improved further beyond this study period due to the introduction of the automated calculator within communitycontrol PHC centers.

With the majority of clients in this study presenting for acute care, time and PHC center capacity may be a barrier to investigating CVD precursors in people with no diagnosis of a chronic condition (22). There was an association between CVRA and adult health checks suggesting that the checks are an important initiator of CVRA. Through a rebate system, the Australian Government has encouraged uptake of comprehensive health checks for Indigenous people that includes examination of physical, psychological, and social well-being [Medicare Benefits Schedule (MBS) item 715]. The use of health checks has increased over time and as at December 2014, 46% of regular Indigenous clients aged over 25 years in the NT had a health check within the last 24 months (44% nationally) (19). However, while the health check collects risk factor and biomedical information used to calculate cardiovascular risk, CVRA is not directly specified within the MBS 715 item descriptor. To more immediately address the CVD burden in the Indigenous population, a standalone rebate item for CVRA has been recommended to promote uptake rates beyond that of general health checks (8).

We found a lower proportion of Indigenous people assessed as having moderate/high CVD risk in comparison to that reported for the NT government sector in 2014 (8). This may be partly explained by differences in the study cohorts, with the preventive care audit excluding clients with a record of chronic disease. Based on this criteria, 65% of the NT clients in this study were under the age of 35 years and 15% were 45 years or older. Almost one-third of clients with moderate/high CVD risk were under 35 years, emphasizing the importance of targeted screening for adults younger than the current CVRA national guideline for Indigenous people.

Despite improvements in risk screening, evidence to practice gaps persist regarding follow-up treatment to reduce risk once identified (8, 20). Lack of follow-up has been attributed previously to barriers at various levels of the health system such as lack of time, staff capacity, availability of culturally appropriate referral services, and that practice incentives focus on assessment rather than follow-up (23). The lack of documentation of management plans for abnormal findings demonstrates the challenge of maintaining client continuity of care, particularly in remote contexts. Building client/health provider relationships is an important enabler for sustained engagement required to effect lifestyle change, where responsibility for care is shared between client and practitioner and where health centers work beyond traditional roles to influence social and economic determinants for individuals and their communities (24, 25). Lifestyle modifications such as improving nutrition and exercise and lowering rates of smoking will reduce the current and future burden of disease, not only with respect to CVD but other chronic conditions contributing to the health gap between Aboriginal and Torres Strait Islander and other Australians.

Large-scale examination of CVRA to Indigenous people has been made possible by the centers enrolled in the ABCD CQI program. However, the voluntary nature of their participation limits the generalizability of study findings, with the majority of centers from remote areas in the NT and the government sector in Queensland. In addition, as data are collected from client records, delivery of CVRA and associated follow-up for clients at risk may be underestimated due to poor documentation. However, accurate recording is an essential aspect of care quality and should be addressed as part of CQI processes.

The "*Better cardiac care for Aboriginal and Torres Strait Islander people*" strategy has increased attention on capturing CVRA data on a broad scale, initiating its introduction to the nKPI dataset (26) (although the age criteria begins at 35 years, some 10 years older than clients documented with elevated risk in the NT). A routine data source allows the assessment of "unwarranted" variation in the delivery of key service items, a necessary first step to examine potential health system factors that when leveraged, may enhance consistent, appropriate care on a broad-scale (27, 28). We demonstrate the value of CQI processes in systematically capturing, reporting and reviewing data on the variation in CVRA and follow-up. Systematic collection of data for primary prevention also emphasizes the need for a specific calculator to accurately predict CVD risk for this population.

### CONCLUSION

Our findings show that there is substantial room for improvement in CVRA and follow-up as an important primary prevention strategy within the Aboriginal and Torres Strait Islander population. Systematic CVRA provides an opportunity for the PHC sector to curb rates of early CVD onset as emphasized by the young "healthy" cohort of Aboriginal and Torres Strait Islander adults documented with elevated risk. Shared learnings from successful system interventions as demonstrated in the NT (integration of automated calculators, CQI processes, and dedicated staff support) have the potential to reduce unwarranted variation and increase rates of screening on a broad scale, enabling early intervention where necessary. Further work is required on improving follow-up of clients identified at risk and facilitating supportive health center–client/community relationships.

### ETHICS STATEMENT

Ethics approval was obtained from human research ethics committees (HRECs) in each jurisdiction: Northern Territory HREC-EC00153 & HREC-12-53; New South Wales HREC/11/ GWAHS/23; Queensland HREC/11/QTDD/47; South Australia Aboriginal Health Research Ethics Committee 04-10-319; Western Australia Curtin University HR140/2008; WA Country Health Services 2011/27; WA Aboriginal Health Information and Ethics Committee 111-8/05; and University of Western Australia RA/4/1/5051.

### AUTHOR CONTRIBUTIONS

RB and VM conceived and designed the study. VM analyzed the data, drafted, and revised the paper with important intellectual input from all authors (PB, CC, EM, DP, DS, ST, SL, and RB). RB played a lead role in design and development of the ABCD

#### REFERENCES


research project. CC, EM, DS, ST, and SL were co-investigators on the ABCD project. All authors read and approved the final manuscript.

#### ACKNOWLEDGMENTS

The development of this manuscript would not have been possible without the active support, enthusiasm, and commitment of staff in participating primary health care centers and members of the ABCD National Research Partnership and the Centre for Research Excellence in Integrated Quality Improvement. We would like to thank Jodie Bailie and Alison Laycock for their revision of the manuscript.

### FUNDING

The National Health and Medical Research Council has funded the ABCD National Research Partnership Project (# 545267) and the Centre for Research Excellence in Integrated Quality Improvement (# 1078927). In-kind and financial support has been provided by the Lowitja Institute and a range of communitycontrolled and government health agencies.


a cross-sectional analysis. *Front Public Health* (2016) 4:34. doi:10.3389/ fpubh.2016.00034


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Matthews, Burgess, Connors, Moore, Peiris, Scrimgeour, Thompson, Larkins and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# System-Level Action Required for Wide-Scale Improvement in Quality of Primary Health Care: Synthesis of Feedback from an Interactive Process to Promote Dissemination and Use of Aggregated Quality of Care Data

#### *Jodie Bailie\*, Alison Laycock, Veronica Matthews and Ross Bailie*

*Menzies School of Health Research, Charles Darwin University, Casuarina, NT, Australia*

#### *Edited by:*

*Edward Broughton, University Research Co., LLC, USA*

#### *Reviewed by:*

*Jason Richter, Army-Baylor University, USA Arnold Bosman, European Centre for Disease Prevention and Control (ECDC), Sweden*

#### *\*Correspondence:*

*Jodie Bailie jodie.bailie@menzies.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 07 December 2015 Accepted: 20 April 2016 Published: 04 May 2016*

#### *Citation:*

*Bailie J, Laycock A, Matthews V and Bailie R (2016) System-Level Action Required for Wide-Scale Improvement in Quality of Primary Health Care: Synthesis of Feedback from an Interactive Process to Promote Dissemination and Use of Aggregated Quality of Care Data. Front. Public Health 4:86. doi: 10.3389/fpubh.2016.00086*

Introduction: There is an enduring gap between recommended practice and care that is actually delivered; and there is wide variation between primary health care (PHC) centers in delivery of care. Where aspects of care are not being done well across a range of PHC centers, this is likely due to inadequacies in the broader system. This paper aims to describe stakeholders' perceptions of the barriers and enablers to addressing gaps in Australian Aboriginal and Torres Strait Islander chronic illness care and child health, and to identify key drivers for improvement.

Methods: This paper draws on data collected as part of a large-scale continuous quality improvement project in Australian Indigenous PHC settings. We undertook a qualitative assessment of stakeholder feedback on the main barriers and enablers to addressing gaps in care for Aboriginal and Torres Strait Islander children and in chronic illness care. Themes on barriers and enablers were further analyzed to develop a "driver diagram," an improvement tool used to locate barriers and enablers within causal pathways (as primary and secondary drivers), enabling them to be targeted by tailored interventions.

Results: We identified 5 primary drivers and 11 secondary drivers of high-quality care, and associated strategies that have potential for wide-scale implementation to address barriers and enablers for improving care. Perceived barriers to addressing gaps in care included both health system and staff attributes. Primary drivers were: staff capability to deliver high-quality care; availability and use of clinical information systems and decision support tools; embedding of quality improvement processes and data-driven decision-making; appropriate and effective recruitment and retention of staff; and community capacity, engagement and mobilization for health. Suggested strategies included

**Abbreviations:** ABCD, audit and best practice for chronic disease; AHWs, Aboriginal and Torres Strait Islander health workers/practitioners; CQI, continuous quality improvement; ESP, engaging stakeholders in identifying priority evidence– practice gaps and strategies for improvement in primary health care; PHC, primary health care.

mechanisms for increasing clinical supervision and support, staff retention, reorientation of service delivery, use of information systems and community health literacy.

Conclusion: The findings identify areas of focus for development of barrier-driven, tailored interventions to improve health outcomes. They reinforce the importance of system-level action to improve health center performance and health outcomes, and of developing strategies to address system-wide challenges that can be adapted to local contexts.

Keywords: health system and staff attributes, primary care, Aboriginal and Torres Strait Islander health, tailored interventions, quality improvement, driver diagram, aggregated quality of care data

### INTRODUCTION

Despite efforts to promote best-practice clinical guideline use, adherence to guidelines remains variable between health centers and between health professionals (1–3). Interventions designed to address known barriers to care and based on evidence are more likely to produce the desired change in clinical care (4–7). Despite this knowledge, few interventions implemented are based on theory or a systematic assessment of barriers (3, 8, 9). Methods to identify barriers and to tailor interventions to address barriers need further development (4, 8, 10). In the context of major disparities in health outcomes between population groups – as between Aboriginal and Torres Strait Islander people and non-Indigenous Australians (11) – the importance of developing tailored interventions is even greater.

Large-scale improvement in the delivery of primary health care (PHC) requires change at multiple levels of the health system, not only at the local health center level (12, 13). The health system can be understood as consisting of "all organizations, people, and actions whose primary interest is to promote, restore, or maintain health" (14). Where aspects of care are not being done well across a range of PHC centers, this is likely due to inadequacies in the broader system. Ferlie and Shortell describe the health system as having four levels and argue that change is required at all four levels – those of the individual, the group or team, the overall organization, and the larger environment in which organizations are embedded – in order to improve care quality and outcomes (13). Taking a system-wide approach to continuous quality improvement (CQI) is associated with achieving large-scale improvements in health outcomes (15).

Gaps in the provision of care that may escape notice at a local level – for example, because of small numbers – can become noticeable when data are aggregated (16). In the context of limited availability of data on PHC system performance, we propose that aggregated CQI data can provide a useful source of evidence for identifying common and important gaps in care across health centers and for developing and implementing system-wide improvement efforts. A recent systematic review identified the need to seek perspectives from a range of stakeholders, such as policy and decision makers and service providers, on the health center and system attributes that lead to improved Aboriginal and Torres Strait Islander PHC outcomes (17). Reflections from a range of stakeholders in the health system can provide important insights on barriers, enablers, and strategies for improvement (17–20). The aim of this paper is, therefore, to describe stakeholders' perceptions of the main barriers and enablers to addressing identified priority gaps in chronic illness care and child health, and to identify drivers for improvement in Aboriginal and Torres Strait Islander PHC as reflected in these stakeholder perceptions.

We have developed and implemented an active dissemination strategy – "*Engaging Stakeholders in Identifying Priority Evidence-Practice Gaps and Strategies for Improvement in PHC*" *(the ESP Project)* – that aims to promote wide-scale improvement in quality of care by applying a system-wide approach to CQI (21–23). The ESP Project is designed to engage a wide range of stakeholders in a theory-driven approach to interpret aggregated CQI data on health system performance and to reflect on barriers, enablers, and strategies for improvement (see **Box 1**). The rationale for the ESP Project is that involving diverse stakeholders in a phased approach of using aggregated CQI data should stimulate discussion and information sharing, and enhance ownership of the development of interventions to address system gaps. The theoretical and conceptual base for the ESP Project is described in more detail in a separate publication (21). The focus of this paper is on addressing the specific aim as described above.

#### BOX 1 | Australian context and the ABCD National Research Partnership.

In Australia, a high-income country with a universal health insurance scheme (Medicare), Aboriginal and Torres Strait Islander populations experience inequitable access to health care and poorer health outcomes than for non-Indigenous Australians (11). Access to PHC for Aboriginal and Torres Strait Islander people is through private general practice, government managed and Aboriginal community-controlled health services.

Strengthening PHC is critical to closing the gap in health inequalities between Aboriginal and Torres Strait Islander people and other Australians. A wide-scale CQI project, the Audit and Best Practice in Chronic Disease (ABCD) National Research Partnership (2010–2014), has employed a systems approach to improving Aboriginal and Torres Strait Islander PHC delivery (24–27). PHC centers have used evidence-based best-practice clinical audit and system assessment tools to assess and reflect on system performance, typically on an annual basis. Available CQI tools cover various aspects of PHC (e.g., chronic illness care, child health, preventive, mental, and maternal health care). Audit tools are developed through a process of expert consensus that draws upon current evidence-based care guidelines [such as CARPA (28) and Queensland Chronic Disease Guidelines (29)] (30). Over 175 Aboriginal and Torres Strait Islander PHC centers using these CQI processes voluntarily provided de-identified CQI audit data to the ABCD National Research Partnership for analysis. The strongest engagement has been from health centers in the Australian jurisdictions of the Northern Territory and Queensland.

health.


### MATERIALS AND METHODS

The theoretical basis for the phased approach for the ESP Project draws on the methods outlined by French et al. (5) to develop interventions based on evidence and identified barriers and enablers, as outlined below and in **Figure 1** (21). We ran the ESP process separately for child health care and then for chronic illness care. We briefly describe each of the phases of the ESP project below by way of background. For the purpose of addressing the aim of this paper, we focus primarily on the analysis of the qualitative data derived from phase 2 of the ESP project for both child health and chronic illness care. Detailed reports on the process and findings from each of the phases of the ESP project for child health (22), chronic illness care (23), and the clinical audit tools and the audit process (25) have been published previously.

#### ESP Project Phases

#### Phase 1 – Identifying the Priority Evidence-Practice Gaps

In Phase 1, a consensus-driven approach was used to identify priority evidence-practice gaps for child health and for chronic illness care. We prepared separate reports for these two areas of care, using the most recent clinical audit data (2012–2013) to describe current delivery of guidelinescheduled care across health centers, and distributed them to a range of stakeholders.

For all phases, we aimed to include individuals and organizations representing diverse roles and geographical areas, identified as having an interest and experience in Aboriginal and Torres Strait Islander health delivery, management, policy, and research. They included health practitioners (e.g., doctors, nurses, allied health professionals, Aboriginal Health Workers), managers and policy-makers at various health system levels, researchers, and staff of peak bodies and support organizations that represent the interests of community-controlled health services and Aboriginal and Torres Strait Islander communities.

Aggregated CQI data were available from 123 health centers (6,523 patient records; 90 system assessments) for chronic illness care (23) and from 94 health centers (4,011 patient records; 62 systems assessments) for child health care (22). Preliminary evidence-practice gaps were determined with the assistance of clinical experts by identifying (a) areas of clinical care that were being delivered or recorded at a relatively low level by services; (b) aspects of care where there was more general wide variation in recorded delivery of care; and (c) components of the PHC center systems that were relatively poorly developed (23). Through the survey, we asked respondents to rate the relative importance of each preliminary priority evidence-practice gap identified in the report on a scale of 1–10; and the extent to which the listed priorities resonated with their experience. Open-ended questions were used to elicit explanatory information on reasons for scores and further comments.

#### Phase 2 – Identifying the Barriers and Enablers to Addressing the Identified Evidence-Practice Gaps

In Phase 2, trend data were presented for each of the priority evidence-practice gaps identified in Phase 1 by (a) calendar years and (b) audit cycles, to show trends in variation over time and across CQI cycles. We aggregated clinical audit and systems assessment data on adherence to best practice guidelines from 160 health centers (17,879 patient records; 390 systems assessments), over the period 2005–2013 for chronic illness care; and from 132 health centers (10,405 patient records; 265 systems assessments), 2007–2013 for child health.

Through the survey, we encouraged stakeholders to reflect on the influences underlying the data trends, and on their experience in PHC, to identify barriers and enablers to improving care. The survey tool for this phase drew on international and Australian literature on health system and staff attributes (or domains) relevant to implementation of change interventions and behavior change of health care professionals (5–7, 27, 31, 32) (Table S1 in Supplementary Material lists the attributes).

Respondents were asked to rate each attribute identified according to its relative importance in improving evidence-practice gaps on a five-point Likert scale of strongly agree, agree, disagree, strongly disagree, and "do not know/cannot say." Respondents were asked to relate their responses to providing best practice care as relevant to the priority evidence-practice gaps across the PHC system for Aboriginal and Torres Strait Islander people, rather than for any specific health center or service. Open-ended questions were also used to elicit explanatory data from stakeholders on aspects of the health system or health center environment, or staff attributes, which pose significant enablers or barriers to providing best practice care and strategies for improvement.

#### Phase 3 – Identifying the Strategies for Improvement to Address the Identified Evidence-Practice Gaps

In Phase 3, we presented a report on the barriers and enablers to addressing gaps in care identified in the previous phase. We asked respondents to comment on whether the report provided a fair reflection of the main barriers and enablers to improvement. Respondents were asked to suggest new or existing strategies to address the most common barriers and enablers.

#### Draft Final Report

In this step, we presented a draft final report on the whole process. We asked stakeholders to confirm that we had accurately reflected their feedback about strategies to address the gaps identified and whether they wished to provide additional comments that could be used to finalize the project report.

Respondents could enter data as individuals or on behalf of a group in order to encourage engagement of people who were less likely to provide individual responses. We used an iterative process to develop and refine reports through the project phases, making adjustments to content and presentation over time in response to stakeholder feedback.

Circulation lists for the ESP reports and surveys were based on networks developed over several years of the ABCD National Research Partnership. Invitations to participate were emailed by the project leader. We used a snowballing distribution technique, encouraging people to forward reports and surveys through their professional networks.

The child health ESP phases were undertaken in early 2014 and chronic illness care ESP phases in late 2014. A reminder email was sent 1 week before the closing date of surveys.

#### Data Synthesis for Identifying Common Barriers and Enablers, and Drivers for Improvement

Analysis of the difference in quantitative responses on barriers and enablers for individuals and groups showed similar response patterns. Individual and group responses were, therefore, analyzed together. Our primary interest was in the qualitative nature of responses.

For the analysis, we performed a deductive thematic analysis as described by Miles and Huberman (33). A three-step iterative process was used to identify, analyze, and describe patterns in the data, as follows. (1) The lead author (Jodie Bailie) undertook multiple readings of the interview responses. Her initial assessment of emerging themes was refined in consultation with the other authors, all of whom had been involved in the design and implementation of the ESP project and were, thus, familiar with the data. (2) Interview data were then coded systematically by the lead author, using an organizing matrix of "health system attributes" or "staff attributes" to identify common themes relating to barriers and enablers to improvement in care. (3) Three authors (Jodie Bailie, Alison Laycock, and Ross Bailie) then reviewed and conferred on the identified themes and the lead author revised the themes in light of this discussion.

We then drew on the thematic analysis to produce a "driver diagram," an improvement tool that is used to locate barriers and enablers within causal pathways (as primary and secondary drivers), enabling them to be targeted by tailored interventions (34, 35). The process for developing the driver diagram involved the lead author further analyzing the themes on the barriers and enablers to identify system issues that could contribute to improving delivery of care ("primary drivers") and, second, issues that could impact on these primary drivers ("secondary drivers"). Enablers and barriers were viewed as "drivers" if they represented features of the system that enabled or constrained care quality. The strategies for improvement identified by stakeholders in Phase 3 were organized according to the drivers with which they were most clearly aligned. The driver diagram then went through several iterations of review and refinement involving all authors. The refined driver diagram was subsequently presented to a group of 30 experienced practitioners and researchers working in Aboriginal and Torres Strait Islander PHC and CQI for critical feedback based on their collective knowledge and diverse roles. This feedback was incorporated and the diagram further revised to reflect this input.

#### Ethics Approval

Ethical approval for the ABCD National Research Partnership was obtained from research ethics committees in each relevant Australian jurisdiction – Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research (HREC EC00153); Central Australian Human Research Ethics Committee (HREC-12-53); New South Wales Greater Western Area Health Service Human Research Committee (HREC/11/GWAHS/23); Queensland Human Research Ethics Committee Darling Downs Health Services District (HREC/11/ QTDD/47); South Australian Aboriginal Health Research Ethics Committee (04-10-319); Curtin University Human Research Ethics Committee (HR140/2008); Western Australian Country Health Services Research Ethics Committee (2011/27); Western Australia Aboriginal Health Information and Ethics Committee (111-8/05); and University of Western Australia Human Research Ethics Committee (RA/4/1/5051).

### RESULTS

Responses for each phase and for each area of care are detailed in **Table 1**. For child health, there were 10–26 individual responses and 1–3 group responses in the various phases. For chronic illness care, there were 17–45 individual responses and 3–10 group responses in different phases. For chronic illness care, 10 groups who provided feedback comprised more than 20 people.

#### Phase 1 – Identifying priority evidence-practice gaps Phase 2 – Barriers and enablers to addressing gaps in care Phase 3 – Suggested strategies for improvement Phase 4 – Review of draft final report Child health Chronic illness care Child health Chronic illness care Child health Chronic illness care Child health Chronic illness care Ind. Gr. Ind. Gr. Ind. Gr. Ind. Gr. Ind. Gr. Ind. Gr. Ind. Gr. Ind. Gr. No. of responses 17 3 45 10 26 3 11 4 11 1 15 3 10 3 17 6 No. of attendees per group Less than 5 – 1 2 1 – – 3 3 5–10 3 2 – 1 1 1 3 11–20 – – 1 – – 1 – More than 20 – 7 – 2 – 1 – Jurisdictions of interest for respondentsa National 2 5 2 1 1 2 2 4 New South Wales – 1 2 1 – – 1 – Queensland 11 22 6 3 4 4 4 6 Northern Territory 6 25 16 8 5 8 5 12 South Australia 2 5 3 1 2 4 2 6 Western Australia – 0 1 – – – – 1 Rurality of population group to which responses relatea Urban 8 15 6 4 3 9 4 8 Regional 9 25 14 5 4 9 6 7 Remote 15 41 24 13 10 13 11 17 Position types Nurse 4 14 5 6 3 5 4 7 Middle Manager 1 7 1 2 2 1 2 3 Doctor 1 11 1 4 1 5 3 3 Public Health Physician 1 8 – 3 – 3 1 1 Other Medical Specialist 4 4 2 – 1 1 1 1 Senior Management/ executive – 7 – 2 – 5 1 4 CQI facilitator 2 12 4 2 3 3 3 4 Board member – 2 – – – 1 1 – Policy officer 1 1 3 3 1 2 – 3 Aboriginal Health Worker – 5 1 1 1 1 2 2 Research/Academic – 9 7 2 2 2 4 2 Other 6 12 5 1 1 4 3 1 Organization types Community-controlled health center 1 7 4 2 6 2 6 5 Community-controlled peak body 3 3 2 1 1 3 1 1 Government health center 2 13 2 1 2 3 2 6 Government health department 2 16 11 7 1 5 1 11 Medicare local 1 2 1 – 1 – 1 – General practice – 3 – 1 – – – 2 University/Research 2 8 7 2 5 3 5 2

#### TABLE 1 | Survey respondents for child health and chronic illness care ESP project phases.

*a Numbers may not tally with total number of respondents as respondents were able to select multiple answers.*

*Ind, individual; Gr, Group; ESP, Engaging stakeholders in identifying priority evidence-practice gaps and strategies for improvement in primary health care.*

organization

Other 7 16 5 1 2 4 2 –

The majority of respondents indicated that they were responding from the Northern Territory or Queensland with a remote and/or regional health-care perspective. Respondents included nurses, CQI facilitators, policy officers, doctors, researchers/ academics, medical specialists, Aboriginal Health Workers, and senior and middle managers (**Table 1**).

Seven priority evidence-practice gaps were identified for chronic illness care and five for child health (**Table 2**). Common gaps across these two areas of care were related to follow-up of abnormal findings; recording of advice on risks to health; and systems for links between health centers and communities.

Common barriers to addressing care priorities for child health and chronic illness care are outlined below according to the main themes that emerged from the data. In general, the enablers were the inverse of barriers, and we have described each theme according to the balance of stakeholder perceptions as positive or negative. These main themes and illustrative quotes are presented in **Table 3**.

For both chronic illness care and child health care, respondents felt that health center and system attributes were of greater or equal importance compared to staff attributes in improving quality of care. In addition to the responses to a direct question on this issue, stakeholders' perceptions of the relative importance of health center and system attributes were reflected in the qualitative comments. Health system barriers, such as staff shortages,

#### TABLE 2 | Priority evidence-practice gaps identified in child health care and chronic illness care (22, 23).


were perceived to impact on staff attributes, such as emotion and intentions.

### Health Center and System Attributes Staffing, Recruitment, and Retention

Respondents considered lack of systems to ensure PHC staff have support from experienced staff to be a significant barrier to improvement, especially when health centers are commonly affected by turnover and shortages of staff. Inadequate staffing levels overall, in particular a lack of Aboriginal and Torres Strait Islander-specific positions, were seen to impact on the ability to address gaps in care. Poor links between health centers and communities were viewed as a barrier to care, connected to the lack of Aboriginal and Torres Strait Islander staff to fulfill this vital role. This theme is discussed in more detail below.

Staff in remote health centers were perceived to be skilled in acute care, but not necessarily in the specific skills required for providing chronic illness or child health care. Furthermore, the demands of acute care impact on the ability of staff to provide chronic illness care. For example, staffing shortages left staff feeling unable to offer self-management support to patients and experiencing frustration because of lack of time. High staff turnover was perceived to impact negatively on the ability to implement new programs, build linkages with communities, and increase demands for health centers to offer orientation and training for new staff. Staffing issues were pervasive and accounted for the majority of respondent comments, impacting on staff morale and optimism.

#### Training and Skill Development

Skill areas in need of development included: use of clinical information systems, principles of self-management, principles of patient-centered care (especially for chronic illness care), and immunization delivery (for child health care). The limited capability of health teams to use CQI tools and processes was highlighted, with management widely perceived as being inadequately trained to support effective use of CQI tools and resources. In remote areas, nursing staff were reported to be trained in and focused on acute care rather than preventive care – this was perceived as a barrier to care in chronic illness and child health. While respondents viewed access to training, including self-directed learning packages, as generally good, high workload and time pressures on staff prevented wide uptake.

#### Decision Support and Clinical Information Systems

Respondents perceived clinical information systems as having the functionality to support provision of best practice care, and access to best practice guidelines and other decision support resources as good. They highlighted the challenge of high staff turnover and the need to constantly orient new staff to use clinical information systems effectively. There were calls for an integrated health record, accessible by multiple providers, to address challenges of providing care to populations that commonly move between communities. This population movement is particularly high in remote settings. Poor internet access for remote health centers was seen to impact on ability to use decision support guidelines and information systems, with slow systems hampering efforts to document discussions with patients.

and organizational support for quality improvement systems


#### TABLE 3 | Health center/system and staff attributes that were identified as predominant barriers with example quotes.

*(Continued)*

#### TABLE 3 | Continued


*a The term "clinic" is commonly used to refer to a health center. CQI, continuous quality improvement; PHC, primary health care.*

#### Quality Improvement

Good quality improvement tools and processes were perceived as being available – particularly for chronic illness care – however, limited skill levels of managers in CQI, and in data management more generally, were seen as a barrier to supporting effective use of CQI tools and resources. CQI was reportedly not viewed as a core component of work by many staff, and was, therefore, often not prioritized. This was linked by respondents to the high turnover of staff and the need to continually train and support new staff. The identified lack of capability to use clinical information systems was seen to inhibit the effective electronic extraction and use of data for CQI purposes.

#### Community Capacity, Engagement, and Mobilization

Limitations in this area were widely regarded as barriers to improving care, with inadequate systems for increasing the expectations of community members with regard to best practice care, strengthening community leadership for health, or enhancing health literacy. Capability to build and support PHC staff to develop links to work in partnership with communities was seen as lacking, with high staff turnover impacting on the ability to build and maintain relationships between PHC staff and communities. This barrier could be linked with the strong agreement about the need for more Aboriginal and Torres Strait Islanderspecific positions within health centers.

#### Leadership and Teamwork

A lack of effective leadership (often due to high staff turnover) was seen to hamper efforts to implement systems to support best practice approaches, implement CQI, and use data to inform decision-making. Inadequate use of clinical information systems by staff was seen, in turn, to hamper efforts to advance teambased approaches to care.

### Staff Attributes

In relation to assessment of staff attributes, respondents indicated that despite the good intentions of staff to provide best practice care, other episodic care arrangements and workforce shortages were seen to impact on the delivery of care. Thus, the extent to which staff attributes were impacting on quality of care appeared to be at least partly symptomatic of system-level factors. There were mixed responses in relation to the domain of emotion. Generally, respondents reported that staff enjoy their normal day-to-day activities, but they were also commonly perceived to feel unhappy, anxious, or depressed about their work.

### Drivers for Delivery of High-Quality Care

In our analysis of enablers and barriers, we identified five primary drivers that have the potential for direct impact on identified priority evidence-practice gaps in child health and chronic illness care. They were: staff capability to deliver high-quality care; availability and use of clinical information systems and decision support tools; embedding of quality improvement processes/systems and data-driven decision-making; appropriate and effective recruitment and retention of staff; and community capacity, engagement, and mobilization for health. Eleven secondary drivers were identified – these are health center system or staff attributes that have a direct impact on each of the primary drivers. There was not a one-to-one relationship between the main barriers and enablers and the primary drivers, with more than one theme being relevant to a number of the primary drivers. For example, the themes "training and skill-development," "leadership and teamwork," and themes related to staff attributes all have relevance to the primary driver of "staff capability to deliver high-quality care." The drivers are presented in a Driver Diagram (**Figure 2**), with associated strategies. The strategies reflect actions identified by ESP respondents as having potential to influence the effect of the "drivers" in a positive direction.

### DISCUSSION

Drawing on data from stakeholder perspectives on barriers, enablers, and strategies for improvement derived from a largescale CQI program, we have developed a framework of 5 primary drivers and 11 secondary drivers of high-quality care. The framework offers opportunity for policy-makers to develop multi-level, barrier-driven, tailored interventions to improve delivery of care for a population that experiences marked inequities in access to health services and health outcomes.

The perception that the major barriers and enablers to improving quality of care for Aboriginal and Torres Strait Islander people relate to system design attributes, workforce, provider and patient relationships, clinical care pathways, and access is consistent with international and national literature (36). While some researchers highlight the difficulty of identifying the most important barriers or enablers for change (4), health systems strengthening frameworks have identified resources (human resources, infrastructure, financing, and knowledge); service delivery; and governance and leadership as being the core axes of the system (12). Recent work on improving Aboriginal and Torres Strait Islander PHC highlights the need to provide patient-centered care – care that is culturally safe and built on the establishment of long-term relationships (36, 37), rather than solely on implementation of evidence-based guidelines. Similarly, Van Olmen et al. (12) identify values and principles as fundamental to strengthening health systems. The feedback through the ESP process confirms that people working in the Aboriginal and Torres Strait Islander PHC sector recognize this need, and the importance of Aboriginal and Torres Strait Islander staff in meeting it, but are hampered by multi-level system constraints and practicalities of time, workload, and available workforce. The drivers and suggested strategies reflected in the driver diagram identify areas of opportunity for those developing PHC policy and interventions to develop barrier-driven, tailored interventions to improve health outcomes for Aboriginal and Torres Strait Islander people. Our hope is that the suggested strategies will spark conversations and ideas on how to address barriers to care, and that these will lead to wide-scale action for improving care.

Continuous quality improvement data from the ABCD National Research Partnership provide the most comprehensive picture available to date on the quality of PHC care received by Aboriginal and Torres Strait Islander people (26). A strength of the ESP project is that it is informed by evidence (i.e., context-specific aggregated CQI data) to identify priority evidence-practice gaps in child health and chronic illness care.

Other strengths and limitations arise from the open process used to engage stakeholders. Individuals and groups could choose to participate in any or all ESP project phases. The ESP project has relied, in part, on stakeholders sending reports to others. Thus, a limitation of the study is that it has not been possible to accurately measure reach or response rates. However, the feedback gathered through the ESP process reflects the experience and tacit knowledge from a diverse range of stakeholders, and their perceptions of the barriers, enablers, and strategies. The geographic spread of respondents, although broad, primarily represents the Northern Territory and Queensland and a remote/regional context – this may limit the generalizability of the findings. However, according to a large majority of respondents from the other jurisdictions, the priority evidence-practice gaps appear reasonably generalizable to a national level, mitigating this potential limitation. While some respondents may have had more limited experience across the PHC system than others, collectively the respondents represent perspectives from a wide range of organizations and geographic locations (as reflected in **Table 1**).

In recognition that barriers exist across multiple levels of the health sector, we encouraged reflection on the broader health center and system determinants of performance (27, 36). These higher system-level influences on quality of care have not been validated in the same way as questions about individual attributes based on the theoretical domains framework (5).

The collated views and ideas provide a basis for stakeholders to continue to work collaboratively across regions and jurisdictions

FIGURE 2 | Driver diagram showing survey responses organized as primary and secondary drivers of high-quality child health and chronic illness care and suggested strategies for addressing identified evidence-practice gaps.

to share knowledge and experience and develop strategies to address these known barriers and enablers. While it is widely recognized that strategies to improve the quality of care need to take account of local context, these findings reinforce the importance of multi-level action across the system to improve health center performance and Aboriginal and Torres Strait Islander health outcomes.

### AUTHOR CONTRIBUTIONS

JB conceived the manuscript, performed the synthesis of stakeholder feedback, and drafted the manuscript. AL contributed to the synthesis of feedback and drafting of the manuscript. VM contributed to conception of ESP Project, and led the quantitative analysis of the ABCD data and the development of the ESP reports on child health and chronic illness care. RB is the leader of the ABCD National Research Partnership of which the ESP Project is an important component. He contributed to synthesis of feedback, conceptualization, and revision of the manuscript. All authors read and approved the final version of the manuscript.

#### REFERENCES


### ACKNOWLEDGMENTS

The development of this manuscript would not have been possible without the active support, enthusiasm, and commitment of members of the ABCD National Research Partnership, especially the participating primary health care centers that have provided de-identified data. The ABCD National Research Partnership Project is supported by funding from the National Health and Medical Research Council (545267) and the Lowitja Institute, and by in-kind and financial support from a range of Community Controlled and Government agencies. Ross Bailie's work is supported by an Australian Research Council Future Fellowship (100100087). We thank Professor Andy Carsons-Stevens (Cardiff University, Wales) and Associate Professor Christine Phillips (Australian National University, Canberra) for their comments on an earlier draft.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fpubh.2016.00086


School of Health Research (2014). Available from: http://www.menzies.edu. au/icms\_docs/211565\_ESP\_Project\_FINAL\_REPORT\_Primary\_health\_ care\_for\_Aboriginal\_and\_Torres\_Strait\_Islander\_children.pdf


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Bailie, Laycock, Matthews and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Wide Variation in Absolute Cardiovascular Risk Assessment in Aboriginal and Torres Strait Islander People with Type 2 Diabetes

*Bhakti R. Vasant1 , Veronica Matthews1 , Christopher P. Burgess2 , Christine M. Connors2 and Ross S. Bailie1 \**

*1Menzies School of Health Research (Brisbane Office), Brisbane, QLD, Australia, 2 Top End Health Services, Primary Health Care Branch, Northern Territory Government, Casuarina, NT, Australia*

Background: Absolute cardiovascular risk assessment (CVRA) is based on the combined effects of multiple risk factors and can identify asymptomatic individuals at high risk of cardiovascular disease (CVD). Aboriginal and Torres Strait Islander people, the Indigenous people of Australia, are disproportionately affected by CVD and diabetes. Our study aimed to investigate variations in the use of absolute CVRA in patients with diabetes at Indigenous community healthcare centers and to identify patient and health center characteristics that may contribute to this variation.

#### *Edited by:*

*Edward Broughton, University Research Company, USA*

#### *Reviewed by:*

*Susan Mello, Northeastern University, USA Donna Bing-Ying Mak, University of Notre Dame, Australia*

> *\*Correspondence: Ross S. Bailie ross.bailie@menzies.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 08 January 2016 Accepted: 22 February 2016 Published: 08 March 2016*

#### *Citation:*

*Vasant BR, Matthews V, Burgess CP, Connors CM and Bailie RS (2016) Wide Variation in Absolute Cardiovascular Risk Assessment in Aboriginal and Torres Strait Islander People with Type 2 Diabetes. Front. Public Health 4:37. doi: 10.3389/fpubh.2016.00037*

Methods: Audits of clinical records of 1,728 patients with a known diagnosis of diabetes across 121 health centers in four Australian States/Territories [Northern Territory (NT), South Australia, Western Australia, and Queensland] over the period 2012–2014 were conducted as part of a large-scale continuous quality improvement program. Multilevel regression modeling was used to quantify variation in recording of CVRA attributable to health center and patient characteristics.

Results: The proportion of eligible patients with documented CVRA was 33% (*n* = 574/1,728). The majority (95%) of assessments were conducted in the NT. Multilevel regression analysis showed health center characteristics accounted for 70% of the variation in assessments in the NT. Government-operated health centers had 18.8 times the odds (95% CI 7.7–46.2) of recording CVRA delivery compared with other health centers.

Conclusion: Health centers in the NT delivered the majority of absolute CVRA to Indigenous patients with diabetes in our study. Health systems factors that may have facilitated provision of CVRA in the NT include decision support tools and a reporting process for CVRA delivery. Implementation of similar systems in other jurisdictions may help improve CVRA delivery. Early identification and treatment of high risk individuals through wider use of CVRA may help reduce the burden of CVD in Indigenous Australians with diabetes.

Keywords: absolute cardiovascular risk assessment, indigenous, diabetes, audit, quality improvement, primary health care

**Abbreviations:** ABCD, audit and best practice for chronic disease; CARPA, Central Australian Rural Practitioner's Association; CCMM, chronic conditions management model; CKD, chronic kidney disease; CQI, continuous quality improvement; CVD, cardiovascular disease; CVRA, cardiovascular risk assessment; NHMRC, National Health and Medical Research Council; NT, Northern Territory; NTG, Northern Territory government; NVPDA, National Vascular Disease Prevention Alliance; PCIS, primary care information system.

### INTRODUCTION

Absolute cardiovascular risk assessment (CVRA) provides an assessment of cardiovascular risk based on the effects of a combination of risk factors. CVRA helps identify individuals who are asymptomatic but at high risk of cardiovascular disease (CVD), including stroke, transient ischemic attack, myocardial infarction, peripheral arterial disease, and heart failure (1). Early identification of individuals at high risk of CVD (>15% risk of CVD within the next 5 years) using CVRA can enable effective management of modifiable risk factors, including lifestyle changes, pharmacotherapy, and improvement of CVD-related clinical targets (1–8). CVRA and targeted treatments are cost-effective and have the potential to reduce the burden of CVD (9, 10).

Globally, CVD is the leading cause of death and imposes a substantial social and economic burden at population and household levels (9, 11). Socioeconomically disadvantaged populations, including Indigenous peoples, have a disproportionate burden of CVD and risk factors of CVD, including diabetes (12–14). In Aboriginal and Torres Strait Islander people, the Indigenous people of Australia, CVD and diabetes, respectively, account for 27 and 8% of deaths (15). CVD and Type 2 diabetes also account for 23 and 12%, respectively, of health gaps between Indigenous and non-Indigenous Australians (16). Preventive efforts targeting modifiable risk factors for both CVD and diabetes offer significant opportunity to improve Indigenous health in Australia and internationally (12–14, 16). A recent National Health and Medical Research Council (NHMRC) document on improving performance in the management of CVD in Australian hospitals noted that significant improvements in cardiovascular health care delivery are possible, including provision of preventive services such a CVRA to Aboriginal and Torres Strait Islander people (17).

The National Vascular Disease Prevention Alliance (NVDPA) recommends the Framingham Risk Equation (derived from the Framingham Heart Study) be used to calculate absolute CVD risk and includes age, sex, systolic blood pressure, smoking status, total cholesterol, HDL cholesterol, diabetes, and electrocardiogram evidence of left ventricular hypertrophy in CVRA calculations (1). NVPDA recommends that "adults with diabetes who are 60 years or less" without known CVD or increased risk of CVD should receive absolute CVRA at least every 2 years depending on the level of risk of CVD (1). Although the Framingham Risk Equation has been found to underestimate risk in Aboriginal and Torres Strait Islander people and in people with diabetes, absolute CVRA using the Framingham risk equation is a more accurate predictor of future vascular events for these populations than any single CVD risk factor alone (1, 18, 19).

The use of absolute CVRA in the primary care setting for Aboriginal and Torres Strait Islander people with diabetes has not been previously reported. Previous studies have been limited to assessment of individual risk factors and the measures taken to address these risk factors (20–23). The aim of our study is to investigate the delivery of absolute CVRA in patients with Type 2 diabetes attending Indigenous primary healthcare services between 2012 and 2014 and to identify the influence of health center and individual patient level factors on the delivery of absolute CVRA.

### MATERIALS AND METHODS

From 2002, the Audit and Best Practice for Chronic Disease (ABCD) project has been operating across several jurisdictions and aims to investigate variation in and to improve the quality of care in a range of priority areas in Indigenous primary care, including chronic diseases, maternal health, and child health. From 2010, One21seventy, a not-for-profit entity, continued the service support for continuous quality improvement (CQI) to over 200 Indigenous health centers (24). These health centers use clinical audit tools developed by the Menzies School of Health Research to assess recording of service delivery and quality of service provision. One hundred and seventy of these services have voluntarily provided their de-identified audit data to the ABCD National Research Partnership for research of variation in quality of care, barriers, and strategies for improvement.

This study used the ABCD/One21seventy Type 2 diabetes audit dataset. This dataset included records of participating health centers' delivery of recommended services for patients with Type 2 diabetes. Of 170 Aboriginal and Torres Strait Islander community health centers participating in the ABCD National Research Partnership, 121 health centers in four states and territories audited clinical records to assess the quality of Type 2 diabetes care between January 2012 and December 2014. For the participating community health centers, the records of Indigenous patients who met the following criteria were eligible for audit: (1) A definite diagnosis of Type 2 diabetes according to health center records; (2) aged 15 years and above; and (3) lived in the community for 6 months or more in the last year. Residence in the community for ≥6 months in the last year was determined by examining health center records and discussions with Aboriginal and/or Torres Strait Islander Health Workers/ Practitioners familiar with the community.

From December 2011, the diabetes audit tool included a question on CVRA delivery, "Is there a record that an absolute CVR assessment has been performed within the last 12 months?" (25). The audit protocol specified that there needed to be a clear record that a CVRA had been completed. While a record of a statement such as "CVRA done" was sufficient to satisfy this requirement, there was also usually a record of level of risk. To determine current status of absolute CVRA delivery, records from the most recent Type 2 diabetes audit between 2012 and 2014 from each health center were included. Data were abstracted by health center staff using standard protocols and who had been trained in the use of ABCD/One21seventy audit tools, with support from quality improvement facilitators and One21seventy staff.

We excluded records of patients greater than 60 years of age in line with the NVDPA recommendations (1, 26). Patients with chronic heart disease, congestive heart failure, systolic blood pressure greater than 180 mmHg, or diastolic blood pressure greater than 110 mm Hg and previous acute myocardial infarction were excluded from the study because they have clinically determined high risk of CVD (1). Patients with estimated glomerular filtration rate (eGFR) less than 45 mL/min/1.73 m2 have clinically determined high risk of CVD (1). Categories of eGFR included in the audit tool were ≥60 mL/min/1.73 m2 , 30–59 mL/ min/1.73 m2 , 15–29 mL/min/1.73 m2 , <15 mL/min/1.73 m2 , and "no record." As <45 mL/min/1.73 m2 was not specifically included as a category of eGFR in the audit tool, we excluded patients with eGFR <60 mL/min/1.73 m2 because of likely high CVD risk.

Health center characteristics included in the audit tool were size of service population, type of health center (community controlled or government operated), CQI experience, and location based on the Australian Standard Geographical Classification System (very remote, remote or non-remote). The classification of remoteness is based on the physical distance of a populated locality to the nearest urban center and reflects access to goods and services (27). Patients were excluded from the analysis if they did not attend the health center over the past 12 months or if CVRA was recorded as "not due."

The audit records included the following patient-level characteristics: age, sex, documented comorbid conditions, and complications. Hypertension, dyslipidemia, chronic obstructive pulmonary disease (COPD), chronic kidney disease (CKD), depression, and other mental illness were each recorded as present or absent. Similarly, complications of diabetes (retinopathy, neuropathy, foot ulcer, amputation, and gastroparesis) were recorded as present or absent.

Ethics approval was obtained from research ethics committees in each jurisdiction: Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research (HREC-EC00153); Central Australian Human Research Ethics Committee (HREC-12- 53); Queensland Human Research Ethics Committee Darling Downs Health Services District (HREC/11/QTDD/47); South Australian Aboriginal Health Research Ethics Committee (04- 10-319); Curtin University Human Research Ethics Committee (HR140/2008); Western Australian Country Health Services Research Ethics Committee (2011/27); Western Australia Aboriginal Health Information and Ethics Committee (111- 8/05); and University of Western Australia Human Research Ethics Committee (RA/4/1/5051).

#### Statistical Analysis

Our data had a multilevel dependency structure, with individual patients clustered within health centers and health centers clustered within jurisdictions. Multilevel mixed effects logistic regression analysis was used to quantify variation in the delivery of absolute CVRA attributable to health center and patient level characteristics. We calculated unadjusted odds ratios to measure the unadjusted association between dependent and independent variables. All variables, including non-significant variables, were included in further analyses. In a hierarchical approach, we included health center variables (Model A) and then patient variables (Model B). The amount of variation due to introduction of the different groups of variables in the models was determined by the proportional change in variance at different levels. Because CVRA delivery outside the Northern Territory (NT) was limited, we restricted our analysis to this jurisdiction and compared results to the same analysis for the NT and other states. A *p* value ≤0.05 was considered significant. We completed statistical analyses with STATA software, version 14.

#### Sensitivity Analyses

We evaluated the inclusion of patients with CKD *and* eGFR greater than 60 mL/min/1.73 m2 , that is separate analyses were conducted for datasets including (1) patients with CKD *and* eGFR greater than 60 mL/min/1.73 m2 and (2) patients with no evidence of CKD or eGFR greater than 60 mL/min/1.73 m2 .

### RESULTS

The participating health centers had 2,389 clinical records for Indigenous patients with Type 2 diabetes between 15 and 60 years of age. The records for 1,728 patients met the inclusion criteria for the study (**Figure 1**). There was wide variation in delivery of absolute CVRA across jurisdictions (**Table 1**). In the NT, 56% of eligible patients received CVRA, compared to 3% in other jurisdictions. The majority (95%) of assessments were conducted in the NT.

Of the 52 participating health centers in the NT, 87% were located in very remote areas, 52% had a service population of 500 or fewer, and 77% were government operated (**Table 2**). The proportion of government and community-controlled health centers were estimated to be 76 and 52%, respectively, of total government and community-controlled health centers in the NT. Eight hundred and seventy seven patients (89%) from the NT were between 30 and 60 years of age (**Table 2**). Sixty-two percent of

patients were females. Fifty-one percent of patients were recorded as having dyslipidemia. Diabetic neuropathy was the most common complication of diabetes and was recorded for 4% of patients.

The strongest predictor of CVRA delivery was location of health center in the NT. (Table A1 in Supplementary material). For the NT, unadjusted logistic regression showed that government-operated

TABLE 1 | Number and percentage of patients who received absolute CVRA in participating health centers between 1 January 2012 and 31 December 2014 in the Northern Territory and other jurisdictions.


*a Other jurisdictions include Queensland, South Australia, and Western Australia.* health centers were more likely to record delivery of CVRA than other health centers (unadjusted odds ratio 13.6, 95% confidence interval 6.1–30.5). In the adjusted analysis (Model A), the association between government-operated health centers and CVRA delivery increased (OR 21.0, 95% CI 8.7–50.4). Remote location was also associated with provision of CVRA (OR 12.3, 95% CI 1.6–93.5) in Model A. The proportional change in variance from the unadjusted model to Model A showed that health center factors explained 70% of variation in the administration of absolute CVRA across health centers in the NT (**Table 3**). The addition of patient level variables did not appreciably explain further variation in CVRA. Remote location (OR 13.0, 95% CI 1.6–103.7), government-operated health centers (OR 18.8, 95% CI 7.7–46.3), and female gender (OR 1.5, 95% CI 1.1–2.1) were significantly associated with CVRA delivery in Model B.

#### Sensitivity Analyses

When patients with CKD *and* eGFR greater than 60 mL/min/1.73 m2 were excluded, predictors of CVRA delivery were similar to the results described above. (Table A2 in Supplementary material)

TABLE 2 | Health center and patient characteristics of participating health centers in the Northern Territory and other jurisdictions between 1 January 2012 and 31 December 2014.


TABLE 3 | Unadjusted and adjusted multilevel regression analysis of health center and patient level characteristics on recording of absolute CVRA for patients with diabetes in participating health centers in the Northern Territory between 1 January 2012 and 31 December 2014 (*n* **=** 52 health centers; 984 patient records).


*a Statistically significant.*

*bComorbidities and complications were compared with patients without the specific comorbidity or complication, such that an odds ratio of 1 relates to not having the specific comorbidity or complication.*

*c The six patients with gastroparesis had no recorded CVRA.*

### DISCUSSION

Our study found wide variation in the delivery of absolute CVRA across jurisdictions over the study period. The majority (95%) of assessments were conducted in the NT. Health center factors accounted for 70% of the variation in CVRA delivery in the NT. Factors associated with increased recording of CVRA delivery were government-operated health centers, remote health centers, and female gender (Model B, **Table 3**).

Men represented 38% of eligible patients in the NT and were less likely to receive CVRA than women. While fewer Aboriginal and Torres Strait Islander men than women report a diagnosis of diabetes (7.3% compared to 8.9%) (28), they also access preventive health services less often than women (29, 30). Poor health seeking among men results in poor use of health services and limited opportunities for access to health information, promotion, and preventive care. Further research is required to better understand how to deliver preventive health services and gender-appropriate healthcare to men (29).

Consistent with other studies, we found gaps in recording absolute cardiovascular risk (6, 19, 31). The Treatment of Cardiovascular Risk using Electronic Decision Support (TORPEDO) study recorded relatively high levels of CVRA. In this study, 30 primary healthcare centers in Queensland and New South Wales were randomized to receive a computerguided quality improvement intervention, and 30 centers were randomized to usual care. At intervention sites, 63% of eligible patients received appropriate CVD screening compared to 53% of patients attending non-intervention sites (RR 1.25; 95% CI 1.04–1.50; *p* = 0.02) (32). In the Kanyini Audit, 53% of 1,165 randomly selected case records of adults attending Indigenous health centers lacked information for one or more Framingham risk variables. Screening and management gaps were similar to those found in non-Indigenous health settings, suggesting a need for improvement across the health system (19). Despite similarities in the gaps in screening, ongoing disparities in cardiovascular risk and complications persist between Indigenous and non-Indigenous Australians (16, 22). Absolute CVRA may help reduce these disparities by encouraging early detection and intervention for Indigenous adults at high risk of CVD (6, 22).

There are a number of possible explanations for relatively high delivery of CVRA in government-operated health centers in the NT. First, Burgess et al.'s study of CVRA in Arnhem Land in 2005 showed that CVRA during the adult health check may contribute to better and earlier detection of cardiovascular risk and population reductions in cardiovascular risk (6). These findings led to the incorporation of CVRA into the Central Australian Rural Practitioner's Association (CARPA) Manual (33). Second, following the study in Arnhem Land, CVRA was promoted in Preventable Chronic Disease Strategy Workshops in the NT in 2006–2007. Primary care providers participating in the workshop received education about the utility of CVRA in the adult health check (34). Chronic care educators and CQI facilitators have continued to provide outreach support to primary care providers to implement CVRA. Third, following the commencement of the Chronic Conditions Management Model (CCMM) in 2012, Northern Territory Government (NTG) health services received regular feedback about (i) the proportion of Indigenous patients aged >20 years who received CVRA, (ii) gaps between screening and initiating treatment to allow patients who are missing care to be identified and recalled, and (iii) achieving targets for modifiable risk factors for individuals identified at high risk of CVD. Burgess et al. found that implementation of CCMM led to improvements in NT population coverage of CVRA from 23% in mid-June 2012 to 58.5% in August 2014 (8). Introducing similar auditing processes and national CVRA-related key performance indicators may improve CVRA delivery in other jurisdictions.

Wide-scale CVRA can be supported by including a userfriendly CVRA calculator in clinical information systems. For example, the computer-guided decision support tool in the TORPEDO study improved CVD risk measurement. The absence of an electronic CVRA calculator has been identified as a barrier to CVRA delivery in primary care settings (35). Community-controlled health centers in the NT have campaigned for incorporation of a CVRA calculator in clinical information systems but this is not yet available. As a result, clinicians calculate CVRA manually. Lack of access to such a tool may contribute to underreporting of CVRA in communitycontrolled health centers in the NT.

There is contention about the usefulness of CVRA in Aboriginal and Torres Strait Islander people and people with diabetes because the Framingham risk equation can underestimate risk in these populations (1, 18). To date, no studies have been published evaluating adjustments for Indigenous Australians that are similar to adjustments for Maori, Pacific Islander, and Indian patients recommended in New Zealand guidelines (6, 36). While further work is required to develop accurate CVD risk estimates for Indigenous Australians, adjustments included in the CARPA Manual risk calculators may provide a more reliable estimate of CVD risk than the Framingham risk equation (6, 33). Further research is also required to assess whether CVRA leads to improved clinical outcomes in Indigenous people with diabetes.

The strength of this study is the inclusion of 76% of NT government-operated services and 52% of community-controlled health services in this study. As a significant proportion of NT health services are included in this audit, the results are likely to be generalizable to the NT, and perhaps to a more limited extent to other jurisdictions. The age groups of patients included in the study are similar to the age groups for which Aboriginal and Torres Strait Islander adult health checks are recommended (aged 15 years and over but under the age of 55 years) (37). Our study assessed delivery of CVRA in regular residents in the community (i.e., individuals who had lived in the community for at least six of the past 12 months) as visitors or short-term residents of the community may not be regarded by health center staff as being within the priority target group for preventive care.

The main limitations of this study are (1) potential participation bias as only community health centers that participated in the One21seventy clinical audit cycles were included in the study. For example, the TORPEDO study found relatively high levels of CVRA in community controlled health services in Queensland (32). Less than 5% of community controlled health services in Queensland participated in this study. (2) The study relied on patient records to abstract information about CVRA. Under-documentation may result in underestimation of service delivery and contribute to over-servicing, inefficiency and poor coordination and continuity of patient care. However, clear, accurate documentation is essential for coordinating health service delivery (23). (3) Clinical audits were carried out by multiple data abstractors and inter-rater reliability was not formally assessed in this study. Although inter-rater reliability was satisfactory in previous studies that used similar audit forms (23, 38), it is possible that manual CVRA calculations may have been missed. (4) The ABCD audit protocol did not record total cholesterol >7.5 mmol/L, familial hypercholesterolemia, microalbuminuria, or eGFR <45 mL/min/1.73 m2 . Adjustments to the audit protocol may improve detection of these patients with clinically determined high risk of CVD in future studies.

While the clinical audit included information on the date of diagnosis of diabetes, in many cases there was not a clear record of date of diagnosis. We therefore included all patients with a diagnosis of diabetes, regardless of the date of diagnosis. There may therefore be a small proportion of patients who may have been diagnosed so recently that the period between the diagnosis and the date of the audit may have been so short that it would not be reasonable to expect that some of the scheduled services specified in the audit tool would have been delivered. The same approach was applied in all audits, and so comparisons between groups are unlikely to be affected. The proportion of patients affected is also likely to be too small to have made a meaningful impact on the findings. Despite limitations of the audit method, the study provides valuable guidance for policy, practice and further research.

### CONCLUSION

In conclusion, we found wide variation in the delivery of CVRA across jurisdictions in our study. The NT delivered the majority of CVRA to Aboriginal and Torres Strait Islander patients with diabetes. Health center characteristics accounted for the majority of variation in CVRA delivery. A number of health systems factors, including the electronic CVRA calculator and the CCMM, may have facilitated delivery of this service. Further understanding of the impact of CVRA on the clinical outcomes of Aboriginal and Torres Strait Islander people with diabetes may provide additional guidance to clinicians and policy makers. Efforts to improve early identification and treatment of high risk individuals in the primary care setting may play a critical role in reducing the burden of CVD in Indigenous Australians with diabetes.

### AUTHOR CONTRIBUTIONS

BV completed a literature review for this article, drafted the project proposal, performed the statistical analysis, and drafted the manuscript. VM manages the ABCD dataset and provided assistance with the statistical analysis and drafting of the manuscript. RB is responsible for overall leadership of

### REFERENCES


the ABCD National Research Partnership, helped develop the project proposal, provided feedback for the literature review, and assisted with drafting the manuscript. CB and CC assisted with interpretation of the analysis and drafting the manuscript.

#### ACKNOWLEDGMENTS

We are grateful to Cynthia Croft, Frances Cunningham, Elizabeth Moore, and Jacqueline Murdoch for their helpful comments in the preparation of this manuscript. This project would not have been possible without the active support, enthusiasm, and commitment of the staff and management of the participating health services and by the contribution made by the wider ABCD National Research Partnership Project Team.

### FUNDING

The ABCD National Research Partnership Project is supported by the National Health and Medical Research Council (ID No. 545267) and the Lowitja Institute, and in-kind and financial support from a range of Community Controlled and Government Agencies. RB's work is supported by an ARC Future Fellowship (#FT100100087).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fpubh.2016.00037

communities in Australia's Northern Territory. *Heart Lung Circ* (2015) **24**(5):50–7. doi:10.1016/j.hlc.2014.11.008


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Vasant, Matthews, Burgess, Connors and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Exploring Systems That Support Good Clinical Care in Indigenous Primary Health-care Services: A Retrospective Analysis of Longitudinal Systems Assessment Tool Data from High-Improving Services

#### *Edited by:*

*Edward Broughton, University Research Co., USA*

#### *Reviewed by:*

*Moira Jean McKinnon, Midwest Population Health, Western Australian Health, Australia Anthony Pascual Calibo, Department of Health, Philippines*

#### *\*Correspondence:*

*Karen Carlisle karen.carlisle@jcu.edu.au*

> *† Lead author.*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 16 December 2016 Accepted: 24 February 2017 Published: 24 March 2017*

#### *Citation:*

*Woods C, Carlisle K, Larkins S, Thompson SC, Tsey K, Matthews V and Bailie R (2017) Exploring Systems That Support Good Clinical Care in Indigenous Primary Health-care Services: A Retrospective Analysis of Longitudinal Systems Assessment Tool Data from High-Improving Services. Front. Public Health 5:45. doi: 10.3389/fpubh.2017.00045*

*Cindy Woods1† , Karen Carlisle2,3,4 \*, Sarah Larkins2,3,4 , Sandra Claire Thompson5 , Komla Tsey3,6 , Veronica Matthews7 and Ross Bailie8*

*1School of Health, University of New England, Armidale, NSW, Australia, 2College of Medicine and Dentistry, James Cook University, Townsville, QLD, Australia, 3Anton Breinl Research Centre for Health Systems Strengthening, James Cook University, Townsville, QLD, Australia, 4Australian Institute of Tropical Health and Medicine, James Cook University, Townsville, QLD, Australia, 5Western Australian Centre for Rural Health, University of Western Australia, Geraldton, WA, Australia, 6Cairns Institute and College of Art, Society and Education, James Cook University, Cairns, QLD, Australia, 7Menzies School of Health Research, Brisbane, QLD, Australia, 8University Centre for Rural Health, Lismore, NSW, Australia*

Background: Continuous Quality Improvement is a process for raising the quality of primary health care (PHC) across Indigenous PHC services. In addition to clinical auditing using plan, do, study, and act cycles, engaging staff in a process of reflecting on systems to support quality care is vital. The One21seventy Systems Assessment Tool (SAT) supports staff to assess systems performance in terms of five key components. This study examines quantitative and qualitative SAT data from five high-improving Indigenous PHC services in northern Australia to understand the systems used to support quality care.

Methods: High-improving services selected for the study were determined by calculating quality of care indices for Indigenous health services participating in the Audit and Best Practice in Chronic Disease National Research Partnership. Services that reported continuing high improvement in quality of care delivered across two or more audit tools in three or more audits were selected for the study. Precollected SAT data (from annual team SAT meetings) are presented longitudinally using radar plots for quantitative scores for each component, and content analysis is used to describe strengths and weaknesses of performance in each systems' component.

Results: High-improving services were able to demonstrate strong processes for assessing system performance and consistent improvement in systems to support quality care across components. Key strengths in the quality support systems included adequate and orientated workforce, appropriate health system supports, and engagement with other organizations and community, while the weaknesses included lack of service infrastructure, recruitment, retention, and support for staff and additional costs.

Qualitative data revealed clear voices from health service staff expressing concerns with performance, and subsequent SAT data provided evidence of changes made to address concerns.

Conclusion: Learning from the processes and strengths of high-improving services may be useful as we work with services striving to improve the quality of care provided in other areas.

Keywords: quality improvement, Indigenous health, primary health services, primary health care, systems improvement

### INTRODUCTION

There are clear disparities in the health of Indigenous and non-Indigenous Australians, with higher ranks of morbidity and mortality among Aboriginal and Torres Strait Islander populations and rural and remote populations (1, 2). High-quality primary health care (PHC) delivered consistently by PHC services is essential (but not alone sufficient) to "close the gap" in Aboriginal and Torres Strait Islander health outcomes (3). Quality PHC in general relates to the degree with which care complies with agreed best practice and is often defined in terms of efficiency, effectiveness, capability, accessibility, safety, appropriateness, continuity, responsiveness, and sustainability (4, 5). However, despite agreed clinical practice guidelines, there is wide variation in delivery of care and processes for evaluating quality of care. Continuous Quality Improvement (CQI) aims to facilitate ongoing improvement in the quality of primary care by using objective information to analyze and improve systems, processes, and outcomes (6). The evidence for the effectiveness of CQI is mixed and context dependent (7, 8); however, studies have shown that in some settings, it can be effective in improving quality of care (9), professional practice (10), and patient outcomes (11), particularly when used over longer periods of time (12).

Modern CQI approaches are increasingly participatory in their methods, support the use of collaborative team-based discussions and have a "customer focus," which may be more suited to the Indigenous Australian setting (6, 13–15). Building on the Audit and Best Practice in Chronic Disease (ABCD) tools, the One21seventy CQI tools aim to improve the quality and consistency of PHC provided to Aboriginal and Torres Strait Islander people by using clinical audit data to analyze and improve systems, processes, and outcomes (16). The name One21seventy reflects the commitment to increasing life expectancy for Australian Aboriginal and Torres Strait Islander people beyond 1 year in infancy, 21 years in youth, and 70 years in the lifespan (3). The One21seventy CQI process is an annual Plan + Do + Study + Act cycle that uses a number of tools to gather data to facilitate health centers' planning, goal setting, and implementation of improvements. The One21seventy process include a range of clinical audit tools and a Systems Assessment Tool (SAT). The clinical audit tools are used to collect data to measure the overall adherence to the delivery of guideline-scheduled services to prevent or manage chronic conditions and provide maternal and child health care (Menzies School of Health Care, 2011). As part of the CQI audit cycle, health services are encouraged to conduct a systems assessment to identify strengths and weaknesses in clinical care and health service systems and areas that should be addressed to enhance quality of care. The SAT is an Australian developed scale used to assess the organizational systems of Indigenous PHC services as part of the CQI process. Use of the SAT initially began in 2002–2005 in 12 Northern Territory PHC services. Between 2005 and 2009, use of the SAT expanded to 63 PHC services in four Australian states and territories and continued to expand from then (17).

A SAT process is ideally undertaken by means of a group meeting involving clinical, administrative, and management staff, following the CQI audit. The SAT is based on the Assessment of Chronic Illness Care scale and adapted for use in Indigenous PHC. It was designed to assess systems across multiple areas of care and to be delivered in group setting. Scores are reached by consensus with prompts provided to increase standardization and reproducibility in scoring [(18), see also **Table 1**]. In some health services, an external facilitator is brought in to assist with the process. The SAT is used by primary health service staff as both a measurement and developmental tool, thereby enabling health service staff to score their health service systems across various domains necessary for effective care: delivery system design, information systems and decision support, self-management support, links with community and other health services, and organizational influence and integration. The SAT allows PHC services to identify priority areas for system improvement and to track variations in systems performance over successive CQI audit cycles. Following the audits and SAT, health service staff are encouraged to undertake collective goal setting and action planning to enhance the quality of evidence-based care to patients over the next 12 months. Governance varies among Australian PHC services, including government-operated services, community-controlled services with Board management, and a combination of both. Systems of governance are ultimately responsible for the implementation of CQI in Indigenous PHC services, whereas PHC services are generally responsible for the planning and conduct of CQI audits and SAT process.

This study aimed to identify the processes used in systems assessment and the strengths and weaknesses of the systems in place to support the provision of quality client care using quantitative and qualitative SAT data from five consistently highimproving Indigenous PHC services.

#### TABLE 1 | Components of the Systems Assessment Tool.a


*a Reproduced with the permission from Menzies School of Health Research.*

### MATERIALS AND METHODS

#### Service Selection

To select high-improving services, we calculated quality of care indices for Indigenous health services participating in the ABCD National Research Partnership. These indices were based on the delivery of scheduled services against the recommended service provision in four audit areas: maternal health, child health, preventive health, and chronic disease (type 2 diabetes). Highimproving services were then selected on the basis of continuing high -improvement over at least two of the four audit tools over at least three audits. The method used to calculate the consistent high-improvement category of health services is described in full elsewhere (19).

Six health services met the inclusion criteria of continuous high improvement. Examination of SAT data from the six health services revealed that one service did not record text (qualitative data) to justify SAT scores and was therefore not included in the analysis. Thus our analysis and findings are based on data from five high-improving Indigenous PHC services.

#### Study Design

Longitudinal quantitative and qualitative SAT data from five Indigenous PHC services between 2005 and 2014 were analyzed to identify the strengths and weaknesses that supported or constrained the provision of quality health care.

### Data Collection

Pre-existing prospectively collected longitudinal One21seventy SAT data from five high-improving Indigenous PHC services located in northern Australia were analyzed for this study.

Systems Assessment Tool data are recorded for the five main systems components, and items within each component (**Table 1**) were as follows: delivery systems design (8 items); information systems and decision support (3 items); self-management support (2 items); links with community, other health services, and other services and resources (4 items); and organizational influence and integration (3 items). When each component of the system is assessed in the team meeting, a score from 0 to 11 is allocated to all items within the component, 0–2 for limited or no support, 3–5 for basic support, 6–8 for good support, and 9–11 for fully developed support. An overall score for each component is the average of the item scores. The overall score for each component is presented on a radar plot, displaying the strengths and weaknesses of the systems components.

Although there is some variation in how services conduct a SAT assessment, the process usually involves a facilitated discussion of a mixed group of staff members. They discuss the performance of the service using descriptors against each criterion and then reach consensus on an agreed score for each element. Full detail is available in the SAT tool, coding guide, and facilitator guide (http://www.menzies.edu.au/icms\_docs/256788\_Systems\_ Assessment\_Tool.pdf).

Health service staff who participate in the systems assessment can also enter free text to justify the score for each item. The five identified consistently high-improving health services each completed between two and five SATs. Two health services completed a combined SAT covering two and three audit tools.

#### Study Services

The characteristics of the five health services categorized as consistent high-improvers in this study are described in **Table 2**, along with a summary of how SAT processes were conducted at these services. Most are government-operated health services located in remote locations with relatively small populations (<1,000 people). One of the services is a community-controlled service, and another service is in a larger regional community located at a community "crossroads." All health services are located in northern Australia, and four are located in communities with predominantly Indigenous populations.

#### Data Analysis

Quantitative data ranking systems performance for each service is displayed using radar plots. Free text comments associated with these SAT data were analyzed using qualitative content analysis to identify strengths and weaknesses reported over the period of time each PHC service participated in One21seventy audits. Qualitative content analysis is a technique for systematic, replicable text analysis, used to reduce large amounts of text into fewer manageable codes, and to determine the presence of certain concepts within texts (20).

Each item of text justifying the SAT score for each health service, year, and audit tool was analyzed, and concept occurrences were summarized. These concepts were then descriptively coded. Coding categories were based on the stated strengths and weaknesses within each score justification. Identification of concepts allows for conclusions and generalizations to be drawn based on trends indicative of larger ideas (20).

Quantitative data were analyzed using the non-parametric Wilcoxon Signed Rank test to compare the first raw SAT cycle scores of each site with the final raw SAT cycle scores. The Wilcoxon Signed Rank test was selected due to small numbers (scores range from 0 to 11) and the non-normal distribution of data, and the paired nature of the data. Alpha of <0.05 was considered statistically significant.

### RESULTS

**Table 3** shows the year and tool for which SATs were undertaken at each health service.

Table S1 in Supplementary Material provides a summary of the strengths and weaknesses for each service by each component derived from the free text of the SATs. The process for undertaking CQI audits and completion of SATs varied across the highimproving health services. Some of the services adopted a formal approach which involved all staff members, while in other services, they were carried out by an external team with limited involvement from the health service staff. **Figures 1**–**5** show radar plots demonstrating changes over time in SAT component rankings at each health service. The overall trend showed improvement in


*QLD, Queensland; WA, Western Australia; NT, Northern Territory; SAT, Systems Assessment Tool.*

each of the SAT components at each service over time; however, as can be seen by the shape of the radar plots, individual services differed in the speed and degree to which various components were addressed.

An overall view of the changes in SAT component rankings over time alongside the reported weaknesses and strengths within each component item are presented in the following paragraphs. Exploration of the justifications for the rankings for each item within each component provides further clarification on why particular rankings were given, illustrate some of the strengths and weakness within the services, and show examples of the

TABLE 3 | Year and tool of Systems Assessment Tool (SAT) completed at each site.


*X, CQI audit and SAT completed.*

*A, CQI audit completed but not SAT.*

impact changes within the service had on rankings of component items within delivery system design. Selected quotations from the free text comments of the SAT process are used to illustrate examples of identified weaknesses and strengths.

#### Delivery System Design

Examination of the rankings given across the five services indicated that most of the selected services reported an improvement in this component over time (**Figures 1**–**5**). Two services reported improvement from early SAT cycles; however, the most recent SATs showed a contraction (**Figures 1** and **3A,B**). Key factors related to weaknesses and strengths of delivery system design as identified in the qualitative data were related to recruitment and retention of staff, presence of supportive clinical leadership, appropriate health service systems and processes to support client care and a culturally appropriate and accessible health service. **Table 4** shows that sites 3 (T2DM), 4 (maternal health) and 5 (preventive and child health) reported significantly improved delivery system design scores from the first cycle to the final cycle.

#### Team Structure and Function

Qualitative data from early SATs indicated that staffing constraints, recruitment, and retention of staff and issues associated with developing a functional team were key influences on team structure and function. Weaknesses related to staffing included a lack of specialist staff, gender appropriate staff for client groups, appropriate practitioners (particularly Indigenous practitioners) for a team approach, specific staff to oversee programs, and consistent staff. Staffing constraints impacted on the services in different ways. Staff shortages affected training opportunities, regularity of team meetings, the ability of the team leader to coordinate delivery of care, and a lack of a consistent GP affected client follow-up. The following comment highlights the impact

of staff shortages on the professional development of key staff, and the Indigenous health worker (IHW) became underutilized.

Trainee HW not being utilised and getting enough training due to no Admin, Trainee is having to do admin duties and they aren't employed to do admin duties. (Service 1)

In addition, four services reported weaknesses associated with team functioning, including a poorly defined team leadership role, uncertainty and confusion about roles and responsibilities, and irregular team meetings.

Improvement in rankings for this component was associated with adequate staffing levels, thus transforming weaknesses into strengths. When staff issues were resolved, services reported having gender appropriate staff for client groups, practitioners for a team approach, dedicated staff to do recalls, increased training and professional development opportunities, and IHWs were able to fulfill their role and continue their training program. However, many of the improvements were dependent on retention of staff, and when staff left, services were vulnerable. For example, in Service 1, many of the issues with team functioning resurfaced when they lost key staff. Overall, reported improvement in team functioning was associated with clear definition of team roles, team communication and cohesion, and an established team approach for practitioners. Support from management or established reporting mechanisms were also noted as strengths at three services as illustrated below.

Team leadership clearly defined and recognised, leader has formal authority. Definition of team roles, lines of reporting and integration in system design are good. Very good communication and cohesion within the team; team meetings regular; decision-making is very good. Development of team members' skills and roles is very good. (Service 2)

### Clinical Leadership

Early SAT data in some of the services identified inadequate clinical leadership as limiting the capacity of services to deliver quality health care. Leadership issues varied across the health services and were associated with the availability of leadership staff, tools to support clinical leadership, and the provision of support for on the ground staff. Weaknesses cited included poor medical support for remote health clinics, a lack of clinical leadership, lack of up-to-date evidence-based research, lack of support and direction for staff and programs, and no permanent manager. The comment below from an early SAT process at Service 3 provides an example of some of the factors related to poor clinical leadership.

Poor support and availability from doctors for remote clinics, uncertainty about which clinical guidelines to follow, lack of clinical leadership from GPs (Service 3)

Later SATs showed improvements in clinical leadership. In some cases, leadership staffing issues were resolved or resources were updated to support staff. Strengths such as the presence of strong and supportive leadership, stable management, and access to specialist support were reported as impacting on improved clinical leadership. In other services, innovative solutions were found, for example, comments from Service 3 describe how one midwife extended her skills to carry out her role with high-risk pregnant women, compensating for no on-service support.

The role of the midwife is as a sole practitioner within a community health team but often with no on-site professional skills and support. She either phones or emails

the DMO in [nearest regional centre] (250kms away) or the regional obstetrician based in [nearest city] 450 kms away. Therefore the midwife requires extended skills to enable her to carry out her role with high risk pregnant women i.e ultrasound dating scans. (Service 3)

### Appointments and Scheduling

Qualitative data from early SATs indicated that lower rankings for the item appointments and scheduling were due to a lack of an appointment system, staffing to manage appointments, and lack of routine planning for community programs. For some services, inflexibility with their appointments and community programs were described as justification for a lower ranking.

Clients are used to DE [Diabetes Education] clinics on specific days in specific communities… specialist appts are not flexible, surgery list needs to be longer to accommodate clients (Service 3)

Later SATs from three of the services reported that when an appointment system and routine clinics were established, rankings improved. Strengths identified in those settings included, regular clinics, activities and programs, and regular doctor and specialist staff visits.

The [Diabetes Educator] works in specific areas on specific days so people get into a routine…Doctors have routine visits to communities. Specialists and Allied Health have an appointment system. (Service 3)

By contrast, other services found that the communities they worked with expressed a preference for the flexibility of a drop-in

service and thus made changes to their systems to meet the needs of the community. One health service noted that "*People know they can come to the health service at any time (service 5)*" indicating there is flexibility built into the system to handle clients without an appointment.

### Care Planning

Care planning processes varied between health services but in most cases improved over time. Early SATs indicated weaknesses in care planning were associated with no electronic medical records or integrated IT system, lack of IT access and training (for remote clinics), no electronic recall system, inaccessibility and inconsistency of care plans, and poor or no documentation.

IT systems did not help with [Adult Health Checks] AHCs which were mostly opportunistic, & generally not completed. Men's health was not good, and there was a need for a more integrated IT system. (Service 4)

A second issue identified by some services was the capacity of clinicians to undertake planning activities. Qualitative data from the SATs identified issues such as lack of staff to update care plans, doctors not using care plans on IT system, and hospital doctors not seeing the development of care plans as their responsibility.

Improvements in care planning in later SATs were related to improvements in the availability and use of IT systems. For example, the adoption of flags and follow-up reminders used in paper client records, the routine use of the IT system for information sharing and electronic recall lists were reported by services as strengthening care planning. In addition, other reported strengths of care planning included routine use of care plans, a

team/specialist approach to complex care, and case conferences with families/specialists. The comment below describes a solution adopted at one health service where there was limited capacity at local level; therefore, support in relation to care planning was provided at the regional board level.

Complex clients' care incorporates local GP, Maternity Services, [nearest tertiary] hospital, all involved when necessary. (Service 4)

### Systematic Approach to Follow-up

Similar to care planning, client information system weaknesses and staffing constraints were reported in early SATs on the component item systematic approach to follow-up. Issues such as poor actioning of electronic flags/reminders, poor IT access and training, and inconsistent use of IT system for patient records and recalls were some of the reasons cited for lower rankings as part of the SAT process. The comment below describes one example of some of how the health service staff underutilized the IT system.

No use of recall system; C/care [IT system] in remotes; client records multiservice location & not complete; some data sharing between providers; multi-entering of same data. (Service 3)

A review of systematic approach to follow-up in later SATs highlighted improvements to how the IT systems were utilized. Key reported strengths across the services were an increase in the routine use of the IT systems to support follow-up and the availability of dedicated staff.

#### TABLE 4 | Mean scores for Systems Assessment Tool cycles.


*–, indicates numbers are too small to perform statistical test.*

*Bold font indicates statistically significant P values.*

Flags/reminders consistently used to support client care. Follow-up of clients for regular reviews is becoming part of routine practice. Follow-up of abnormal test results is becoming part of routine practice. (Service 2)

#### Continuity of Care

Rankings for continuity of care given by health service staff fluctuated over time for two reasons, staff turnover and poor communication. Poor communication at the point of discharge, from visiting specialists and across IT systems were some of the reasons cited as impacting on continuity of care. The excerpt below highlights how one service identified that its delivery system was not designed to facilitate continuity of care, and steps were being made to address the issue.

The delivery system was not designed to enhance continuity of care, and a system for routine post-discharge communication between hospital [and] the health centre was becoming established (Service 2)

When health services reported improvements in continuity of care, strengths included establishing a system of information sharing, shared care, and shared planning with other health-care providers. A later SAT conducted in Service 2 ranked continuity of care higher than in the previous SAT with the reasoning detailed below.

Very well-designed delivery system enhanced continuity of care (with all or almost all elements in place), and a system for routine post-discharge communication between the hospital and the health centre was fully established. (Service 2)

#### Client Access/Cultural Competence

Only one of the five services reported weaknesses in client access. Many of the issues were related to the physical infrastructure of the health services, such as wheelchair access, appropriate waiting room, and consultation rooms. Other issues were related to availability of transport and distance required to travel to attend appointments. The comment below describes an example of lack of an appropriate, private space that created barriers for women who may want to access maternal and child services.

Women have to ask the hospital receptionist to see the midwife or child health nurse. This could be a barrier to access if women are shy having to ask to see a midwife- which has to be said publicly in the waiting area. (Service 3)

In terms of issues related to cultural competence, two of the health services reported weaknesses. Similar to other component items within delivery system design, many of the reported weaknesses were as a result of staff constraints (for example, no or limited IHW) and availability of appropriate training to support staff (for example, cultural competence and gender awareness training).

Conversely, key strengths associated with higher rankings of client access included a clinic designed for client privacy and confidentiality and private consultation areas for men and women. Identified strengths within this component item comprised the availability of cultural orientation and training; in some contexts, this training was provided by Indigenous persons; Indigenous knowledge valued and gender appropriate staff are available for users of the services. The comment below is an example of some of the reasons why cultural competence was ranked highly at one health service.

Level of attention to cultural competence is good; usually included in orientation and training. Respect for gender-related issues is very good. Respect for Indigenous knowledge and IHW experience is very good. (Service 1)

### Physical Infrastructure, Supplies, and Equipment

Data collected as part of early SATs showed that physical infrastructure was reported as a weakness at three of the health services and reasons cited overlap with client access issues reported above. Weaknesses identified included inappropriate and lack of privacy of the waiting area and consultation rooms, lack of disability access, and space constraints for staff, visiting specialists, and clients. The comment below describes the physical infrastructure of one service which impacted on client care.

Clinics unsuitable for client care due to cramped conditions, lack of equipment and no consultation rooms. (Service 3)

Indeed, a more recent SAT process acknowledged that some of the infrastructure issues were resolved with the addition of new buildings and renovation and maintenance of existing buildings.

New clinic, new office, maintenance of equipment is timely, remote clinics being renovated/maintenance. (Service 3)

Health services that ranked physical infrastructure highly reported strengths such as appropriate infrastructure, quality equipment, and systems in place to manage timely maintenance.

### Information Systems and Decision Support

The scoring of this component as shown in the radar plots indicated a mixed picture of rankings over time. Two health services (**Figures 3** and **4**) showed improvement in the rankings over subsequent cycles. Two of the health services (**Figures 1** and **5A**) showed an improvement from cycle 1 to cycle 2; however, there was a contraction in the third cycle. One service (**Figure 2**) reported no change over time. Factors associated with weaknesses and strengths in information systems and decision support were related to the embeddedness of systems, the extent to which they were used to inform planning and support client care and collaboration with other health providers. **Table 4** shows increases in information systems and decision support scores are nearing statistical significance at sites 3, 4, and 5. Although changes in scores did not reach statistical significance, it is probable they represent clinical significance in terms of improvement in the quality of planning, client care, and collaboration at these sites.

### Maintenance and Use of Electronic Client List (ECL)

Early SAT data indicated that low rankings of maintenance and use of ECLs were due to a lack of routine use of the client lists and out of date information within the client lists. Those services that experienced a contraction in the rankings in later SATs cited irregular use of the ECLs as the reason for a lower ranking. The comment below is an example from one of the services who assigned a low ranking to this component item.

List available but not reviewed and out of date (covers less than 80% of clients, up-to-date residence and Medicare information sometimes recorded). Use of the list to identify regular clients for planning and delivery is ad hoc. (Service 1)

Services that reported improvements in maintenance and use of client list in later SATs cited strengths such as current and regularly updated ECLs, regular use of recall lists, and use of ECLs for planning and service delivery. One health service described the change in rankings over time as a result of the introduction and routine use of an electronic system.

A barrier of preventive health maintenance and use of electronic client list was the lack of an electronic system. This improved with the introduction of an electronic system although initially it was irregularly reviewed. The electronic client list is used routinely for planning service delivery and reaching client groups and is updated regularly. (Service 4)

#### Specialist–Generalist Collaborations

The processes to support the planning, delivery and coordination of care at the PHC level are, to a certain extent, dependent upon collaboration with other health-care providers. Only one of the health services reported weaknesses in collaborations that include no consultation, communication or feedback from specialists, limited specialist visits, and no client access to follow-up after specialist appointments.

Little support from Obstetrician. Gynaecologist visit 5 hrs 3 times a year not enough. No visiting Endocrinologist yet most women diabetic & high risk. Hard to know if specialists recall patients. (Service 3)

The reported weakness in collaboration was resolved in later cycles as a result of engagement in collaborative activities to improve relationships and links with specialists.

Good working relationship with staff and co-location of clinics helps with communication, Paediatrician and Obstetrician contribute to the MCH workshop which helps build relationships. Effective specialist links. (Service 3)

Other health services that ranked specialist-generalist collaborations highly reported similar strengths such as building good relationships, communication, and availability of support.

#### Self-management Support

The radar plots (**Figures 1**–**5**) show that this component item generally improved over time across all the selected health services. Three services showed continuous improvement over the SAT cycles (**Figures 1**, **3A,B** and **4A,B**). Two services showed an improvement from early SATs, which were maintained (**Figures 2** and **5**). Factors associated with weaknesses and strengths in self-management support included processes and resources to support self-management, availability of appropriately trained staff, and engagement with families and communities.

### Assessment and Documentation

Five services identified weaknesses with assessment and documentation within self-management support in early SAT processes. Processes that supported clients and families to maintain their health were reported with varying success across the services depending on the wider context of the health service. For example, one health service reported that although they were aiming for self-management, it was considered idealistic due to low client health literacy. For other services, high staff workload impacted on the routine use of self-management needs assessment with clients. Furthermore, another health service reported the use of hand-held records as a weakness due to clients declining them.

Don't always have time to educate, clients not always ready to be educated on self-management. (Service 5)

Improvements in this component item in later cycles were attributed to strengths such as consistent use of self-management needs assessments and ongoing engagement with clients and their family in goal setting and care planning.

Assessment and documentation of self-management needs is routine practice. Clients/families engagement in assessment and documentation is routine practice. (Service 1)

### Self-management Education and Support, Behavior Risk Reduction, and Peer Support

Early SAT data from most of the services identified weaknesses in support structures to help clients and families to manage their health problems. Some issues were related to supporting health service staff such as adequate staffing and staff time, time constraints, and the lack of staff education, training, and skills. Others were related to the ways in which health service staff engaged with the community as illustrated by the comment below.

Ad hoc engagement of families in education/support activities (Service 2)

Qualitative data from later SATs showed that identified weaknesses resolved into strengths when staff had relevant training, skills, and appropriate resources to provide self-management education. In addition, ongoing engagement with families in education/support activities was noted as a reason for improved rankings within this component item. This comment from one of the services describes how changes made within the service provision of training and embedding processes—improved selfmanagement education and support.

Good self-management education and support by staff with relevant training and skills. Engagement of families in education/support activities becoming routine practice. Use of resources to support self-management becoming routine practice. Behavioural interventions by staff with relevant training and skills becoming part of routine practice. Promotion and support for peer support is becoming central, strategic part of care. (Service 2)

## Links with Community, Other Health Services, and Other Services

Examination of the rankings given across the five services within this component item indicated that most of the selected services reported an improvement over time (**Figures 1**–**5**). However, two of the services reported improvements from initial SAT processes, but then, more recent SATs showed a contraction (**Figures 1** and **3A**). Many of the identified weaknesses and strengths associated with this component item were attributed to the systems in place (or lack of) to facilitate engagement with communities and other service providers. **Table 4** shows that sites 2 (T2DM, child and preventive health) and 4 (preventive and maternal health) reported significantly improved links with the community and other services scores from the first cycle to the final cycle.

### Communication and Cooperation on Governance and Operation of the Health Center and Other Community Based Organizations and Programs

The use of community or external linkages to inform service planning varied between services but generally strengthened over time. Weaknesses reported by the health services included a lack of community and client feedback and formal agreements in place for collaboration with other services. This comment from Service 1 is an example of limited communication and cooperation in relation to health service planning and governance.

No community input to governance, no client involvement in planning and feedback, no formal agreements with other services, and client satisfaction rarely assessed. (Service 1)

Over time, all services developed partnerships and communication with other services and community groups. Health services reported strengths such as having formal agreements with organizations, and systematically collecting and using client feedback to inform service planning.

Community input to governance is good. Service population involvement in planning and feedback is becoming systematic. Assessment of client satisfaction becoming systematic and routine. Formal agreements with other services with very good communication and levels of activity. Partnerships with community groups are very good. Health orientation of community programs is very good. (Service 2)

However, some services reported lower rankings of this component item in more recent SATs indicating that more work was required in maintaining the process of engagement with the community and other services.

## Linking Health Center Clients to Outside Resources

The extent that health services linked clients to outside resources was initially low across all services. Weaknesses cited by the health services were related to not having up-to-date referral directories, limited use of the referral directories, and linkages were not well integrated into staff orientation and training. Service 4 cited this component item as an area for improvement and part of their improvement plan.

Included on business plan as an area to improve on. Limited links, some referrals. Some links with QUIT, Healthy Living NT – but no directory present, random, when needed. (Service 4)

Over time services developed a comprehensive, updated and accessible resource directory, and linkages were included in staff orientation and training, and clients were regularly linked to outside resources. The comment below from one health service describes steps made to improve this component item.

Arrangements for linking clients to outside resources becoming systematic, comprehensive resource directory with good updating accessibility and use, and fair integration of linkage arrangements in staff orientation or training. (Service 1)

### Working in the Community

Initially, health service staff work in the community was minimal. Staffing constraints, a high workload, and minimal staff engagement with health promotion and development activities were cited as weaknesses, as illustrated by the comment from Service 3.

Midwife and [health service] staff trained in Core of Life for teenagers but difficult to deliver program due to high workload. (Service 3)

Over time, working in the community became part of most of the health services' core business. Four of the five health services engaged in community health promotion and development activities, community activities had become integrated into the health service program, and outreach into schools and community education days were occurring. This comment from Service 5 describes some simple steps taken to improve their working in the community.

Staff frequently visit families at home to discuss their kids - all the staff know everyone in the community - go to crèche to talk to mums & kids (Service 5)

### Communication and Cooperation on Regional Health Planning and Development of Health Resources

The extent to which the health services contributed to regional planning and resource development improved over time. Initial SAT data indicated that services had no or minimal engagement in regional health planning or resource development, nor local planning.

No or minimal engagement in regional planning, no or minimal contribution to the development of resources, no or minimal use of community plans (Site 1)

Improvements in rankings for involvement in regional planning were reported in later SATs. Strengths include engagement in regional planning, writing and reviewing regional protocols, representation on regional interagency committees, planning partnerships, and consultation in resource development. Service 3 reported in an early SAT that they were "*only involved in local planning, not regional*"; however, a more recent SAT indicated their involvement had increased.

Strong planning and involvement through partnerships and [Service 3] Futures Forum. Development of contextually appropriate health resources noted. (Service 3)

#### Organizational Influence and Integration

Rankings for this component generally indicated improvements over time for most services. One service reported improvement in two of the SATs; however, the most recent SAT showed a contraction (**Figure 1**). Another service reported no changes to the ranking over time (**Figure 2**). Qualitative SAT data suggested that weaknesses and strengths of organizational influence and integration were related to adequate funding, appropriate staffing levels, and conditions of work. **Table 4** shows increases in organizational influence and integration scores are nearing statistical significance at sites 3–5. Although changes in scores did not reach statistical significance, it is probable that increased scores represent clinical significance in terms of improvement.

#### Organizational Commitment

Early SATs indicated that organizational structures and processes that promoted safe, high-quality care were constrained by funding issues and staffing levels. The key weaknesses were reported as staff recruitment and retention, which affected staff workload, training and professional development opportunities, and staff morale.

Staffing levels don't meet the client's needs. No specific funding or job description. (Service 4)

With adequate funding and staffing levels, rankings of component item organizational commitment improved. Reasons cited for improvements included manageable workloads, availability of training and professional development opportunities, and communication and staff morale improved.

Plans in place; level of commitment is good. Specific funding, level is fair and/or short term. Level of staffing is good; most roles defined and reflected in job descriptions. Relationships and communication are very good. Morale is very good. Range of training and in-service opportunities is very good. Range of service delivery strategies is good. (Service 2)

### Quality Improvement (QI) Strategies

The key with QI strategies related to participation in QI processes and support. Reported weaknesses in early SATs include issues with the QI process itself, consistent use of QI processes, and limited support from senior staff. The comment from Service 4 is an example of issues raised in terms of preparing staff for CQI and how the CQI process was conducted.

Participation by staff was limited due to lack of training. Staff were expected to review their own processes, the rotating roles within the improvement process was not ideal. Electronic systems were not fully integrated, however incident reporting processes were systematic but no feedback/outcomes. (Service 4)

Later SATs in two health services identified a whole team approach to conducting clinical audits and SATs as a strength. Other strengths included systematic processes for CQI reporting, regular QI education and training, participation in QI collaboratives, and regular assessment of performance against key performance indicators. Examples of activities are provided below.

One21seventy audits and systems assessment undertaken as a team QI activity. (Service 3)

Participate in collaboratives (though time is an issue), ABCD audits, SAT workshops; everyone involved. (Service 5)

#### Integration of Health System Components

Key weaknesses of integrated health system components mainly related to staffing and resources issues. At one service, staffing, training, and resource constraints limited the provision of an integrated service.

Recruitment and retention are key issues, IT support from Broome is poor, lengthy delays for new staff to get IT access, and IT access problems at remote sites. (Service 3)

For other services, weaknesses related to IT systems were limitations.

Limited work outside of HC, within community. Information systems not optimal. Business plan reflects the need for partnerships. (Service 4)

Over time, three services reported a good or high standard of integrated service, while a fourth service recognized the importance of integration of service for effective and culturally appropriate care and were working toward this goal.

### DISCUSSION

Evaluation of approaches to CQI is crucial given the rapid growth in the available research on methods of QI and the variability in responses to quality programs (21–23). This paper explored the SAT data from five high-improving Indigenous primary health-care services to understand systems used to support quality care.

Using these data, staffing and support for staff were most commonly reported as influencing the component of delivery systems design. Issues of recruitment and retention also impacted on team work and delivery of quality health care, but these issues resolved over time with adequate staffing levels. Conversely, issues with recruitment and retention of staff led to instances where there was an improvement within this component in one SAT followed by contraction in the following SAT. Leadership and support was limited by remoteness and staffing constraints, and in the absence of adequate staffing or support, innovative solutions were found. An appointment system was viewed as either a strength or a weakness depending on whether clients used the system or preferred a drop-in approach. Scheduling was a strength with regularly planned clinics, activities, programs, and specialist visits. Care planning was constrained by the tools available for planning and the capacity of clinicians to undertake planning activities. Processes in place to support care planning varied across the health services, were dependent on staff availability but in general improved over time. Similarly, a systematic approach to follow-up relied on information systems and adequate staffing levels. Communication between clinicians, visiting staff, the hospital, and other health-care providers was a key strength contributing to continuity of care. The strengths of client access and cultural competence were not only physical access but also cultural awareness, culturally safe practice, and cultural respect. Physical infrastructure impacted negatively on cultural competence when the physical space could not accommodate culturally safe practice.

Within the component of information systems and decision support, information structures such as ECLs, were strengths in the planning and delivery of care when they were current, regularly updated and used as a planning tool. The use of evidencebased guidelines was a strength for clinical decision-making when they were available, accessible, and staff trained in their use. Where information structures and evidence-based guidelines were identified as weaknesses, later SATs revealed changes adopted to ensure they were updated and embedded in practice. Relationships, communication, and support were reported key strengths to coordinate delivery of care with other care providers.

In terms of self-management, identified weaknesses included staff with relevant training, skills, and time to undertake selfmanagement needs assessments, education, and support and to engage with families. Over time, the weaknesses were reported as strengths when staffing issues resolved and training was put in place.

The extent to which the health services used external linkages to inform service planning, linked clients to outside resources, worked out in the community, and contributed to regional planning and resource development varied between services but generally strengthened over time. PHC services developed partnerships and communication with other services and community groups and used client feedback to inform service planning. Working in the community became part of the health services' core business. PHC services engaged in community health promotion and development activities, and integrated outreach and education into their programs. Engagement in regional planning and resource development increased over time. Staffing and training constraints, systematic reporting processes, and IT issues were weaknesses identified in assessment of organizational influence and an integrated health system. Over time, services were able to resolve these issues and report either a fully integrated health system or progress toward this goal. Two of the services reported weaknesses in later SATs due to external factors impacting on how they worked with other organizations. Evidence from the free text responses indicated a transition period in the setting up of MOUs with other organizations and a review of how feedback from the community was collected were the reasons for the lower scores.

Similar to other component items, inadequate staffing levels and availability of funding were identified as weaknesses in organizational influence and integration. Later SATs showed that when funding and staff levels were addressed, services reported improved ranking of this component.

Our findings indicate that the challenges facing Indigenous PHC services such as lack of service infrastructure, recruitment, retention, and support for staff, and additional costs remain. This is so even in these services selected on the basis of "highimprovement" suggesting a high level of functioning and leadership. The selected high-improving PHC services operate within a complex system responding to different and changing contexts. Despite this complexity, a number of key supportive factors were identified such as adequate and orientated workforce, appropriate health system supports including supportive IT systems and relational factors such as communication and engagement with other organizations and community. Analysis of data collected over time also highlighted the utility of the SAT to help Indigenous PHC services identify areas for change, implement improvements, and monitor those changes over time.

### Strengths and Limitations of the Study

One of the strengths of this study is the availability of longitudinal quantitative and qualitative SAT data from the five selected case study services. Longitudinal quantitative data represented by the scores on the radar plots showed changes in each of the systems components over time. The accompanying qualitative data provided justifications for the SAT scores. The use of both approaches, together with our in-depth knowledge of each service and how QI works from the parallel multiple case studies, makes it possible to capture rich contextual information which in turn can increase understandings of why components are given a particular score. The availability of data over time, showing changes in scoring and justifications why those changes occurred, can allow for some discussion on the degree of amenability to change of each of the important factors. Overall, this increases our knowledge of the extent to which particular factors or conditions can impact on other components of the health system. The inclusion of qualitative and quantitative data from five case study services allows for examination of similarities and differences, which can increase the dependability (24) of the findings.

Our analysis is based on pre-existing SAT data, supplemented by additional information about how the SAT process works. As described earlier, each component of the SAT is assessed in the team meeting and a score is allocated. Health service staff record justifications for the scores as free text. One of the limitations of the data set is that the information was drawn from participating health service staff, providing an incomplete view of the health service; even in those participating, some providers may be more dominant than others. Furthermore, the use of pre-existing SAT data limited the collection of demographic information about the health professionals and their clients involved in the process. The score components (quantitative data) are agreed by a group of staff members from the service with a facilitator. One limitation is that we are unable to correlate their perceptions with actual health system performance retrospectively. In addition, in some services, the SAT process was conducted by an external facilitator and others solely by heath service staff, which may have impacted on how the process was conducted. Generalization of the findings from these five services in northern Australia to other services and contexts must be done cautiously.

### Discussion of the Findings in Relation to Other Relevant Research

Our analysis provides further evidence about the multifaceted contexts within which Indigenous PHC services are operating and significance of these contextual conditions in terms of how they might impact on QI processes. The documentation of rich contextual information allows for a greater understanding of how context and processes might influence QI and goes some way to explaining the variability in responses to particular interventions which may have proved successful in one setting but less successful in another. Two examples reported in our data are (i) difficulties in employing staff and how these impact the provision of health services; or (ii) adaptations made to an appointments system to ensure greater acceptance by the community. Schierhout et al. (23) also identified the complex interaction between context, CQI implementation and variability in responses to CQI. Øvretveit and Gustafson (25) argued that attention to the wider context in implementation of QI interventions "allows exploration of whether and how aligned changes at different levels may result, through complex influences, in better outcomes and how these can be sustained …. this in turn allows decision makers to assess better likely results locally and how to adapt the change" (pi22).

There is general consensus within the literature in terms of the organizational factors influencing successful QI (8, 26). Engels et al. (27) identified five domains of quality in general practice: infrastructure, staff, information, finance, and quality and safety. Similarly, the SAT framework gives due attention to such factors. Our case study data from these same services showed that within the Indigenous PHC service setting, conditions such as adequate staffing levels with strong, supportive clinical leadership in addition to the provision of appropriate orientation, and ongoing training were key strengths for a prepared workforce. Health service system factors were also identified in terms of embeddedness of appropriate, up-to-date and flexible systems to support the planning and delivery of care. Si et al. (21) found that patient-level characteristics contributed substantially to variation in processes of care and suggested that health-care providers need to strengthen their efforts to deliver care and to manage services in a way that most effectively meets the varying needs of individual patients. Our identification of relational factors such as building relationships and regular communication with clients, other health-care providers, and the wider community echoes the review of the literature by Crossland et al. (28), which reported patient-centered approaches, the importance of community, and communication as being integral to high-quality general practice. Furthermore, a report on stakeholder views on strategies for improvement in chronic illness care for Aboriginal and Torres Strait Islander people (29) called for greater partnership working with other health services and more effective links with communities.

Analysis of the SAT data provides evidence of these highimproving health services engaging with the QI process and making changes over time as a result of this engagement. Qualitative data provide valuable insights into reasons why SAT scores changed and the strategies put in place which may have influenced the change. Recent work conducted by Cunningham et al. (18), on the application of SAT data in PHC services, found that respondents reported changes in their health services as a result of using the SAT tool and valued the tool as a lever in implementing improvement. Indeed, Schierhout et al. (23) also found potential causal linkages between CQI activities and outcomes that were achieved. Proposed mechanisms were that the process allowed for identification of issues and prompted change or, alternatively that it, provided evidence and explanations for why things were improving (23).

### Implications of Findings

This study adds to the existing literature on the application of the SAT within an Indigenous PHC setting. The utility of the SAT for CQI is demonstrated through the availability of rich information, which can support service providers in identifying areas of their health system that facilitate QI and increase understandings of how components of the health service interact. Learning from the strengths of high-improving services and identification of what services can do to mediate quality health care may be useful for services striving to improve the quality of care provided in other areas.

### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of James Cook University (H6264), Queensland Health (HREC/14/QCH/12-890), Northern Territories Department of Health and Menzies School of Health Research (2014-2299), Western Australian Aboriginal Health Ethics Committee (615), and Western Australia Country Health Service (2015:09) with written informed consent from all subjects. All the subjects gave written informed consent in accordance with the Declaration of Helsinki.

### AUTHOR CONTRIBUTIONS

The study was conceived and designed by CW, SL and RB. CW, SL, KC, ST, KT, VM and RB contributed to the data collection, analysis, and interpretation; contributed to the synthesis of the findings reported here, critically revised the manuscript for important intellectual content, approved the final version, and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

### ACKNOWLEDGMENTS

This project would not be possible without the active support, enthusiasm and commitment of participating health services and the contribution made by the Centre for Research Excellence— Integrated Quality Improvement.

### REFERENCES


### FUNDING

This project was funded by the Australian National Health and Medical Research Council (ID 1062377).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at http://journal.frontiersin.org/article/10.3389/fpubh.2017.00045/ full#supplementary-material.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Woods, Carlisle, Larkins, Thompson, Tsey, Matthews and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Interactive Dissemination: Engaging Stakeholders in the Use of Aggregated Quality Improvement Data for System-Wide Change in Australian Indigenous Primary Health Care

*Alison Laycock\*, Jodie Bailie, Veronica Matthews and Ross Bailie*

*Menzies School of Health Research, Charles Darwin University, Casuarina, NT, Australia*

Background: Integrating theory when developing complex quality improvement interventions can help to explain clinical and organizational behavior, inform strategy selection, and understand effects. This paper describes a theory-informed interactive dissemination strategy. Using aggregated quality improvement data, the strategy seeks to engage stakeholders in wide-scale data interpretation and knowledge sharing focused on achieving wide-scale improvement in primary health-care quality.

#### *Edited by:*

*Diane Cooper, University of Western Cape, South Africa*

#### *Reviewed by:*

*Hanani Tabana, University of the Western Cape, South Africa Godisang Bridget Mookodi, University of Botswana, Botswana*

#### *\*Correspondence:*

*Alison Laycock alison.laycock@menzies.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 02 March 2016 Accepted: 18 April 2016 Published: 03 May 2016*

#### *Citation:*

*Laycock A, Bailie J, Matthews V and Bailie R (2016) Interactive Dissemination: Engaging Stakeholders in the Use of Aggregated Quality Improvement Data for System-Wide Change in Australian Indigenous Primary Health Care. Front. Public Health 4:84. doi: 10.3389/fpubh.2016.00084*

Methods: An iterative process involving diverse stakeholders in Australian Aboriginal and Torres Strait Islander health-care delivery uses aggregated audit data collected across key areas of care. Phases of reporting and online feedback are used to identify: (1) priority areas for improvement; (2) health center, system, and staff attributes that may be important in addressing the identified priority evidence-practice gaps; and (3) strate-

gies that could be introduced or strengthened to enable improvement. A developmental evaluation is being used to refine engagement processes and reports as the project progresses.

Discussion: This innovative dissemination approach is being used to encourage widescale interpretation and use of service performance data by policy-makers, managers, and other stakeholders, and to document knowledge about how to address barriers to achieving change. Through the developmental evaluation, the project provides opportunities to learn about stakeholders' needs in relation to the way data and findings are described and distributed, and elements of the dissemination strategy and report design that impact on the useability and uptake of findings.

Conclusion: The project can contribute to knowledge about how to facilitate interactive wide-scale dissemination and about using data to co-produce knowledge to improve health-care quality.

Keywords: dissemination, knowledge translation, stakeholder engagement, quality, quality of care, primary health care, Indigenous, improvement

**Abbreviations:** ABCD, audit and best practice for chronic disease; CQI, continuous quality improvement; ESP, engaging stakeholders in identifying priority evidence-practice gaps, barriers and strategies for improvement; PHC, primary health care.

## BACKGROUND

Integrating theory when developing and evaluating complex quality improvement interventions can help to explain clinical and organizational behavior, inform strategy selection, and understand effects – thereby developing generalizable knowledge (1, 2), shortening the time needed to identify conditions required for success and optimizing intervention design (3). Researchers and practitioners need to make explicit the theories used (3, 4), as specifying the logic behind continuous quality improvement (CQI) research and practices assists replication and adaptation (5).

Implementation research suggests that by using evidence to identify and link priority gaps in care to theoretical domains that are known to be system enablers or barriers, strategies can be developed that will most likely produce the desired change (6–8). Improvement strategies are more likely to succeed if barriers to effectiveness are identified and addressed at the outset (9, 10).

Despite this evidence, there are few examples of how to select and apply theory when developing implementation interventions (11), and limited examples in the literature of how to choose strategies to overcome barriers to implementing care guidelines (12). This paper describes a wide-scale knowledge translation strategy that draws on implementation theory on addressing barriers to improving health care, to implement what we have termed "interactive dissemination."

Dissemination is often linked to implementation of research findings, where interventions aim to reduce or remove barriers and promote change – Hailey and colleagues highlight the challenges of matching research findings to the wider perspectives or requirements of groups being addressed (13). Our "interactive dissemination" strategy is consistent with definitions of dissemination as knowledge transfer and exchange, in which there is interactive exchange between researchers and those they intend to influence and an intention to provide and use information as input to decisions or policies leading to change (13–16). The strategy design synthesizes and translates evidence relevant to the CQI program, supports understanding and use of data, and draws on practical knowledge to identify strategies aligned with implementation settings. These elements are identified as necessary for bridging the "how to" gap between dissemination of evidence and implementation in practice (17, 18). Our interactive dissemination strategy, thus, contributes to co-production of knowledge (19, 20), which is inherent in our CQI approach within a national research partnership (21, 22).

### Collaborative Knowledge Production and CQI

There is recognition of the value of collaborative knowledge production processes through which researchers and service providers share explicit and tacit knowledge to find practical, contextually relevant strategies to improve care quality and health outcomes (23, 24). Such processes have potential to help bridge the enduring gap between recommended practice and care delivered (23, 25, 26). Gaps in care provision that occur across multiple health centers are likely to be due to inadequacies in the broader primary health-care (PHC) delivery system. Improving care quality requires change in approaches that operate at multiple levels of the health system and recognize their interdependencies (27). Stakeholders working at different system levels can help in identifying and addressing inadequacies, sharing knowledge to strengthen systems to achieve wide-scale improvement in care delivery, thereby reducing inequities in health-care access and outcomes between population groups (21, 28, 29).

Continuous quality improvement activities are widely implemented in PHC. Typically characterized by feedback of systematically collected data, adaptation to local conditions and involvement of participant leaders, they use iterative processes and recognized change methods (e.g., Six-Sigma, Plan-Do-Study-Act cycles) (30). The participatory nature of CQI enables teams to draw on context-specific and experiential knowledge to develop improvement strategies. There is limited understanding of how these CQI principles and processes can be applied at scale to achieve system-wide improvement.

### An Interactive Dissemination Strategy Using Aggregated CQI Data

The dissemination strategy uses CQI data from a program of CQI research and development in Aboriginal and Torres Strait Islander (Australia's Indigenous peoples) PHC in Australia (**Box 1**). Data contributed over 8 years by 175 health centers delivering care to Indigenous people are aggregated at the national level, and at the Australian state/territory level where sufficient data are available. They comprise clinical audit data on adherence to best practice guidelines representing 56,000 patient records, and data from 492 systems assessments completed by health teams, in priority aspects of PHC. Evidence on this scale enables identification of gaps in care that occur across health centers, and offers a foundation for developing evidence-informed policies and programs to achieve high-level system change and large-scale improvement.

### Engaging Stakeholders in the "Identifying Priority Evidence-Practice Gaps, Barriers and Strategies for Improvement (ESP)" Project

The purpose of the interactive dissemination strategy – the ESP project – is to engage stakeholders working in Indigenous PHC

#### BOX 1 | The Audit and Best Practice for Chronic Disease (ABCD) National Research Partnership.

In 2010, the ABCD National Research Partnership brought together PHC services, policy, and support organizations and research institutions to guide and support research in improving the quality of Indigenous PHC across Australia (22). Concurrently, the National Centre for Quality Improvement in Indigenous Primary Health Care (www.One21seventy.org.au) was established to provide tools, processes and training to support CQI and strengthen the implementation of clinical care guidelines. Almost 80% of health centers using One21seventy services agreed to share their de-identified CQI data for research purposes, forming the most comprehensive and broad-scale dataset relating to health center performance currently available for Indigenous PHC.

Partnership research has highlighted wide variation in performance between different aspects of care and between health centers. While many aspects of care are delivered well in many health centers, there are important gaps between evidence and practice in some aspects of PHC (31–33).

delivery, management, policy, and research with these aggregated CQI data in order to:


Targeted stakeholders include health practitioners (e.g., doctors, nurses, allied health professionals, Indigenous Health Practitioners), managers and policy-makers working at various levels of the health system, researchers, staff of health service support organizations, and peak bodies representing the interests of Indigenous communities and community-controlled health services.

#### Context

Indigenous Australians experience an inequitable burden of illhealth, shorter life expectancy and poorer access to health services compared with the general population (34, 35). Contributing factors are complex, relating to colonization and discrimination, social and economic inequalities, and cultural safety. Indigenous Australians access PHC through Indigenous communitycontrolled health and government-managed services designed to meet their needs (36), and through private general practices. Indigenous PHC settings are diverse in geography, governance, and resource provision, and characterized by complex political, cultural, and social interactions.

Continuous quality improvement activities are implemented in many PHC centers that serve Indigenous people, for example, through use of audit and system assessment tools, and Plan-Do-Study-Act approaches. A national CQI framework for Aboriginal and Torres Strait Islander PHC (37) is being established. In this complex health-care environment, it is important to build on strengths and existing knowledge, making optimal use of CQI data and research to help address health inequities.

### METHODS

#### Theoretical Framework

The ESP project design is adapted from systematic methods designed to link interventions to modifiable barriers to address evidence-practice gaps. French and colleagues designed a fourstep process comprising guiding questions to identify: (1) an evidence-practice gap, and what needs to be done differently by whom to reduce it; (2) barriers that should be addressed by intervention strategies, based on previously tested theoretical domains relevant to behavior change of health-care professionals (7, 8); (3) intervention components that could overcome the barriers and enhance enablers, and; (4) how behavior change can be measured and understood (6). French et al.'s process has provided the theoretical base for the design of the ESP project, which is guided by the questions: "What are the priority evidence-practice gaps evident in the aggregated CQI data?" "Which barriers and enablers need to be addressed?" "Which strategies could overcome modifiable barriers and enhance enablers?" and "How can we improve dissemination methods to encourage engagement with the data and use of findings?"

### Iterative Participatory Approach

The ESP project uses an iterative and participatory approach. Drawing on action research principles, cycles of systematic enquiry, collaboration, and refinement are applied for the purpose of effecting change (38) and developing theoretical understanding (39).

Separate ESP processes are implemented using audit data collected for child health, chronic illness care, rheumatic heart disease, preventive, maternal, and mental health care. Each process comprises four phases of reporting and stakeholder feedback, culminating in a final report. Each phase comprises a report and linked online survey that uses Likert-scale and openended questions to elicit interpretive and reflective responses. We distribute the reports by email to people in partner organizations and extended networks, encouraging further distribution, discussion, and facilitated group input. The survey tool distinguishes between individual and group responses.

#### Phase 1

The first report includes the most recent available CQI data in one aspect of PHC delivery (e.g., child health), aggregated and presented as box and whisker plots with interpretive information and preliminary analysis. This analysis is done by the research team, in collaboration with clinical experts, to identify priorities for improvement. Through the phase 1 survey, we seek feedback on the preliminary priorities, whether they align with respondents' pre-existing perspectives on priorities for improvement and whether other priorities should be included.

#### Phase 2

The second report includes the findings from the phase 1 survey (consensus evidence-practice gaps) and trend data over time and by audit cycle for indicators relevant to the identified improvement priorities. We ask respondents to reflect on the trend data and their experience, and answer survey questions to rate potential barriers to improving the priority gaps experienced at different levels of the health system, including system factors relevant to the Indigenous PHC sector (40). Listed domains relating to health center systems, the broader system environment, and staff attributes are drawn from international and national research (7–9, 40, 41). Questions about barriers and enablers relating to individual attributes are informed by the Theoretical Domains Framework (7, 8, 42, 43). Respondents are also asked to rate the accessibility, usability, and usefulness of the report and suggest improvements.

#### Phase 3

The third report includes the Phase 2 findings and a summary of published evidence about successful strategies used in CQI, which is intended to stimulate thought and discussion about possible strategies for improving care. We use the Phase 3 survey to find out how stakeholders think existing strategies could be refined, or new strategies developed, to build on system strengths and enablers and overcome the main barriers to addressing the priority evidence-practice gaps. Respondents are also asked if the report provides a fair reflection of the main barriers and enablers to improvement in relation to the priority evidence-practice gaps, and how the report could be refined.

#### Review

The team incorporates feedback to develop and distribute a draft report of the overall project findings and invites project participants to comment on the representation of findings using a brief online survey. Responses are used to finalize the ESP report in that particular aspect of PHC.

The purpose and elements of ESP phases are summarized in **Figure 1**.

#### Concurrent Developmental Evaluation

A developmental evaluation is being conducted to refine the ESP project structure, materials, and processes as it progresses. One member of our research team (AL) has the lead role on the evaluation, which is expected to contribute to the team's learning and the project's dissemination goals. The evaluation method and resulting project modifications will be described separately.

### DISCUSSION

#### Use of Aggregated Data for Wide-Scale Quality Improvement

There is need for innovative dissemination approaches that encourage use of service performance data by policy-makers, managers, practitioners, and community members to identify and address barriers to achieving change. Researchers need to be involved in dialog with these groups to understand policy contexts and how evidence may translate into action (44), and to plausibly link the development of scientifically sound advice with knowledge exchange processes (45). A recent systematic review found timely access to good quality relevant research evidence, collaborations, relationship- and


FIGURE 1 | Phases of the ESP project. Note: this process is repeated for each area of care (e.g., child health, chronic illness care). Source: Matthews et al. (26). PHC, primary health care; CQI, continuous quality improvement; ESP, "Engaging Stakeholders in Identifying Priority Evidence-Practice Gaps, Barriers and Strategies for Improvement (ESP)" Project.

skills-building to be important factors influencing policymakers' use of evidence (46). An Australian review found limited evidence that managers and policy-makers could use to assess the impact of system- and service-level attributes on health outcomes for Indigenous peoples, concluding that more mixed-method research that includes multiple stakeholder perspectives, including those of Indigenous community members, is required (28).

Continuous quality improvement programs typically bring health teams together to plan evidence-informed improvements utilizing clinical data and contextual knowledge to address local evidence-practice gaps in care. In this large-scale project, the challenge of engaging people in "discussion" about care quality based on aggregated data is heightened by limited opportunities for face-to-face or individual-level communication between research team members and stakeholders. Research is needed to determine how CQI processes can be scaled up for higher-level policy and management purposes. It stands to reason that interpretation and use of aggregated CQI data and input by stakeholders in varying roles has potential to identify common and important improvement priorities, and to utilize the collective strengths within PHC services to continue improving health-care quality for Indigenous Australians.

### Opportunities for Learning about What Works in Dissemination and Knowledge Co-Production

Through the developmental evaluation, the team expects to learn more about stakeholders needs and preferences in relation to the way data and findings are described and distributed, and elements of the dissemination strategy and report design that impact on the usability and uptake of findings (47) – including the use of implementation science theory. There is a positive correlation between stakeholder engagement in knowledge production and implementation (23). We hope that by developing understanding of factors that impact on stakeholder participation in the project, and gathering feedback about how to better capture and present stakeholder input, we can contribute knowledge to strengthen the design and impact of knowledge translation processes.

The project should assist in understanding the potential and limitations of online communication to engage health-care stakeholders in wide-scale interactive dissemination processes.

### Opportunities for Learning about What Works to Improve PHC Systems and Quality

The input provided by stakeholders on barriers and enablers, and on strategies for improvement, is valuable in that it reflects tacit knowledge of people working within the health system. We have made innovations to an existing implementation tool used for exploring individual attributes that influence care. Additional questions in the tool are designed to capture knowledge about determinants of performance that operate at health center and system levels (40, 41). This exploratory work may inform further studies in health systems and implementation research, including the development of tools to identify barriers to improvement at multiple system levels.

The CQI process used to assess health center systems includes a domain about community linkages (48). A priority for improvement reflected in the aggregated system assessment data is the strengthening of links between health centers and Indigenous communities. Related enablers identified through the ESP process to date include strengthening of community engagement in service delivery design and community leadership for CQI (26). The ESP process has a higher system focus than the health center CQI process; therefore, input from Indigenous peak bodies is important for achieving linkages to influence policy and program design at higher system levels.

The design of wide-scale improvement strategies in the Australian Indigenous PHC context needs to reflect understanding of the holistic nature of Indigenous wellbeing beyond physical health, including healthy connections to culture, community, and land, as well as published evidence and expert knowledge. Findings relating to identified barriers, enablers, and strategies will be reported separately.

Documentation and evaluation of implemented strategies will contribute knowledge about what works and in what contexts to improve PHC for Indigenous communities, and will support adaptation to other settings.

### Strengths and Limitations

A strength of the ESP project is its iterative design using multiple phases. In conjunction with the developmental evaluation, implementing a new dissemination process with each PHC audit tool dataset provides the team with multiple opportunities to reflect and respond to stakeholder feedback, drawing on evidence and available resources to make and test refinements to processes, reports, and supporting materials. To our knowledge, the level of detail of the data made available from a large number of services across wide geographic scope through this project has not been achieved by other projects.

We are using an open process to engage stakeholders, inviting those who receive reports to distribute them online through their workplaces and networks, and respond to surveys individually or through groups. This strength in the project design puts no limit on the number and diversity of possible participants, thereby enhancing data interpretation and enriching knowledge sharing. We encourage peak bodies representing Indigenous communities to use the reports as a basis for group discussion, enabling further opportunity for community members' input.

The open process also makes it difficult to assess reach and response rates relative to distribution. On balance, the advantages of this snowballing distribution process outweigh the potential limitation in relation to accurate reporting of survey distribution and responses, as a goal of the project is to provide wide-scale access to these CQI data and ESP project findings.

## CONCLUSION

The ESP project uses an innovative theory-informed approach to advance the use of large-scale aggregate CQI datasets, enabling a range of stakeholders to identify priority gaps and related barriers in the delivery of best practice PHC in Indigenous communities. Using aggregate CQI data to stimulate discussion among diverse stakeholders on priority evidence-practice gaps in care, and how best to achieve improvement, will contribute knowledge about how to facilitate interactive dissemination and data use.

This process will identify major themes for improving PHC delivery through changes at the health center and community, regional, and national levels. We expect common themes identified across key areas of PHC to be relevant to developing policy and implementing large-scale change to strengthen systems and improve the provision of comprehensive PHC for Indigenous communities across Australia. We anticipate that lessons learned about applying theory to inform the development of improvement interventions, and engaging stakeholders in use of aggregated CQI data for knowledge co-production and system-wide change, will be transferable to other settings.

### ETHICS STATEMENT

Ethical approval for the ABCD National Research Partnership was obtained from research ethics committees in each relevant Australian jurisdiction – Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research (HREC EC00153), Central Australian Human Research Ethics Committee (HREC-12-53), New South Wales Greater Western Area Health Service Human Research Committee (HREC/11/GWAHS/23), Queensland Human Research Ethics Committee Darling Downs Health Services District (HREC/11/ QTDD/47), South Australian Aboriginal Health Research Ethics Committee (04-10-319), Curtin University Human Research Ethics Committee (HR140/2008), Western Australian Country Health Services Research Ethics Committee (2011/27), Western Australia Aboriginal Health Information and Ethics Committee (111-8/05), and University of Western Australia Human Research Ethics Committee (RA/4/1/5051). All participants in the ESP Project surveys and evaluation provide individual informed consent.

### REFERENCES


## AUTHOR CONTRIBUTIONS

AL planned and wrote the manuscript. JB provided input and commented on all drafts. VM contributed to conception of the ESP project, led the quantitative analysis of the ABCD data and commented on the draft. RB is the leader of the ABCD National Research Partnership, of which the ESP project is a dissemination strategy. He played a lead role in conceptualization of the ESP process, and contributed to conceptualization and review of the manuscript. All authors read and approved the final manuscript.

### ACKNOWLEDGMENTS

The development of this manuscript was made possible through the active support, enthusiasm, and commitment of members of the ABCD National Research Partnership, especially the participating primary health-care centers that have provided the de-identified data used in the ESP dissemination strategy. The authors gratefully acknowledge Dr. Frances Cunningham, Professor Gillian Harvey, and Dr. Nikki Percival for their comments on the manuscript.

### FUNDING

The ABCD National Research Partnership Project has been supported by funding from the National Health and Medical Research Council (545267) and the Lowitja Institute, and by inkind and financial support from a range of Community Controlled and Government agencies. Alison Laycock is supported by a National Health and Medical Research Council Postgraduate Scholarship (1094595), and by the Centre of Research Excellence: An Innovation Platform for Integrated Quality Improvement in Indigenous Primary Health Care (CRE-IQI, funded by the NHMRC ID 1078927). Ross Bailie's work has been supported by an Australian Research Council Future Fellowship (100100087). The views expressed in this publication are those of the authors and do not necessarily reflect the views of the funding agencies.

*Int J Occup Saf Ergon* (2013) **19**(1):41–62. doi:10.1080/10803548.2013.1107 6965


professional practice and health care outcomes. *Cochrane Database Syst Rev* (2010) (3):CD005470. doi:10.1002/14651858.CD005470.pub2


illness care in Australia's Northern Territory – use of the Chronic Care Model. *BMC Health Serv Res* (2008) **8**:112. doi:10.1186/1472-6963-8-112

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Laycock, Bailie, Matthews and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Enabling Continuous Quality Improvement in Practice: The Role and Contribution of Facilitation

*Gillian Harvey1,2 \* and Elizabeth Lynch1,3*

*1Adelaide Nursing School, University of Adelaide, Adelaide, SA, Australia, 2Alliance Manchester Business School, University of Manchester, Manchester, UK, 3 Florey Institute of Neuroscience and Mental Health, Parkville, VIC, Australia*

Facilitating the implementation of continuous quality improvement (CQI) is a complex undertaking. Numerous contextual factors at a local, organizational, and health system level can influence the trajectory and ultimate success of an improvement program. Some of these contextual factors are amenable to modification, others less so. As part of planning and implementing healthcare improvement, it is important to assess and build an understanding of contextual factors that might present barriers to or enablers of implementation. On the basis of this initial diagnosis, it should then be possible to design and implement the improvement intervention in a way that is responsive to contextual barriers and enablers, often described as "tailoring" the implementation approach. Having individuals in the active role of facilitators is proposed as an effective way of delivering a context-sensitive, tailored approach to implementing CQI. This paper presents an overview of the facilitator role in implementing CQI. Drawing on empirical evidence from the use of facilitator roles in healthcare, the type of skills and knowledge required will be considered, along with the type of facilitation strategies that can be employed in the implementation process. Evidence from both case studies and systematic reviews of facilitation will be reviewed and key lessons for developing and studying the role in the future identified.

#### *Edited by:*

*Sarah Larkins, James Cook University, Australia*

#### *Reviewed by:*

*M. Rashad Massoud, University Research Co., USA Janya McCalman, Central Queensland University, Australia*

#### *\*Correspondence: Gillian Harvey*

*gillian.harvey@adelaide.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 20 December 2016 Accepted: 09 February 2017 Published: 22 February 2017*

#### *Citation:*

*Harvey G and Lynch E (2017) Enabling Continuous Quality Improvement in Practice: The Role and Contribution of Facilitation. Front. Public Health 5:27. doi: 10.3389/fpubh.2017.00027*

Keywords: facilitation, facilitators, continuous quality improvement, implementation, context

## INTRODUCTION

Quality improvement interventions are typically complex and multifaceted, encompassing both technical and social components (1). As such, whether and how well they work depends on the dynamic interplay between the components of the improvement interventions, the people involved, and the organizational and health system context in which they work. Understanding and managing these complex interactions presents a challenge to those charged with the responsibility for designing, planning, implementing, and evaluating quality improvement initiatives in the real world (2).

With the growing science of healthcare improvement, knowledge has advanced in a number of key areas. First, there is a greater understanding of the components to combine together within an improvement intervention. For example, activities centered around audit and feedback, goal setting, and the use of Plan-Do-Study-Act (PDSA) cycles can help to establish a clear need for improvement, guide the direction of travel, and enable progress toward agreed aims to be monitored (1, 3). Second, our understanding of the influence of contextual factors on the way that improvement programs play out in practice has increased substantially (4). Various frameworks have been developed and applied to help assess and make sense of these contextual factors at a program, organizational, and wider system level (5, 6). Third, acknowledging the complexity of implementation and the fact that "context matters" (7), the need to tailor improvement interventions accordingly is recognized (8). This raises an important question—and one that we address in this paper—namely, how best to achieve effective facilitation of improvement interventions that are tailored to take account of the specifics of the improvement project, the contextual setting, and the individuals and teams involved.

### TAILORING TO CONTEXT—WHO AND HOW?

Some contextual factors are amenable to modification, whereas other factors must be identified and "worked around." Typically, factors within the inner context, such as the extent of local leadership support or employee motivation to engage in improvement, can be addressed with specific strategies. However, wider contextual factors at the health system or policy level need to be acknowledged as influential, but are unlikely to be amenable to change. Therefore, as part of planning and implementing an improvement intervention, it is important to assess and build an understanding of the contextual factors that are likely to influence implementation, both positively and negatively. This includes looking at factors such as the organizational culture and receptiveness to new ideas and change; what the past experience of improvement has been; whether leaders (clinical and executive) are supportive; how much authority people have to make decisions and introduce new ideas; and whether resources will be available to support the introduction of the proposed improvements. On the basis of this initial assessment, it should then be possible to design and implement the intervention in a way that is responsive to contextual barriers and enablers—or, in other words, to "tailor" the improvement program. However, even with prior knowledge and understanding, achieving effective tailoring in practice can be challenging (2). Having some form of human component within the process is seen as beneficial to actively manage the barriers and enablers that are encountered (9). Various roles are described in the implementation and improvement literature, for example, knowledge brokers and boundary spanners (10), opinion leaders (11), academic detailers (12), improvement coaches (13, 14), and educational outreach visitors (15), to name just a few. While terminology varies and often overlaps, there are some distinct differences in terms of how specific roles function (16). For example, opinion leaders typically operate through peer influence and being accepted as a credible expert; academic detailers employ social marketing techniques; improvement coaches emphasize goal setting and self-reflection; and educational outreach encompasses learning processes.

Given the complexities associated with quality improvement and the challenge of tailoring improvement interventions appropriately to context, the question could be which of the roles described above are the best ones to employ in which particular setting or situation? Or is it more a case of knowing which combination of roles to bring together and how? In attempting to address these questions, we draw on our own experiences of applying facilitation roles and processes in quality improvement in healthcare, supported by empirical evidence from the literature. We begin by defining what we mean by facilitation and how it relates to other roles that support quality improvement. We then consider how facilitator roles can be employed in quality improvement initiatives and what existing research can tell us about the effectiveness of the role and key issues to consider. We conclude by outlining what we see as fruitful questions for further study.

### DEFINING FACILITATION AND THE FACILITATOR ROLE

Taken literally, the word facilitation means to "make easier." As a concept, facilitation began to emerge in healthcare in the latter half of the twentieth century, influenced by humanistic principles of participation, engagement, and shared decision-making and enabling others (17). This philosophy aligns well with the principles underpinning continuous quality improvement (CQI), influenced by the early work of theorists such as Deming and Juran, who emphasized the importance of employees taking responsibility for quality, rather than being subject to performance assessment and inspection by managers (18).

Facilitator roles in primary healthcare improvement are evident from the 1980s onward, supporting a range of initiatives in screening and prevention [see, for example, Ref. (19–21)]. Our own experience of facilitation and facilitator roles similarly has its roots in quality improvement (22), and more recently, in the related field of knowledge translation and implementing evidence-based healthcare (23, 24). In the widely used Promoting Action on Research Implementation in Health Services framework, facilitation has been defined as the active ingredient that aligns the proposed innovation or improvement to the individuals and teams involved and the context in which they work, thereby enabling successful implementation (25). In order to operationalize facilitation, it is important to develop and support individuals in facilitator roles to act as the human agent in implementation, whether they are facilitating a new innovation in practice, the implementation of evidence-based clinical guidelines or agreed goals for practice improvement (25).

A question frequently posed is "who can take on the facilitator role"? The quick answer is that there is no single job specification. Facilitators can be internal or external to the organization, from a clinical or non-clinical background, and be operating at different organizational levels from a clinical team through to the wider health system level. The key is that they meet the requirements of the role, in terms of their personal attributes, knowledge, and skills. Commonly described personal characteristics of facilitators include being empathetic, sensitive, flexible, pragmatic, authentic, credible, resilient, and passionate (26, 27). Alongside these individual attributes, typical skills required include a mix of technical and process skills as facilitators characteristically occupy a hybrid role, balancing the achievement of improvement goals with the development of effective teamwork processes and building improvement capacity in individuals. This requires skills in project management and improvement methods, combined with skills in interpersonal communication, group processes, negotiation, and empowering others (28). It is increasingly clear that the role—and the range of skills and knowledge required—is typically too complex for lone individuals to take on and that in order to manage large improvement programs, a network of facilitators with varying levels of expertise and responsibility is required (16).

#### HOW IS FACILITATION DIFFERENT TO OTHER ROLES?

The literature illustrates many examples of facilitators working in different settings on a wide range of different improvement initiatives. These include improving the management of nutrition in hospital (29), reducing neonatal mortality in a rural community (30) and improving chronic disease management in primary care (31), to name just a few examples. The main distinguishing feature of the role is the focus on enabling, as opposed to persuading, influencing, directing, or coercing. The facilitator does not control or mandate what needs to be done, but rather helps the individuals in an improvement team to work collaboratively to agree areas for improvement and create and sustain change in healthcare provision. Clearly there may be overlap with other change agent roles and a close working relationship with formal and informal leaders is essential (32); however, the facilitator typically takes on a more generic, coordinating role that involves working with and through others to achieve improvement. In turn, this helps to embed capacity for change within teams and organizations by influencing workplace culture, and empowering and upskilling team members to facilitate change (33, 34).

### HOW DO FACILITATORS ENACT THE ROLE?

In fulfilling a role that involves enabling teams to achieve their improvement goals, the facilitator employs a range of facilitation methods and processes. Some of these are more directly concerned with the improvement task (for example, setting goals and agreeing audit measures); others focus on managing the process (for example, establishing ground rules for the improvement team, responding to contextual barriers, and managing conflict). Facilitation is an iterative process, but comprises a number of core elements. A typical facilitator's "toolkit" includes attention to issues such as clarifying and engaging stakeholders; undertaking a baseline assessment; planning and implementing; and reviewing, sharing, and recognizing success (35). **Table 1** summarizes the type of activities that the facilitator is likely to be engaged in to address these issues.

#### WHAT EVIDENCE IS THERE FOR FACILITATION?

Baskerville and colleagues undertook a systematic review of practice facilitation in primary care. From analysis of the 23

TABLE 1 | Example facilitation strategies [adapted from Ref. (35)].


included studies, the review concluded that practices supported by facilitators were 2.76 times more likely to adopt evidence-based clinical guidelines (36). In all 23 studies, facilitators used audit and feedback as part of their implementation strategy. Other facilitation strategies included interactive consensus building and goal setting, reminder systems, tailoring to context, and the use of improvement tools such as PDSA. Analysis also indicated that tailoring the interventions to the context and needs of the practice led to significantly more positive effects when compared to studies that did not tailor the interventions. When facilitation interventions were provided at higher intensities (more frequently and/or for longer periods of time), the effectiveness was significantly greater.

More recent evidence supporting the impact of facilitation comes from a large cluster-randomized trial in Vietnam. Intervention groups received support from trained lay workers as facilitators of quality improvement in community groups and demonstrated a significant reduction in neonatal mortality after 3 years (37). This study also highlighted the importance of the skills of the facilitator, demonstrating that those teams that received the highest level of facilitation (as judged by a rating of facilitator skills and effectiveness) achieved greater improvements in neonatal mortality (30).

As facilitation is becoming more commonly applied and evaluated in healthcare quality improvement and implementation, then the knowledge base will continue to grow. For example, studies are reported in the literature comparing different methods of facilitation (38) and examining the more detailed mechanisms through which facilitation operates (33). However, there are still important questions that need to be considered and addressed, as we outline in the following section.

### LESSONS LEARNED AND FUTURE DIRECTIONS

Cross-case comparisons of facilitation studies highlight a number of important lessons for others considering this approach within an improvement program (23). These include the importance of balancing a structured approach to project management with flexibility and responsiveness to issues that arise during the course of the project. Marshall and colleagues have recently described their experience of trying to design an effective improvement intervention (2). Despite careful attention to the design process, including starting with an initial program theory, only three of the nine components of their planned intervention were implemented in line with the original proposal. This clearly demonstrates the way that complex interventions "morph" and the real challenges involved in tailoring improvement initiatives in real time. Other learning focuses on the need for realistic expectations and allowing sufficient time to see impact, as the study of neonatal mortality in Vietnam illustrated; here, it was not until the third year of the facilitation intervention that significant improvements were observed (37). This same study also demonstrates the importance of selecting people with the right knowledge and skills into the facilitator role (30). Other research highlights relationship building and the need for leadership championing and support (34). While there is no "one-size-fits-all" specification for the facilitator role, there are some guiding principles around the need for creating a supporting environment for facilitators, including mentoring arrangements for those who are new in the role (29).

So where next in terms of facilitation and facilitators in healthcare quality improvement? We would suggest that there are a number of areas that would be beneficial to explore further around the role and its contribution to enabling CQI.

1. Achieving clarity around the core elements of facilitation and clearly defining what comprises (or not) a facilitation

#### REFERENCES


intervention. This will help in delineating facilitation from other commonly applied improvement and implementation roles; equally it can guard against the role being inappropriately described or applied (39).


Further advancing our understanding of facilitation processes and facilitator roles will help to improve the prospective design and application of improvement interventions and our ability to tailor interventions to context.

### AUTHOR CONTRIBUTIONS

GH and EL equally contributed to the planning and writing of the manuscript; both reviewed and approved the final submission.

in healthcare quality improvement. *BMJ Qual Saf* (2012) 21(1):13–20. doi:10.1136/bmjqs-2011-000010


*A How-to Guide*. Rockville, MD: Agency for Healthcare Research and Quality (2011). AHRQ Publication No 12-0011.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Harvey and Lynch. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Improving Community Health Using an Outcome-Oriented CQI Approach to Community-Engaged Health Professions Education

#### *Amy Clithero1 \*, Simone Jacquelyn Ross2,3 , Lyn Middleton3,4, Carole Reeve5 and Andre-Jacques Neusy3*

*<sup>1</sup> Family and Community Medicine, University of New Mexico School of Medicine, Albuquerque, NM, USA, 2College of Medicine and Dentistry, James Cook University, Townsville, QLD, Australia, 3 Training for Health Equity Network, New York City, NY, USA, 4School of Health Sciences, University of KwaZulu-Natal, Durban, South Africa, 5School of Medicine, Flinders University, Alice Springs, NT, Australia*

#### *Edited by:*

*Edward Broughton, University Research Co., USA*

#### *Reviewed by:*

*Emmanuel D. Jadhav, Ferris State University, USA Afolaranmi Olumide Tolulope, Jos University Teaching Hospital, Nigeria*

> *\*Correspondence: Amy Clithero aclithero@salud.unm.edu*

#### *Specialty section:*

*This article was submitted to Public Health Policy, a section of the journal Frontiers in Public Health*

*Received: 12 December 2016 Accepted: 09 February 2017 Published: 27 February 2017*

#### *Citation:*

*Clithero A, Ross SJ, Middleton L, Reeve C and Neusy A-J (2017) Improving Community Health Using an Outcome-Oriented CQI Approach to Community-Engaged Health Professions Education. Front. Public Health 5:26. doi: 10.3389/fpubh.2017.00026*

Health professionals providing health-care services must have the relevant competencies and clinical experiences needed to improve population health outcomes in different contexts. Current models of health profession education often fail to produce a fit-for-purpose workforce ready and willing to provide relevant, quality care to underserved communities. Evidence is emerging that community-engaged and socially accountable health workforce education, i.e., aligned with priority health needs, produces a workforce ready and willing to work in partnership with underserved regions. This model of education fosters greater affiliation between education and service delivery systems and requires institutions to measure graduate outcomes and institutional impact. The Training for Health Equity Network (THEnet), a partnership of socially accountable health workforce education institutions, has developed and tested a Social Accountability Framework for Health Workforce Education (the Framework) and toolkit to improve alignment of health workforce education with outcomes to assess how well education institutions meet the needs of the communities they serve. The Framework links education and service delivery creating a continuous quality improvement feedback loop to ensure that education addresses needs and maximizes impact on the quality of service delivery. The Framework also provides a unifying set of guidelines for health workforce policy and planning, accreditation, education, research, and service delivery. A key element to ensuring consistent high quality service delivery is an appropriately trained and equitably distributed workforce. An effective and comprehensive mechanism for evaluation is the method of CQI which links the design, implementation, accreditation, and evaluation of health workforce education with health service delivery and health outcomes measurement.

Keywords: social accountability, health professional education, continuous quality improvement, health workforce, accreditation

## INTRODUCTION

The World Health Organization (WHO) estimates that an additional 2.4 million doctors, nurses, and midwives are needed globally but nowhere near enough are being trained, particularly in the areas where they are needed the most; but increased numbers of health professionals is insufficient (1). They need to be equitably distributed, competent to meet the needs of their communities, and be motivated and empowered to deliver quality care that is appropriate and acceptable to the sociocultural needs of the population (2). It is well documented that poverty and social inequity are the most important determinants of ill health worldwide (3, 4). Many, if not all, intractable health problems have as their root cause social determinants, including health inequities between economic and ethnic groups and poor access to health care. Health professionals have a responsibility to address social inequity and its deleterious effects on individual and population health. An appropriately trained and evenly distributed health workforce is essential to reduce the health equity gap within and across borders and to achieve universal health coverage (UHC) and meet the Sustainable Development Goals (SDG 3) (http://www.who.int/topics/ sustainable-development-goals/targets/en/).

The Independent Global Commission on Education of Health Professionals for the twenty first century identified that "*glaring gaps and inequities in health persist both within and between countries* … *and* … *professional education has not kept pace with these challenges*" (4). They acknowledge that the problems are systemic and require a new era of health professional education. Specifically, they classified three successive levels of learning for students to build their knowledge, skills, attributes, and values for how to become a health system change agent. These are:


The Global Strategy on Human Resources for Health: Workforce 2030 recommends education strategies include social accountability (SA) approaches to ensure a better distribution of health workers where they are most needed, emphasizing the underserved and most vulnerable populations (2). This is echoed by the recently released report of the High Level Commission on Health and Economic Growth stressing that socially accountable education should be institutionalized emphasizing the role of training institutions in addressing population and health system needs (5).

Further, there is a growing global consensus recognizing the importance of holding health professional schools accountable to society for achieving these goals. The WHO defines SA as "*the obligation to orient education, research, and service activities towards priority health concerns of the local communities, the region and/ or nation (schools) one has a mandate to serve. These priorities are jointly defined by government, health service organizations, and the public"* (6). The WHO is not alone in recognizing SA as a critical mandate. The 2010 global consensus on SA document, reflecting the agreement of 130 organizations and individuals from around the world involved in health education, professional regulation, and policy-making, called for schools "*to reorient their education, research, and service priorities*" (7) to improve their response to current and future health-related needs and challenges in society. This requires health professional schools to shift their traditional education model toward a socially accountable approach. Despite this, there are limited practical tools to guide health professional schools to transform their curriculum and measure their impact on health outcomes.

### The Training for Health Equity Network

Driven by both their implicit and explicit social mission to address the needs of their communities, a number of schools of medicine and health sciences in high and low resources countries have embraced this challenge by successfully incorporating SA as the central tenet of their mission. Their success in producing graduates with broader and relevant competencies and distributed equitably in geographically isolated, underserved regions led to the development of the Training for Health Equity Network (THEnet). THEnet was founded in 2008 and is an international collaboration of 12 health professional schools committed to SA mandates to direct their educational, research, and service resources toward the priority health and health system needs of their reference populations (8). The first priority of action for THEnet was to develop and test a Framework for Socially Accountable Health Workforce Education (the Framework) to assist health professional educational schools measure their progress toward SA (9). The Framework was informed by Boelen and Woollard's three "expressions of social accountability" namely: "*conceptualization (the type of professional needed and the system that will utilize his or her skills), production (the main components of training and learning) and usability (initiatives taken by a school to ensure that its trained professionals are put to their highest and best use)*" (10). Following its publication in 2012, the Framework has been used by a growing number of health professional schools across the world to evaluate their curriculum, or discuss opportunities for education or policy change (11–14).

### SA Health Professional Education and Continuous Quality Improvement (CQI)

Continuous quality improvement as a process method can be used to continually improve the quality of student learners over time which is a step beyond quality assurance which can be viewed as simply producing technically competent graduates. CQI is a set of principles, concepts, and methods adopted originally in the business world and subsequently introduced to other areas including the higher education sector (15). Quality improvement processes build on quality assurance systems in higher education ensuring quality of teaching and learning and providing public accountability for the standards of programs and the use of resources by meeting accreditation standards (16). Traditionally, higher education quality assurance systems such as accreditation bodies focus more on educational processes than on outcomes and impacts of their graduates and research on societal issues and communities they serve (17). THEnet Framework bridges this gap by aligning education processes with the impact of the graduates in the communities. The THEnet iterative CQI model (**Figure 1**) shows a CQI cycle of health professional education, research, and service within a traditional CQI structure of Plan—Do—Study—Act.

### METHODS

The Framework identifies key factors for schools to educate a health workforce to positively influence health outcomes and health systems performance and provides the training and tools to measure and improve the outcomes across institutions and context. The Framework was developed using a logical framework matrix (NORAD 1999), a well-tested project planning and evaluation tool (18). The Framework helps schools evaluate how well they are doing in terms of meeting priority needs and assists to establish educational improvement and areas for research *via* the four sections of the Framework. Each section addresses an element of the CQI cycle by asking practical questions linked to stages of the quality improvement process. The four sections are (1) what needs are we addressing? (2) how do we work? (3) what do we do? and (4) what difference do we make? These four sections of the Framework inform each of the other sections and provide strategies for transformational learning to generate future cycles leading to continuous improvements over time.

### Creating a Health Professional Education Curriculum to Meet the Health Workforce Needs

THEnet's iterative CQI model (**Figure 1**) and the four sections of the Framework have been linked to showcase how to create

a health professional education curriculum to meet health workforce needs with a continuous evaluation process.

#### Assessment and Identification: Section 1: What Needs Are We Addressing? and Section 2: How Do We Work?

Section 1: the first step is to examine and determine if there is a strong alignment between the school's community needs and the desired graduate competencies. A socially accountable health professional curriculum considers the geographical region the school serves, communities that have difficulty accessing health services, or have poor health outcomes in the region. Inclusion is a quality CQI step, which means involving key stakeholders including community members in the design of a curriculum for buy-in and quality graduate attributes. Other stakeholders such as learners, educators, community members, health service providers, management, and government also bring different perspectives, knowledge, and necessary information to the process.

Section 2: spend quality time ensuring that the learners, educators, leaders, and key stakeholders are aware of vulnerable populations and underserved communities in the region and can identify their priority health and social needs. This knowledge must then become embedded in the curriculum, and be geared toward transformational learning to produce graduates with the competencies and commitment to address identified priority health, cultural, and social needs of the communities they serve with a focus on the underserved.

#### Deliver: Section 3: What Do We Do?

A socially accountable health professional curriculum considers, what, how, and where do our learners learn, and embeds the values of quality, equity, relevance, and efficiency. It also considers who the educators are and how are they trained, and governance needs such as how resources are managed for program operationalization so they are distributed according to priority needs.

The socially accountable curricula emphasize the principles of primary health care, and integrate basic and clinical sciences with population health and social sciences. The second quality improvement step specific to the curriculum should include who will do the curriculum review, what specifically will be done and when will it be done. Consideration also needs to be given to any tools and training that may be needed. For example, if a change is needed, consideration needs to be given as to whether it is feasible to make the change in terms of cost, time, and resources and if there is buy-in by key decision makers.

Undertaking a comprehensive curriculum review can be daunting so think about the one change that might be worth undertaking. For example, review what your learners learn from your curriculum. Suggested indicators from Section 3 of the Framework include (19):

• Does your education program, including curriculum content, reflect identified priority health, cultural, and social needs of the community?

	- desired graduate attributes based on the above needs assessment?
	- principles of primary health care? and
	- integration of basic and clinical sciences with population health and social sciences?

#### Evaluate: Section 4: What Difference Do We Make?

The next stage is monitoring the curriculum for impact. Before evaluating, consider the processes, strategies, outcomes, and the impact that curriculum reform will have on the systems, communities, and individuals it serves. To help with this process, a program logic model can be developed to identify if the needs will be met (20). As an example, THEnet program logic model (**Figure 2**) outlines THEnet's socially accountable health professional education (SAHPE) philosophy, activities, outcomes, regional impacts, and long-term goal of health equity and improved health outcomes. Other evaluation tools can include student satisfaction surveys with learning, graduate competency surveys within the health system (patient and supervisors), and faculty satisfaction with institutional support for undertaking curriculum initiatives toward SA surveys.

Impact factors might include a broader study of improvement in population health derived from local health surveys and statistics, improvement in health workforce numbers and retention across the region, and number of research publications and conference presentations of socially accountable projects by faculty and students, as well as tracking graduate specialty and practice location to determine if they match the priority workforce needs of the community served.

#### Adjust

The final stage of a CQI process is acting upon what is learned and adjusting governance, education, research, and services accordingly and informs the next iteration of the quality improvement cycle. CQI is an ongoing process and each school must continue

FIGURE 2 | Theory of change for how socially accountable health professional education (SAHPE) institutions collaborating under THEnet contribute to health equity (28).

to examine their underlying assumptions, and be proactive and responsive to changing needs and demands.

### DISCUSSION

THEnet recognizes that SA is a fundamental principle that requires a flexible approach in how it is operationalized. The Framework is a generic tool and can be used in phases and allows for creativity and adaptability to different contexts and different resource availability. Over the past 4 years, THEnet schools realized the benefit of the Framework, not only for curriculum transformation but also as a mechanism for CQI for the production of a fit-for-purpose workforce to improve local health outcomes.

A significant investment is required by health professional education institutions and society to develop health professionals who have both technical expertise and professional values that include a service orientation and ethical commitment to not only their individual patients but the communities in which they practice. SA is a principle that translates into educational strategies resulting in value-based competencies that are best demonstrated by health professionals who act as change agents in partnership with their communities. THEnet iterative CQI model links the educational and service delivery systems. As a practical CQI tool, THEnet Framework on SA has filled an important gap and is currently being used by a growing number of health professional education institutions around the world. We propose integrating the Institute of Medicine's definition of CQI with health professional education by including education as a strategy for improving health-care services and by describing targeted patient groups as communities: *"Quality improvement consists of systematic and continuous actions that lead to measurable improvement in education, health care services and the health status of targeted patient groups [communities]"* (21).

### REFERENCES


### CONCLUSION

A CQI approach is useful for understanding and monitoring socially accountable educational mechanisms that lead to fit-forpurpose graduates and improved quality of care and health outcomes at a population level. SA in health professional education is gaining traction internationally as a mechanism for combatting health inequities and advancing UHC (1).

Including SA indicators in health professional education accreditation standards would acknowledge the importance of holding health professional schools accountable to society for addressing population and health system needs (5). We call for key indicators around an appropriately trained and evenly distributed health workforce to be included in all health professional higher education accreditation processes. This is essential to reduce the health equity gap within and across borders and to achieve UHC and meet the Sustainable Development Goals (SDG 3) (http://www.who.int/topics/ sustainable-development-goals/targets/en/).

THEnet's Framework for Socially Accountable Health Workforce Education links education and service delivery creating a CQI feedback loop to ensure that education addresses needs and maximizes impact on quality service delivery. Evidence is emerging that community-engaged and socially accountable health workforce education produce a workforce ready and willing to work in partnership with underserved regions (22–27).

### AUTHOR CONTRIBUTIONS

All authors contributed to the writing and editing of the article. AC wrote the first draft and managed subsequent drafts with revisions from all other authors. SR wrote the second draft. CR and LM reviewed subsequent drafts and provided revisions. A-JN did the final review. All authors gave final approval of the publication of this version of the paper.


James Cook University medical graduates. *Rural Remote Health* (2014) 14:2657.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2017 Clithero, Ross, Middleton, Reeve and Neusy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Improving Health Promotion Using Quality Improvement Techniques in Australian Indigenous Primary Health Care

#### *Nikki Percival1 \*, Lynette O'Donoghue1 , Vivian Lin2 , Komla Tsey3 and Ross Stewart Bailie1*

*1Centre for Primary Health Care Systems, Menzies School of Health Research, Brisbane, QLD, Australia, 2Department of Public Health, School of Psychology and Public Health, LaTrobe University, Melbourne, VIC, Australia, 3 The Cairns Institute, James Cook University, Cairns, QLD, Australia*

#### *Edited by:*

*Colette Joy Browning, Royal District Nursing Service, Australia*

#### *Reviewed by:*

*Armin D. Weinberg, Baylor College of Medicine, USA; Rice University, USA; Texas A&M School of Public Health, USA Kathryn Welds, Cisco Systems, USA*

*\*Correspondence: Nikki Percival nikki.percival@menzies.edu.au*

#### *Specialty section:*

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

> *Received: 13 January 2016 Accepted: 14 March 2016 Published: 30 March 2016*

#### *Citation:*

*Percival N, O'Donoghue L, Lin V, Tsey K and Bailie RS (2016) Improving Health Promotion Using Quality Improvement Techniques in Australian Indigenous Primary Health Care. Front. Public Health 4:53. doi: 10.3389/fpubh.2016.00053*

Although some areas of clinical health care are becoming adept at implementing continuous quality improvement (CQI) projects, there has been limited experimentation of CQI in health promotion. In this study, we examined the impact of a CQI intervention on health promotion in four Australian Indigenous primary health care centers. Our study objectives were to (a) describe the scope and quality of health promotion activities, (b) describe the status of health center system support for health promotion activities, and (c) introduce a CQI intervention and examine the impact on health promotion activities and health centers systems over 2 years. Baseline assessments showed suboptimal health center systems support for health promotion and significant evidence-practice gaps. After two annual CQI cycles, there were improvements in staff understanding of health promotion and systems for planning and documenting health promotion activities had been introduced. Actions to improve best practice health promotion, such as community engagement and intersectoral partnerships, were inhibited by the way health center systems were organized, predominately to support clinical and curative services. These findings suggest that CQI can improve the delivery of evidence-based health promotion by engaging front line health practitioners in decision-making processes about the design/redesign of health center systems to support the delivery of best practice health promotion. However, further and sustained improvements in health promotion will require broader engagement of management, senior staff, and members of the local community to address organizational and policy level barriers.

Keywords: health promotion, quality improvement, Indigenous, primary health care, evidence-based program, feasibility, participatory action research

### INTRODUCTION

The disparities between the health status of Aboriginal and Torres Strait Islander people (Australia's Indigenous populations) and that of other Australians is unacceptable. Indigenous Australians have a life expectancy 10.6 and 9.5 years lower than that of non-Indigenous males and females, respectively, infant mortality is three times higher, and death rates are 1.6 times that of other Australians (1). Although there have been improvements in some social and health indicators (2), chronic diseases, such as cardiovascular disease, diabetes, and renal disease, remain significant contributors to premature and excess mortality and morbidity among Indigenous Australians.

Although the root causes of poor and inequitable health are related more to social, cultural, and environmental factors, the health sector is a vital determinant of health and plays a key role in promoting equity (3) and supporting action to address social determinants of health (4, 5). International experience has shown the positive effect of health systems based on equity, disease prevention, and health promotion in narrowing health inequities (6) and more specifically, in reducing Indigenous health inequities. For example, access to an integrated and comprehensive primary health care (PHC) system, with a strong primary and preventive focus, has been critical in delivering better health for Native Americans and the Maori people of New Zealand (7).

Comprehensive PHC services in Australia are best typified by the Aboriginal community controlled health services (ACCHS). These health services are designed to deliver holistic, comprehensive, and culturally appropriate health care for Indigenous Australians. The National Aboriginal Community Controlled Health Organisation (NACCHO) describes PHC as including not only the provision of medical care but also the provision of services, such as counseling, preventive medicine, health education and promotion, rehabilitative services, antenatal and postnatal care, and maternal and child care programs (8). Although health promotion is recognized as a core function, there has been little published research that has considered the health promotion work of these PHC centers (9).

In the Australian Indigenous PHC context, there is growing appreciation of both the need for and benefits of using continuous quality improvement (CQI) techniques to improve the delivery of a range of PHC services through an emphasis on organizing and strengthening fragmented health systems (10). Sollecito and Johansen (11) define CQI as "*a structured organisational process for involving staff in planning and executing a continuous flow of improvements to provide quality that meets or exceeds the expectations of customers.*" It involves designing and redesigning systems to meet customers' needs by testing and implementing ideas from evidence-based strategies, frontline staff, and customers. Although there has been substantial research on CQI in clinical health care in Australian Indigenous communities, the study of quality improvement in health promotion has been limited.

We conducted a 3-year study exploring the potential of CQI for improving health promotion in collaboration with Indigenous PHC centers in Australia's Northern Territory (NT). Combining the Ottawa Charter for Health Promotion's definition of "health promotion" and the NACCHO's definition of "health," we define health promotion as "the process of enabling Indigenous people to increase control over, and to improve, not just the physical wellbeing of the individual, but the social, emotional, and cultural wellbeing of the whole community in which each individual is able to achieve their full potential as a human being, thereby bring about the total wellbeing of their community" (8, 12). The term "health education" can sometimes be used synonymously with health promotion. To emphasize the distinction, and for the purpose of this study, we consider health education an important and common strategy in health promotion; defined as "the provision of education to individuals (through discrete planned sessions) or groups, with the aim of improving knowledge, attitudes, selfefficacy and individual capacity to change" (13) (p. 20). The study involved developing and implementing health promotion CQI tools and processes with the aim of assisting health center staff to design/redesign their health center systems and strengthening the development and delivery of local health promotion activities. Using the health promotion CQI tools and processes, the purposes of this study were to (a) describe the scope and quality of health promotion activities, (b) describe the status of health center system support for health promotion activities, and (c) examine the impact of a CQI intervention on health promotion activities and health centers systems after two annual cycles.

## MATERIALS AND METHODS

### Study Setting and Processes

Primary health care centers participating in this study are located in regional and remote Aboriginal communities in Australia's NT. The NT is in the central northern region of Australia and spans 1.3 million square kilometers, making it the third largest Australian federal division. However, it is sparsely populated, with an estimated population of 243,800, making it the least populous of Australia's eight states and territories (14).

The NT has the highest proportion of Indigenous Australian residents estimated at 68,850 or 29.8% of the total NT population compared to 3% of the total Australian population (15). About 90% of the NT's Indigenous populations live in discrete, remote communities (15). In remote communities, access to health care is predominately through Indigenous-specific PHC services, including Aboriginal community-controlled or state government PHC centers. There is seldom more than one PHC provider. This differs from most other Australians who can access PHC services through a fee-for-service sector based on general medical practice and a state government-funded and managed sector, which differs from state to state in its forms and functions.

Two separate but complementary governance structures were developed specifically for this study. A project management committee was established comprising the lead study investigators, members of the research team, and senior policy officers and managers from the Northern Territory Health Department (NTDoH). The project management committee was responsible for guiding the research processes and maintaining academic rigor. An Aboriginal and Torres Strait Islander Advisory Committee acted as a reference group for the study, assisting in the development of the data collection tools and facilitating health center and community engagement.

All research procedures related to this study were approved by the Human Research Ethics Committee of the NT Department of Health and Families and Menzies School of Health Research, and by Menzies Indigenous health research subcommittee. Formal participation agreements setting out the roles and responsibilities of the research team and those of health centers and staff (including issues relating to data collection and storage, confidentiality, intellectual property, and research dissemination) were negotiated and signed by health center management and, where appropriate, by health boards. Prior to each site visit, clearance was provided by the relevant Aboriginal community council and local health center.

### Development and Implementation of the CQI Intervention

The health promotion CQI intervention was delivered through an action research design, whereby health center staff, the research team, and other key stakeholders (e.g., Indigenous and non-Indigenous health promotion practitioners and policy officers) were involved in the development and refinement of existing CQI tools that have been extensively used for the improvement of clinical care in Indigenous PHC settings (16). Experience gained and the results of regular data feedback were used to continually revise and improve the tools and processes for health promotion. By drawing on principles of participatory action learning (17) and guidelines for the ethical conduct in Aboriginal and Torres Strait Islander health research (18), our approach aimed to maximize engagement of stakeholders at multiple levels, from local health center management and PHC staff to policy decision makers. The CQI intervention is summarized in **Figure 1**. It comprised annual cycles of health center systems' assessments and audits of local health promotion activities; data analyses and interpretation; feedback and local interpretation of results with participating health center staff; goal setting by health center staff to achieve system changes; action planning; and strategy implementation.

### Data Sources and Methods

The research team and local health center staff used the health promotion audit tool (available from: http://www.one21seventy. org.au/cqi-information/hp-cqi-tools) and audit protocol to review health center records of health promotion activities (for example, project plans, staff reports and presentations, and minutes of meetings) that had been implemented in the preceding 12 months for documentation of key aspects of health promotion planning, implementation, and evaluation. Audits were conducted at baseline and annually for the next 2 years. Details on the audit method have been described previously (19).

The audit and best practice in chronic disease (ABCD) systems assessment tool (SAT) (20) was adapted and used to guide the assessment of health center system support for health promotion. It comprised an interactive process whereby the research team engaged health center staff [health center managers, nurses,

Aboriginal health workers (AHWs), and doctors when available] in discussion and reflection on the strengths and weaknesses of how their health center is organized and functions. They were encouraged to consider the systems currently in place to support planning and delivery of health promotion and to comment on the successes and difficulties.

At the end of each data collection cycle, the research team drew on the findings to facilitate a structured reflection with the health center team about potential actions for system improvement. This feedback process usually occurred within 3 months of data collection and within a timeframe that enabled a response to influence the next cycle. This participatory feedback process was a key component of the action research methodology.

#### Analysis

The types of health promotion activities were categorized on a continuum from health education and skill development activities; health information and social marketing activities; community development activities designed to strengthen capacity of communities to address local health issues; and activities designed to improve healthy environments, settings, and to change socioenvironmental causes of disease (21). Improvements in health promotion activities over the study period (as represented by the number and percentage of activities that provided documentary evidence of key aspects of best practice) were assessed by comparing the Year 1 data (collected after one annual cycle) and Year 2 data (collected after two annual cycles) with baseline data. Matrix displays were used as a way of organizing and visualizing qualitative data collected through the SAT in a systematic way and comparing improvements over time and across participating health centers (22). The research team met to identify and discuss patterns and themes in the descriptions of systems and classified them as a strength (system working well) or weakness (system not working so well) for supporting health promotion for each component at participating PHC centers. These emerging findings were presented to health center staff during annual feedback workshops and to investigators and stakeholders at quarterly project meetings, where interpretations were discussed to check their validity. All health centers were provided final reports describing the results of the health promotion quality assessments and changes over time that are presented in this paper.

#### RESULTS

#### Characteristics of Participating Health Centers

From January 2008 to December 2010, four Indigenous PHC centers were engaged in the study. **Table 1** shows their diversity with regard to governance arrangements, geography, and population size. Three health centers are governed by a board of elected Indigenous community members (community controlled) and one health center is managed and operated by the NTDoH (government service). Two health centers deliver PHC services to populations of more than 1,000 people (but less than 5,000), one health center delivers PHC services to more than 5,000 people, and one delivers PHC services to a population of less than 500 people.

Each health center employs multidisciplinary teams of 5 to more than 50 staff including nurses, allied health workers, doctors, and AHWs. (AHWs are often recruited from their remote communities enabling local participation in the direction and delivery of health services. Although AHWs exist in all Australian states and territories, the title AHW covers many roles. In the NT, the title AHW is regulated in recognition of the specific scope of work practices, particularly clinical work, ensuring that work is carried out without risk to public safety.) With the exception of one health service, staff employed in these health centers were not specialist health promotion practitioners. One health center had appointed a health promotion coordinator approximately 6 months prior to the commencement of this study.

Data about the scope and quality of health promotion activities were gathered from a total of 51 health promotion activities across the four PHC centers. The number of activities audited in each of the 3 years was 8 in 2008 (baseline), 24 in 2009 (Year 1), and 19 in 2010 (Year 2). A total of 11 systems assessments were conducted over the course of the study. Each health center had an annual systems assessment, however in Year 2, the final round of data collection, one health center declined to participate in the SAT due to changes in management and staffing at that time. We facilitated annual feedback workshops with each health center. Some health centers were more proactive than others in documenting their action plans for improvement (see **Figure 1**, step 5).

#### Baseline Results

#### Scope and Quality of Health Promotion Activities

At baseline, the type of health promotion activity was dominated by health education (three activities) and information sessions (three activities), with less emphasis on community development activities (one activity), or activities designed to change socioenvironmental causes of disease (one activity). All health centers delivered activities to raise community awareness about the risks of smoking and other common chronic disease risk factors (nutrition, alcohol, and physical activity). Other targeted areas included diabetes, chronic obstructive pulmonary disease, and hypertension.

Written descriptions of activities in health center records were limited (two out of eight activities) and inadequate for collecting



*a Total population: estimates only (23).*

*bRemoteness (24) (i) access to community: all year by road; part year by road; and all year by air or sea (islands) and (ii) distance to urban center:* <*20 km by road; 20–100 km by road; 101–300 km by road; 301–600 km by road; and by air (islands).* information related to planning, implementation, and evaluation of health promotion activities.

#### Health Center System Design and Support for Health Promotion

Baseline systems assessments showed that health promotion was organized in a variety of ways in PHC centers. Health promotion was considered a discrete area of program service delivery, as a part of all service delivery, or the responsibility of individual members of staff. Some health center staff described visiting and external services as part of their "health promotion delivery team." Health promotion was articulated as a core function of PHC service delivery in some but not all health center strategic plans and/or mission statements.

Staff reported significant perceived weaknesses in health center system support for health promotion. Across health centers, the majority of staff held the view that staffing levels to support the design and implementation of health promotion activities was inadequate and that pressure from "the clinic" mitigated their ability to undertake health promotion. Staff reported their roles and responsibilities (including reporting and communication) for health promotion were not always clearly defined or perceived as an implicit part of their role.

Across and within each health center, staff had different perspectives of "health promotion" and how it is done. Health center staff used the phrases "health education" and "health promotion" interchangeably, as if they were one and the same. Health center staff commented that health promotion language or "jargon" was a major obstacle in understanding and applying principles and concepts in practice.

Systems (or alternative processes) that could be used to collate, report, and monitor local health promotion activities were limited. Existing service planning and monitoring systems did not reflect, or are only partially supportive of, health promotion. For example, patient information systems (either paper based or electronic) designed for delivery of clinical services and health care to individuals had limited capacity to support the design, implementation, and evaluation of health promotion activities.

#### Actions for Improving Health Promotion

Based on findings from the annual cycles, health center staff identified and implemented a range of actions to improve health center system support for health promotion. Summarized in **Table 2**, these actions address four broad system components.

#### Delivery System Design

To improve the way health promotion was delivered and by whom, staff initiated strategies to create greater clarity of individual roles and responsibilities in health promotion and improve communication about health promotion within the health center. For example, at health center D, health promotion was added as an agenda item at weekly staff meetings to improve communication about staff involvement in local activities. Health center A incorporated health promotion as part of each program area portfolio and health center B identified two AHWs to form a

#### TABLE 2 | Examples of actions implemented by health center teams to improve health center systems support for health promotion.

#### Delivery system design

Included "health promotion" as an agenda item at weekly staff meetings Arranged health promotion portfolios for all staff

Identified two Aboriginal health workers to form a health promotion team

Senior Aboriginal health worker designated as "broker" between the local community and health service

Appointed a staff member to coordinate training and professional development in health promotion for staff

#### Information systems and decision support

Created arch lever folders for storing documents and records for health promotion

Developed standardized planning templates and trialed quality improvement program planning system (QIPPS)

Used the Health Promotion Audit Tool as a "check list" for documenting practice

Community board representatives attend feedback sessions

Purchasing of best practice guidelines (e.g., *The Public Health Bush Book*)

#### Organizational environment

Workshops/trainings in health promotion made available to staff Results of health promotion audit presented to health board

Management quarantined time for staff to participate in health promotion CQI processes

Health board chair invited and participated in the CQI feedback workshop Involvement of external practitioners in health promotion CQI processes

#### Adaptability and integration of health system components

Create referral pathway in existing clinical information systems to capture group health education sessions

Sharing "good practice" health promotion plans across health center teams Health promotion officers from NT Department of Health support health service staff to access and use the quality improvement program planning system (QIPPS) to plan health promotion activities

Using clinical service data to develop health promotion project (e.g., storyboard for HbA1c)

health promotion team. These actions were designed to improve staff understanding of their roles and responsibilities in health promotion at their respective health centers.

#### Information Systems and Decision Support

"Information systems and decision support" refers to health center structures and processes that support planning, implementation, and monitoring of health promotion activities. This includes access to evidence-based tools and guidelines and systems for recording and monitoring health promotion activity. At baseline, systems or processes to record and monitor health promotion were lacking. Over the course of the study, health center teams introduced a range of systems to standardize documentation and recording of health promotion activities. For example, health centers C and D trialed the quality improvement program planning system (QIPPS) – an electronic program planning information system (see http://www.qipps.infoxchange.net.au). Health centers A and B introduced paper-based planning templates (using items in the audit tool) to document practice. Additionally, at health center A, staff trialed ways to document health promotion in their existing clinical information systems.

#### Organizational Environment

Actions to improve the broader health center environment for health promotion were focused on raising awareness and engaging senior staff and health board members in health promotion. Baseline analysis revealed that health center staff were aware of the importance of community involvement in health promotion activities; however, the mechanisms for supporting this action, such as community boards and advisory committees, did not exist or were not used for the purpose of strengthening health promotion at the health center. For example, three out of the four participating health centers are governed by a health board made up of community representatives. Results from systems analyses highlighted that some staff perceived the role of the board as decision makers regarding services "at the clinic" and had not considered their involvement or understood their role for supporting health promotion. Over the study period, staff invited board members to participate in data collection and feedback workshops. At health center D, a staff representative presented at a board meeting and discussed audit results with community representatives.

#### Adaptability and Integration of System Components

To ameliorate fragmented health center systems, staff identified a number of strategies to integrate and link different system components. For example, health center staff utilized data from other quality improvement initiatives to inform the development of local health promotion activities. At health center B, a number of patients with type 2 diabetes were identified as having elevated levels of glycated hemoglobin (HbA1c). Based on the traditional ways of storytelling [see "Chronic Disease Storyboard" in Laycock et al. (25)], AHWs developed health education sessions to raise community awareness and understanding of sugar consumption and its impact on the development and management of diabetes.

### Changes in Scope and Quality of Health Promotion Activities Overtime

**Table 3** presents a summary of the aggregated audit data across the four PHC centers. Following the introduction of the CQI intervention, we observed improvements in aspects of planning, implementation, and evaluation of health promotion activities. However, within and across health centers, the type of health promotion activities and targeted health issues remained largely unchanged over the study period.

When compared with baseline, health centers had improved documentation plans for their health promotion activity from 2/8 (25%) at baseline to 17/19 (89%) at Year 2. Improvements in documenting aspects of planning and evaluation were also found. At Year 2, 14/19 (74%) health promotion activities had recorded an activity goal; 15/19 (79%) had recorded the activity target group; 16/19 (84%) recorded strategies for implementing the activity, and for 11/19 (58%) health promotion activities there was evidence of activity evaluation.

Prior to the CQI intervention, recorded participation of community people in health promotion activities was low (1/8; 13%). At Year 1, improvements in community involvement was TABLE 3 | Results from audits of health promotion activities at baseline, Year 1, and Year 2 across four participating health centers [figures are number and percentage (%) of activities].


noted with a record of community participation in 9/24 (37%) activities. There was no further change in recorded community participation in health promotion activities at Year 2 (7/19; 37%).

Our findings also highlight the extent to which health center teams worked with other organizations. At baseline, of the eight health promotion activities, only one recorded involvement of other organizations (13%) in the planning, implementation, and evaluation of health promotion. Of the 24 activities in Year 1 and 19 activities in Year 2, partners were involved in just over half (54%) of the activities in Year 1 and slightly more activities (63%) in Year 2. Partnerships were with organizations from the health sector (for example, state government, national government, and non-government health services and health-related aid organizations).

## DISCUSSION AND CONCLUSION

Our study showed that the introduction of structured and facilitated quality improvement cycles can improve health center systems and quality of health promotion activities in Indigenous PHC centers. At baseline, we found that PHC centers undertake health promotion activities, but what health promotion has done was often not recorded, or when documented, the information was scarce and not comprehensive. We also found that the activities were largely dominated by lifestyle advice and education approaches, and responding to growing levels of chronic disease was the focus of their efforts. Community participation in the planning, implementation, and evaluation of activities was also limited and showed little improvement over time.

The CQI intervention appears to have contributed to an increase in the number and quality of health promotion activities by improving health center staff health promotion capacity and system development and functioning. For example, through PHC, staff are better able to articulate their health promotion work and, subsequently, identify and document health promotion activities; improve systems for recording health promotion activities, thereby enhancing availability of data; and, subsequently, improve workforce capacity to deliver health promotion activities over the study period. The health promotion audit tool is based on health promotion planning and evaluation frameworks and, as such, appears to have assisted health centers to reflect upon the extent to which they incorporate and, subsequently, document these concepts and principles into their health promotion activities.

Limited attention to areas of system development that support aspects related to the "process" of health promotion may provide some explanation for limitations in delivery and recordings of other areas of health promotion activity quality. For example, records of community participation in health promotion activities improved from baseline to Year 1, but no further improvements were achieved in Year 2. Although it may be desirable to improve the ecological approach of health promotion activities (26, 27), the health promotion skills and expertise and the time required to effectively and meaningfully engage community people and partners in this process is likely to be well beyond currently available capacity in the PHC centers. Thus, the importance of a coordinated and partnered approach to health promotion in these communities becomes even more critical if health promotion is to be effective, and action on the social determinants of health is to be realized. A potential approach for coordinating health promotion activities is through the inclusion of relevant stakeholders, such as representatives of the governing health board, other organizations and agencies in the community, and visiting services, throughout the CQI intervention. This would help to avoid duplication of effort and improve local planning processes for health promotion.

Another key finding of this study is that participating PHC health centers focused actions for systems improvement on transactional system change; that is, the day-to-day operations of the organization (28). Improvements were most seen in areas of delivery system design and information and decision support, such as through the introduction of standard templates for recording health promotion activities, purchase of resources to guide practice, and creation of team portfolios for health promotion. Although these are necessary and important system improvements for supporting health promotion activities, broader transformational changes that are more closely linked with leadership, vision, organizational culture, and external environments are necessary if health promotion is to be a core component of PHC service delivery (28). Organizational change of this nature is possible as has been demonstrated for diabetes care (20) and for health promotion (29, 30). For these health centers to become more health promoting, actions to improve systems related to the organizational environment are an important area of influence and for future consideration.

Even though Australian Indigenous PHC centers have been described as exemplary models of comprehensive PHC (31) and that health promotion is a recognized core function of NT PHC centers (32), our findings, together with other studies of health promotion in PHC settings (9, 33, 34), suggest that health centers struggle to implement health promotion as a core component of their service delivery. Notwithstanding health center agreements to participate in this study, and with the exception of a few staff directly involved in health promotion activities, overwhelmingly, health center staff felt overloaded with issues of patient care and delivery of clinical services. This was evident from SAT data but also expressed by the lack of attendance of some health center managers and senior staff in the CQI process. The disparate availability and/or allocation of resources, including health promotion positions or skilled staff, available time, and funding, further suggests that health promotion is not an integral component of PHC service delivery. However, what is clear from our study is that participation in the CQI process gives health center teams' dedicated time to discuss and reflect on health promotion in their health center. With a better understanding of what constitutes good practice and knowledge of health systems that support optimal practice, health center teams can not only build new systems but also identify the potential of existing structures. Even under challenging circumstances, health center teams can take small, incremental steps toward establishing partnerships with local organizations and engage community in aspects of health promotion planning, implementation, and evaluation. This is particularly important in resource constrained environments.

We acknowledge that the use of health center records for assessment purposes may have limited the number of health promotion activities included in the study and recognize that the health center teams may be involved in other activities not captured here. However, we have used a variety of strategies to strengthen our findings, including the use of multiple methods and data sources, and seeking validation of our analysis with health center teams and project stakeholders throughout the study.

Availability and quality of health promotion records created challenges for conducting audits of health promotion and have flow on effects for health promotion practice, including the ability to monitor and to evaluate the impact of health promotion activities. The lack of documentation of health promotion activities (success or otherwise) further perpetuates the lack of evidence of effective health promotion, duplication of effort and repetition of activities with little to no effect, and an inability to "scale up" effective interventions.

The CQI intervention appears to be a useful strategy for identifying and subsequently improving several key areas of health promotion by engaging staff in the design and redesign of health center systems. We recognize the implementation of the CQI cycle is complex and requires investment of resources, both for facilitation and for the provision of relief staff time to allow all members of the health center team to fully participate in the quality improvement process. Previous research investigating health service involvement in CQI indicates that the commitment from health center managers, senior clinicians, and other leadership positions at the regional level is critical for creating an environment where staff can participate and actively engage in the CQI process (35). This level of commitment will also be crucial for expanding the health promotion capacity of the PHC workforce and for sustaining CQI interventions in health promotion.

Since funding for the original research project ended, we have embraced a range of innovative research translation activities to ensure the uptake of research findings in policy and practice. The health promotion audit and system assessment tools have been refined into web-based tools, resources, and a training package for use through the National Centre for Quality Improvement in Indigenous Primary Health Care (www.One21seventy.org. au). This research translation process enables health services, nationwide access to the quality improvement tools, training, and support to improve Indigenous health promotion. Recommendations from the study have informed the development of a NT implementation plan for quality improvement in Indigenous health promotion. We continue to collaborate with NTDoH in supporting the widespread uptake and implementation of the quality improvement tools and processes.

### AUTHOR CONTRIBUTIONS

NP led drafting of the manuscript and development of the CQI tools and worked in data collection and analysis. LO facilitated

### REFERENCES


engagement of health centers, played a key role in developing the CQI tools, worked in data collection and analysis, and ensured cultural appropriateness. RB, KT, and VL oversaw CQI tool development, data collection and analysis. RB played a lead role in the study design. All authors revised and approved the final manuscript.

#### ACKNOWLEDGMENTS

This research was developed with the support of the Cooperative Research Centre for Aboriginal Health (now the Lowitja Institute, Australia's National Institute for Aboriginal and Torres Strait Islander Health Research) and funded by the National Health and Medical Research Council (NHMRC) of Australia (Project Grant #490302). The study conceptualization and methods draws extensively on the intellectual property of the ABCD project. The work of a number of people with key roles in the research is supported by their employing organizations, including Northern Territory government and Aboriginal community controlled health organizations. The authors would like to thank Paula Convery who contributed to the early development of the CQI tools and in data collection and analysis. NP's work was supported by NHMRC Postgraduate Training Scholarship for Indigenous Australian Health Research (#490337). RB's work was supported by an NHMRC Research Fellowship (#283303).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2016 Percival, O'Donoghue, Lin, Tsey and Bailie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*