Juan C. Espinoza
Steven W. Chin
Payal Shah
Maurice Tut
Jennifer K. Raymond- 1Division of General Pediatrics, Department of Pediatrics, Children’s Hospital Los Angeles, Los Angeles, CA, United States
- 2Translational Informatics, Information Services Department, Children’s Hospital Los Angeles, Los Angeles, CA, United States
- 3Department of Pediatrics, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States
- 4Division of Neonatology, Department of Pediatrics, Fetal and Neonatal Institute, Children’s Hospital Los Angeles, Los Angeles, CA, United States
- 5Division of Endocrinology, Department of Pediatrics, Children’s Hospital Los Angeles, Los Angeles, CA, United States
Diabetes is a uniquely quantifiable disease, and as technology and data have proliferated over the past two decades, so have the tools to manage diabetes. Patients and providers have at their disposal devices, applications, and data platforms that generate immense amounts of data, provide critical insights into a patient’s disease, and allow for personalization of treatment plans. However, the proliferation of options also comes with new burdens for providers: selecting the right tool, getting buy-in from leadership, defining the business case, implementation, and maintenance of the new technology. The complexity of these steps can be overwhelming and sometimes lead to inaction, depriving providers and patients of the advantages of technology-assisted diabetes care. Conceptually, the adoption of digital health solutions can be thought of as occurring in five interconnected phases: Needs Assessment, Solution Identification, Integration, Implementation, and Evaluation. There are a number of existing frameworks to help guide much of this process, but relatively little attention has been focused on integration. Integration is a critical phase for a number of contractual, compliance, financial, and technical processes. Missing a step or doing them out of order can lead to significant delays and potentially wasted resources. To address this gap, we have developed a practical, simplified framework for integrating diabetes data and technology solutions that can guide clinicians and clinical leaders on the critical steps in adopting and implementing a new technology.
Introduction
Diabetes is a uniquely quantifiable disease, and as technology and data have proliferated over the past two decades, so have the tools to manage diabetes. Patients and providers have at their disposal devices, applications, and data platforms that generate immense amounts of data, provide critical insights into a patient’s disease, and allow for personalization of treatment plans. However, the proliferation of options also comes with new burdens for providers: selecting the right tool, getting buy-in from leadership, defining the business case, integration with existing systems, implementation, and maintenance of the new technology. The complexity of these steps can be overwhelming and sometimes lead to inaction, depriving patients and providers of the advantages of technology-assisted diabetes care.
Digital health has enormous potential to positively transform the landscape of diabetes management. Ever expanding, the field encompasses a broad set of concepts and technologies, including telehealth, mobile applications, wearables, artificial intelligence, and precision medicine (1–3). The ongoing push to reach this potential has led to a staggering proliferation of digital health solutions. In 2021, more than 350,000 digital health applications are currently available to consumers, with 90,000 of these introduced just since 2020 (4). Funding for digital health companies reached $57.2 billion in 2021, an increase of 79% from 2020 (5). This rapid growth has been paired with a relative lack of standardized frameworks aiding with solution discovery, clinical integration and evaluation. This leaves stakeholders with challenges in planning and execution of an effective implementation of a digital health solution. High profile examples of problematic implementations have demonstrated that the implementation of an intervention is as important as the intervention itself in achieving the desired end goal (6–9).
Conceptually, the adoption of digital health solutions can be thought of as occurring in five interconnected phases (Figure 1):
1. Needs Assessment and Discovery: What is the problem? What is the opportunity? Who are the relevant stakeholders? How does it impact them? What possible solutions exist?
2. Solution Identification: an informal or formal process (such as a request for proposals), through which designated stakeholders evaluate and ultimately select a specific solution.
3. Integration: Bringing the selected technology into alignment with existing technical systems and operational practices.
4. Implementation: Deploying the technology into clinical practice.
5. Evaluation: Ongoing monitoring of outcomes, with a focus on safety, effectiveness, quality improvement, value, and research objectives.
Figure 1 Digital Health Solution Adoption with a focus on Integration. The process is presented here in five sequential phases, with detailed integration framework presented in eight steps.
There are a number of existing frameworks to help guide much of this process, but relatively little attention has been focused on integration (phase 3). In this article, we provide a brief overview of common approaches to various phases in this process, and describe the integration framework we have developed at our institution.
Needs Assessment and Discovery
Within the context of healthcare delivery, a needs assessment is the process by which individuals or teams gather and analyze information about the needs of a specific patient population in order to understand the gaps in their care, the causes of those gaps, and how those gaps might be closed (10). Several research and operational needs assessment frameworks have been developed and published by organizations like the CDC (11), WHO (12), NHS (13), Johns Hopkins University (14, 15), the University of Kansas (16, 17), the American Medical Association (18), and the Agency for Healthcare Research and Quality (19).
Health equity is a critical component of any Diabetes-related needs assessment. Diabetes prevalence, morbidity and mortality disproportionately impacts low-income communities and communities of color (20). Social determinants of health (SDoH) are principal drivers of these disparities, and the ability to access technology and the internet is increasingly being recognized as a contributing factor to healthcare disparities; a concept referred to as the digital divide (21). Thus, every healthcare organization considering digital health solutions should be keenly aware of the impact they have on vulnerable populations and their ability to meaningfully access and receive technology-enabled care.
Solution Identification
Identifying solutions is often an iterative process, and at healthcare institutions, it is sometimes conducted through a formal Request for Proposals (RFP), typically managed by Information Technology (IT) Services. The growing number of emerging solutions leaves organizations the increasingly difficult task of discerning effective, high quality and high value solutions from a crowded landscape. There is a growing body of standards-based frameworks and resources to assist stakeholders in identifying optimal digital health interventions, including:
● National Health Service (NHS) Apps Library (22) - Evaluates digital health solutions across the board in areas of clinical effectiveness, safety, privacy, usability, accessibility and interoperability.
● Wellocracy (23) - Sponsored by Partners Connected Health. Focuses on consumer wellness and self help. Qualitative reviews, no objective evaluation.
● Personalized Healthcare Connected Alliance (24) - 240 provider, payer, pharma, medical device stakeholder groups. Focus on mobile platform interoperability, connected devices, FHIR data standards.
● RankedHealth (25) - MIT Hacking Medicine Institute. Crowdsourced medical technology professionals perform peer reviews of wide range of apps related to heart disease, diabetes, obesity, pregnancy, reproductive health and more.
● Node.Health (26) - Network for Digital Evidence in Health comprised of 20 health systems, accelerators, startups. Creating an evidence-based medicine movement for digital health solutions. Scores apps in areas of clinical efficacy and usability.
Implementation
A number of implementation theories and frameworks already exist that can be used to guide diabetes technology implementations, such as Normalization Process Theory (NPT) (27, 28), Consolidated Framework for Implementation Research (CFIR) (29–31), Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) Framework (32), and Proctor’s Framework for Implementation Research (33). Nelson et al. published a narrative review in 2020 that provides an overview of various implementation frameworks used in technology-delivered diabetes self-care interventions (34). Many of these implementation frameworks are comprehensive and include, at least in part, aspects of needs assessment, integration, implementation, and evaluation. However, they are principally focused on the clinical implementation, and less so on the technical work required to onboard and integrate a new technology into a clinical setting.
Evaluation
Evaluation of digital health solutions presents a unique challenge compared to other interventions. The speed of software development and rapid iteration cycles are inherently mismatched with the length and time-cycle of randomized controlled trials or most means of traditional evidence generation (35, 36). This incongruence has produced a digital health evidence gap, resulting in low levels of rigorously evaluated applications and a lack of objective information for stakeholders to accurately assess an interventions usability, functionality, safety, effectiveness, accessibility and value. This gap is being addressed with an increasing body of guidance and proposed frameworks addressing the evaluation of digital health solutions, from governmental agencies, industry-based consortiums and academic institutions (35):
● Monitoring and Evaluating Digital Health Interventions (WHO) (37): a comprehensive framework for monitoring, evaluation and validation at all points in product life cycle.
● Evidence Standards Framework for Digital health and Care Technologies (UK National Institute for Health and Care Excellence) (38): Guidance on evidence generation for effectiveness and economic standards. Divides guidance into functional categories based on three tiers of digital function.
● Continua Design Guidelines (Personalized Connected Healthcare Alliance) (39): Framework of standards and criteria required to ensure interoperability of apps focused on personal health and wellness.
● Non-adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) Framework (40): Evidence based framework for prediction and evaluation of digital health solution success based on performance in seven domains.
● Digital Health Scorecard (41): Quantitative scoring of mobile app performance in 5 key domains: technical, clinical, usability, cost, and satisfaction of end user requirements.
The Missing Piece - Integration
Integration is the process of aligning a new technology with existing technical systems and operational processes. This process has multiple stakeholders from across an organization, ranging from legal and compliance, to IT services, to clinic staff. At large institutions, there may be Project Management Offices (PMO) who are responsible for shepherding new integrations through multiple governance and approval committees, but even then, the process can be complex. During this phase, the contractual, compliance, financial, and technical realities of a project come into sharp focus. Missing a step or doing them out of order can lead to significant delays and potentially wasted resources. To address these issues, we have developed a practical, simplified framework for integrating diabetes data and technology solutions that can guide clinicians and clinical leaders on the critical steps in adopting and implementing a new technology, based on our experience (36, 42–45). This model does not replace but rather complements the previously mentioned implementation frameworks, which pick up where our model leaves off.
Our framework (Figure 1) consists of eight sequential steps, presented in Table 1 with a brief description and a series of questions project teams should ask themselves. To illustrate the framework in action, we present a brief case study based on our local experience.
Table 1 A Practical Digital Health IntegrationFframework in Eight Steps.
Case Study: Continuous Glucose Monitoring Integration
Hospital System EPTR has identified a gap in the management of their diabetes population using Medtrix continuous glucose monitoring (CGM) devices. An internal root cause analysis pointed to a lack of available structured CGM data/metrics in the EHR has increased the time needed to document clinical notes, creating additional administrative burden on clinicians. To address this, it has been recommended that EPTR create a CGM data integration with Medtrix. From a patient perspective, Medtrix is the most commonly used CGM in our patient population. Many patients do not have access to the internet at home, so they upload their data during clinic visits using hospital computers or hospital wifi. This process is time consuming for patients and staff, and also needs to be addressed.
With this recommendation, EPTR clinical and IT leadership began to assess the feasibility/capability in planning and implementing a CGM data integration. The planning consisted in evaluating the technical, security and legal, financial, and operational considerations in play.
To assess the technical capacities of EPTR, IT leadership requested and reviewed integration strategy documentation from Medtrix. During technical calls with Medtrix, it was determined that EPTR’s current capability to send and receive EHR HL7 messages externally through a third-party integration vendor, made EPTR a great candidate for this type of integration. For providers, they would continue to use the existing computerized physician order entry (CPOE) function to request CGM data within the EHR.
With recent security breaches within the healthcare industry, EPTR’s information security and legal teams have implemented a variety of processes to mitigate liability and risk. EPTR’s IT security team requested comprehensive technical details of Medtrix’s IT security posture and data management. In addition to executing the vendor’s Data Subscription Agreement, detailing the services and cost of the integration, a separate security addendum provided by EPTR was added. The addendum ensures Medtrix has industry standard security practices and maintains certain service levels. With Medtrix’s status as a hybrid covered entity under HIPAA, no Business Associate Agreement needed to be executed. From a legal perspective, the type of data provided through the integration service falls under the Treatment, Payment, and operations clause, but to decrease liability, data is requested via CPOE orders and not pushed automatically to the EHR.
EPTR’s financial assessment consisted of three considerations: Medtrix’s annual data subscription fee of $8,000, internal project team labor cost, and cost of maintaining the integration. Much of the project team’s cost will come from IT design/development work and testing of the HL7 messages and CPOE order sets. Maintenance cost was determined to remain relatively low. Any additional data being requested through the CGM integration can be worked on by EPTR’s internal clinical systems and integration team.
Even though the main objective of a CGM data integration is meant to reduce or eliminate repetitive administrative workflows, it does introduce new ones. Previously, clinical staff was tasked with logging into Medtrix’s data platform to download, print out an AGP report, hand it to the physician, and scan the document into the EHR. With the new CGM data integration, clinical staff were now tasked with submitting CPOE orders to ensure patient consent and account linkage. Once consent is provided by the patient via email, clinical staff can submit a secondary CPOE order to request CGM data for a particular timeframe. Fortunately, no additional staff was needed to accommodate this new workflow but did require adjustments to clinic workflow.
After a comprehensive review of the feasibility of this effort, clinical and IT leadership created a business case to present to EPTR’s capital committee. Not only was the business case essential to requesting capital funds to implement the project, it also provided the institution the rationale of how this new technology was beneficial to their patients and clinicians. The benefits consisted of the following: remote patient monitoring within the EHR, improve clinical documentation with structured glucose metrics (see trend analysis in a flowsheet/results review), eliminate need to print out and scan reports, use structured data for population health management, help achieve value-based metrics, and may provide additional reimbursement.
Once the budget was approved and contracts were executed, a project manager was assigned to implement the data integration. A project charter was developed to provide a detailed project plan and to receive approvals from the various stakeholders involved. After assigning project resources internally from clinical and IT dept, a project kick-off call was held with Medtrix’s project team. It was at this stage where a communication campaign was created, to make providers and staff aware of the implementation of the CGM integration. This allowed the Endocrinology dept and its staff to understand the new technology, but also become aware of the change in their workflow.
The actual implementation of the CGM data integration was carefully planned out and documented within the project charter. For the design phase, the clinical project team worked to define the glucose metrics and the data timeframe preferred. Once defined, IT resources began to work on defining the HL7 messaging to send and receive data for the integration. In addition, a CPOE order set was also designed to allow clinicians to order the sending of patient consents via email to patients and orders for 14, 30, 45, 90 days of glucose data. Clinical staff were also engaged to map out their existing workflow and work to define a new one with the new integration in place. Within the execution phase, the various components were tested to ensure data flow between EPRT EHR and Medtrix. Clinical staff were asked to run through the new workflow within a test environment.
Before go-live, 3 weeks were dedicated to developing training guides and providing virtual or in person training on the new technology and workflow. Go live support was also provided on the day of and 2 weeks after. To ensure project success and identify potential improvements, a report was created to track the usage of the CPOE CGM order sets, and the amount of data transferred through the integration. A dashboard was also created to help with population management using the structure data.
Discussion
Technical and operational integration are critical for the successful implementation of any digital health project. For experienced teams and institutions with robust project management practices, these may be second nature, but even at the most innovative institutions there can be oversights and failures. In the same way that checklists have improved many aspects of safety and quality across healthcare (indeed, across most industries), team may wish to create their own local checklists of processes, individuals, and governing bodies that correspond to the 8 steps in the integration framework, and systematically implement it for all digital health projects.
Some of the components of our proposed Integration Framework may sound familiar or even redundant to teams familiar with the needs assessment, implementation, and evaluation frameworks briefly reviewed here. This was done deliberately; many of these steps, if not addressed at this stage, can lead to critical system failures during implementation and evaluation, resulting in failed implementation, wasted funds and effort, possible patient harm, and liability exposure. For teams interested in implementing our approach, we do not recommend duplicating work or steps, but rather consider the process from beginning to end across all 5 phases, selecting their preferred frameworks, and then mapping the steps of the integration framework to their cognates in other frameworks. This will allow teams to address each step comprehensively in the appropriate phase (e.g., Step 1, ensuring patient access and acceptability, may already be part of your needs assessment or solution identification).
In our experience, many diabetes data and technology solutions do not have explicit patient processes, workflows, or technical capacity requirements, which leads to implementations that are frustrating for the patient, family, provider, and care team. Our hope is that following the outlined steps will facilitate the adoption of technologies that increase patient engagement and empowerment in regard to accessing and reviewing diabetes data. Many patients with diabetes don’t currently have the ability or tools needed to access their data. Not only will increased access improve current care, patients will learn life-long skills related to diabetes care and improve self-efficacy. Finally, successful digital health implementations can enable improved patient care in synchronous and asynchronous models.
There are limitations to the integration framework. It represents a single institution’s experience and has not undergone more robust vetting. It is intentionally generic to accommodate a variety of data and technology solutions, but this means that more detailed information for specific modalities, such as telehealth, continuous glucose monitoring, and school-based care, is not covered. It also assumes that an organization has people and processes in place to address many of the steps discussed (for example, information security assessment), and as such may not be as helpful in resource-limited settings. Finally, this framework assumes local or institutional management of resources; the framework may need to be adapted to meet the needs of large health systems or countries with national public health systems.
Despite these limitations, we believe that following this practical framework can help reduce friction, prevent delays, and avoid potentially disastrous oversights during the technology selection and implementation process. We hope that others find this tool useful in their own implementations of diabetes data and technology.
Author Contributions
JE conceived the manuscript, wrote the first draft, and developed the figures. SC, PS, MT, and JR contributed to the conception and refinement of the manuscript, as well as to writing sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
This work was supported by the Food and Drug Administration under award number P50FD006425 for The West Coast Consortium for Technology & Innovation in Pediatrics, and by grants UL1TR001855 and UL1TR000130 from the National Center for Advancing Translational Science (NCATS) of the U.S. National Institutes of Health. The funding sources had no involvement in the development of this manuscript or in the decision to submit the paper for publication. The content is solely the responsibility of the authors and does not necessarily represent the official views of the FDA or the NIH.
Conflict of Interest
JE is a paid consultant for AI Health.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The authors would like to acknowledge the support of the teams at the CHLA Innovation Studio, CTIP, and the incredible patients and families we serve who make this work possible.
References
1. The Center for Devices and Radiological Health. What Is Digital Health? The United States of America: FDA (2020). Available at: https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health.
2. Espinoza J, Shah P, Raymond J. Integrating Continuous Glucose Monitor Data Directly Into the Electronic Health Record: Proof of Concept. Diabetes Technol. Ther. (2020) 22(8):570–6. doi: 10.1089/dia.2019.0377
3. Bisno DI, Reid MW, Fogel JL, Pyatak EA, Majidi S, Raymond JK. Virtual Group Appointments Reduce Distress and Improve Care Management in Young Adults With Type 1 Diabetes. J. Diabetes Sci. Technol. (2021). doi: 10.1177/19322968211035768
4. The IQVIA Institute. Digital Health Trends 2021 Innovation, Evidence, Regulation, and Adoption. The IQVIA Institute (2021). Available at: https://www.iqvia.com/insights/the-iqvia-institute/reports/digital-health-trends-2021.
5. The CB Insights. State Of Digital Health 2021 Report (2022). Available at: https://www.cbinsights.com/research/report/digital-health-trends-2021/.
6. Cambridge University Hospitals NHS Foundation Trust. Addenbrooke’s and the Rosie Hospitals Quality Report (2015). Available at: https://www.cqc.org.uk/sites/default/files/new_reports/AAAD0111.pdf.
7. The National Programme for IT in the NHS. An Update on the Delivery of Detailed Care Records Systems - Public Accounts Committee Contents (2011). Available at: https://publications.parliament.uk/pa/cm201012/cmselect/cmpubacc/1070/107005.htm.
8. Kellermann AL, Jones SS. What it Will Take to Achieve the as-Yet-Unfulfilled Promises of Health Information Technology. Health Aff Proj Hope (2013) 32(1):63–8. doi: 10.1377/hlthaff.2012.0693
9. Ross J, Stevenson F, Lau R, Murray E. Factors That Influence the Implementation of E-Health: A Systematic Review of Systematic Reviews (an Update). Implement Sci. IS (2016) 11(1):146. doi: 10.1186/s13012-016-0510-7
10. Stevens A, Gillam S. Needs Assessment: From Theory to Practice. BMJ (1998) 316(7142):1448–52. doi: 10.1136/bmj.316.7142.1448
11. The Centers for Disease Control and Prevention (CDC). Community Needs Assessment (2013). Available at: https://www.cdc.gov/globalhealth/healthprotection/fetp/training_modules/15/community-needs_pw_final_9252013.pdf.
12. The World Health Organization (WHO). Needs Assessment . Available at: https://www.who.int/health-cluster/resources/publications/hc-guide/HC-Guide-chapter-10.pdf?ua=1.
13. Hooper J, Longworth P. Health Needs Assessment Workbook. London:Health Development Agency (2002). Available at: http://healthimpactassessment.pbworks.com/f/Health+needs+assessment+workbook+-+HDA+England+-+2002.pdf.
14. Provider Behavior Change Implementation Kit – For Providers. Available at: https://sbccimplementationkits.org/provider-behavior-change/.
15. Abamecha F, Midaksa G, Sudhakar M, Abebe L, Kebede Y, Mamo A, et al. Acceptability and Feasibility of the School-Engaged Social and Behavior Change Communication Approach on Malaria Prevention in Ethiopia: Implications for Engagement, Empowerment, and Retention (EER) of Education Sectors in Malaria Elimination Efforts. BMC Public Health (2021) 21:1909. doi: 10.1186/s12889-021-11995-z
16. The University of Kansas. Chapter 3. Assessing Community Needs and Resources. In: Section 1. Developing a Plan for Assessing Local Needs and Resources | Main Section | Community Tool Box. (2014) Available at: https://ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources/develop-a-plan/main.
17. Burns JC, Teadt S, Bradley WW, Shade GH. Enhancing Adolescent and Young Adult Health Services! A Review of the Community Needs Assessment Process in an Urban Federally Qualified Health Center. Health Equity (2020) 4(1):218–24. doi: 10.1089/heq.2019.0108
18. AMERICAN MEDICAL ASSOCIATION (AMA). DIGITAL HEALTH IMPLEMENTATION PLAYBOOK. The United States of America: AMA (2018). Available at: https://www.ama-assn.org/system/files/2018-12/digital-health-implementation-playbook.pdf.
19. Needs Assessment | Digital Healthcare Research. Available at: https://digital.ahrq.gov/health-it-tools-and-resources/evaluation-resources/workflow-assessment-health-it-toolkit/all-workflow-tools/needs-assessment.
20. Hill-Briggs F, Adler NE, Berkowitz SA, Chin MH, Gary-Webb TL, Navas-Acien A, et al. Social Determinants of Health and Diabetes: A Scientific Review. Diabetes Care (2020) 44(1):dci20–0053. doi: 10.2337/dci20-0053
21. Ramsetty A, Adams C. Impact of the Digital Divide in the Age of COVID-19. J. Am. Med. Inform Assoc. JAMIA (2020) 27(7):1147–8. doi: 10.1093/jamia/ocaa078
22. NHS Apps Library. NHS Digital. (2022). Available at: https://www.nhs.uk/nhs-app/.
23. Kezdlap. Wellocracy. Available at: https://www.wllocracy.com/.
24. Personal Connected Health Alliance. Available at: https://www.pchalliance.org/node.
25. Ranked – Curated Health Apps and Devices. Available at: http://www.rankedhealth.com/.
26. NODE.Health. NODE.Health. Available at: https://www.nodehealth.org/.
27. May C, Finch T. Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory. Sociology (2009) 43(3):535–54. doi: 10.1177/0038038509103208
28. Ross J, Stevenson F, Dack C, Pal K, May C, Michie S, et al. Developing an Implementation Strategy for a Digital Health Intervention: An Example in Routine Healthcare. BMC Health Serv. Res. (2018) 18(1):794. doi: 10.1186/s12913-018-3615-7
29. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering Implementation of Health Services Research Findings Into Practice: A Consolidated Framework for Advancing Implementation Science. Implement Sci. IS (2009) 4:50. doi: 10.1186/1748-5908-4-50
30. Kowatsch T, Otto L, Harperink S, Cotti A, Schlieter H. A Design and Evaluation Framework for Digital Health Interventions. It - Inf Technol. (2019) 61(5–6):253–63. doi: 10.1515/itit-2019-0019
31. Wienert J, Zeeb H. Implementing Health Apps for Digital Public Health - An Implementation Science Approach Adopting the Consolidated Framework for Implementation Research. Front. Public Health (2021) 9:610237. doi: 10.3389/fpubh.2021.610237
32. Glasgow RE, Vogt TM, Boles SM. Evaluating the Public Health Impact of Health Promotion Interventions: The RE-AIM Framework. Am. J. Public Health (1999) 89(9):1322–7. doi: 10.2105/AJPH.89.9.1322
33. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment. Health (2011) 38(2):65–76. doi: 10.1007/s10488-010-0319-7
34. Nelson LA, Williamson SE, Nigg A, Martinez W. Implementation of Technology-Delivered Diabetes Self-Care Interventions in Clinical Care: A Narrative Review. Curr. Diabetes Rep. (2020) 20(12):71. doi: 10.1007/s11892-020-01356-2
35. Guo C, Ashrafian H, Ghafur S, Fontana G, Gardner C, Prime M. Challenges for the Evaluation of Digital Health Solutions-A Call for Innovative Evidence Generation Approaches. NPJ Digit Med. (2020) 3:110. doi: 10.1038/s41746-020-00314-2
36. Lewinter KE, Hudson SM, Kysh L, Lara M, Betz CL, Espinoza J. Reconsidering Reviews: The Role of Scoping Reviews in Digital Medicine and Pediatrics. NPJ Digit Med. (2020) 3(1):158. doi: 10.1038/s41746-020-00368-2
37. World Health Organization. Monitoring and Evaluating Digital Health Interventions A Practical Guide to Conducting Research and Assessment. World Health Organization (2016). Available at: http://www.who.int/reproductivehealth/publications/mhealth/digital-health-interventions/en/.
38. The National Institute for Health and Care Excellence (NICE). Evidence Standards Framework for Digital Health Technologies | Our Programmes | What we do | About. NICE (2022). Available at: https://www.nice.org.uk/about/what-we-do/our-programmes/evidence-standards-framework-for-digital-health-technologies.
39. Personal Connected Healthcare Alliance- Resources (2022). Available at: https://www.pchalliance.org/resources.
40. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies. J. Med. Internet Res. (2017) 19(11):e367. doi: 10.2196/jmir.8775
41. Sedhom R, McShea MJ, Cohen AB, Webster JA, Mathews SC. Mobile App Validation: A Digital Health Scorecard Approach. NPJ Digit Med. (2021) 4(1):111. doi: 10.1038/s41746-021-00476-7
42. Espinoza JC, Chen AM, Deavenport-Saman A, Solomon O, Ponce A, Sikder A, et al. Wearable Devices Decrease Attrition Among Families Participating in an Obesity Intervention at a Federally Qualified Health Center. J. Health Care Poor Underserved (2021) 32(2):13–24. doi: 10.1353/hpu.2021.0048
43. Gold JI, Annick ET, Lane AS, Ho K, Marty RT, Espinoza JC. “Doc McStuffins: Doctor for a Day” Virtual Reality (DocVR) for Pediatric Preoperative Anxiety and Satisfaction: Pediatric Medical Technology Feasibility Study. J. Med. Internet Res. (2021) 23(4):e25504. doi: 10.2196/25504
44. Espinoza J, Crown K, Kulkarni O. A Guide to Chatbots for COVID-19 Screening at Pediatric Health Care Facilities. JMIR Public Health Surveill (2020) 6(2):e18808. doi: 10.2196/18808
Keywords: diabetes, technology, implementation, framework, integration, data
Citation: Espinoza JC, Chin SW, Shah P, Tut M and Raymond JK (2022) Proposing a Practical, Simplified Framework for Implementing Integrated Diabetes Data and Technology Solutions. Front. Clin. Diabetes Healthc. 3:867284. doi: 10.3389/fcdhc.2022.867284
Received: 31 January 2022; Accepted: 01 April 2022;
Published: 11 May 2022.
Edited by:
María Teresa Anarte Ortiz, University of Malaga, SpainReviewed by:
Laurel Haydon-Smith Messer, University of Colorado Anschutz Medical Campus, United StatesCopyright © 2022 Espinoza, Chin, Shah, Tut and Raymond. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Juan C. Espinoza, jespinoza@chla.usc.edu