A holistic approach to integrating patient, family, and lived experience voices in the development of the BrainHealth Databank: a digital learning health system to enable artificial intelligence in the clinic

1. Introduction

Rapid advancements in digital health technologies and artificial intelligence/machine learning (AI/ML) have created tremendous potential for transformative change in healthcare. Applying AI/ML algorithms to large datasets generated by digital health technologies extends potential benefits beyond care provision, including healthcare planning, treatment, prevention, public health, and disease progression (1). In mental health, AI can improve diagnosis and predict risk, support remote monitoring, and enable access to more personalized and de-stigmatized treatment forms (2–5).

Despite the transformative potential, there have been varying degrees of success with implementing digital health and AI, as many initiatives faced barriers that hindered acceptance, uptake, and adoption. Barriers are commonly socio-technological, where socio-cultural factors affect individual perceptions, acceptance, use, and adoption of technology (6). Failed implementations of large-scale health data initiatives highlighted the importance of public perspectives on how data is used and managed (7). Given the complexity of AI initiatives, meaningful engagement is critical to improving the systems' fairness, accountability, and transparency (8). Furthermore, the limited evidence on the effectiveness of digital health and AI/ML for mental health makes public engagement increasingly important, as there are potential unintended privacy and health implications to any design decisions with its development. Dialogues on values, needs, and nuanced insights derived through lived experiences are critical to inform decisions about the governance, design and implementation of these data initiatives (9, 10). As such, sustained engagement from the initiative's outset would ensure meaningful contributions in shaping its development and governance in a trustworthy, ethical, and acceptable manner (11–15).

Engagement can come in many forms and range in degree of involvement (16, 17). The Carman Patient Engagement Framework (18) suggests the spectrum ranges from “consultation” to “involvement” to partnership (or shared leadership), where patients, families, and people with lived experience (PFLE) become increasingly involved, and their perspectives have a more significant impact on organizational decision-making. PFLE engagement guides and toolkits have commonly suggested the need for organizational readiness and commitment to engage; clear visions and opportunities for engagement; a common understanding amongst all stakeholders; accommodations that promote a safe, inclusive environment, evaluation of engagement, and mechanisms to report feedback (19). While these frameworks inform approaches, they may have limited transferability across contexts, requiring organizations or initiatives to adapt approaches to suit their context (20, 21). This perspective article shares our approach and reflection on achieving meaningful PFLE engagement in developing a complex, large-scale digital health care and research initiative called the BrainHealth Databank (BHDB). The BHDB is intended to serve as the foundation for AI/ML applications at the Centre for Addiction and Mental Health (CAMH).

1.1. BrainHealth Databank at the centre for addiction and mental health

The BHDB is a hospital-wide initiative that advances patient-centred, data-driven care at CAMH — Canada's largest mental health teaching hospital and a leading research institute (22). The BHDB is a repository of numerous data types from various sources (e.g., sleep and physical activity, blood samples and brain images, clinical assessments, etc.). As the foundation for a digital health learning system, the BHDB will leverage clinical data to inform research and research data to inform care. Its objective is to improve our understanding of the mental illness of individuals and broader populations to accelerate the ability to deliver personalized care. This transformative initiative is the first in the Canadian mental health context (23).

As a research Centre focused on developing clinical applications of AI/ML and computational modelling, CAMH's Krembil Centre for Neuroinformatics (KCNI) led the development of the BHDB by building upon CAMH's strategic investments in core infrastructure. This includes the Cerner Millennium Electronic Health Record (EHR) and CAMH Neuroinformatics Platform to support multimodal research studies (24, 25). To support care delivery, CAMH clinics have developed and implemented evidence-based integrated care pathways that utilize measurement-based care (MBC) to monitor patient progress and inform clinical decisions (26–28).

The BHDB was developed in collaboration with clinicians, scientists, data engineers, clinical application specialists, privacy officers, legal counsel, ethicists, hospital administrators, and PFLEs to leverage their expertise to inform the various aspects of development and implementation. The first phase of the BHDB enhanced digital technology infrastructure by adding new capabilities, including electronic self-assessment capture and patient trajectory visualization, which have already influenced care. Digital MBC allows patients to complete assessments on their own device or clinic tablet before appointments resulting in efficiencies and increased patient flow (29). It has enabled the generation of real-time visual displays of patient treatment journeys, allowing clinicians to assess patients' progress quickly.

To better understand patient trajectories, the BHDB is enriching EHR treatment trajectory data by integrating the collection of research samples and daily activity and sleep data from wearable devices. The accumulated rich, integrated real-world clinical and research data is ideal for future AI/ML applications. Study participants can consent to donate their data to the BHDB for secondary use by other researchers. A patient-facing version of this dashboard is in development.

2. Engaging patients, families, and people with lived experience

Given that the core BHDB goal is to improve patient-centred, data-driven mental health, there was a recognition that PFLE engagement is critical during the initial planning stages to ensure the foundation of the BHDB was patient-centred and based on community needs. CAMH and KCNI's commitment to meaningful PFLE engagement was intentional, established, and communicated from the onset of the initiative.

2.1. Establishing the BHDB PFLE engagement team

The BHDB steering committee tasked a working group to create a strategy for facilitating PLFE engagement throughout the initiative's life cycle. This task dovetailed with CAMH's clinical and research patient and family engagement strategy and the organizational PFLE engagement roadmap. The roadmap grounded and adapted the Carman patient engagement framework (18, 30). Moreover, four clinical and research family advisory committees were established, which included CAMH PFLE engagement facilitators and coordinators who were trained specialists in PFLE engagement.

Based on the recommendation of CAMH PFLE facilitators and coordinators, the BHDB team was invited to join various PFLE advisory committees to introduce the BHDB and recruit individuals to join the BHDB PFLE engagement team. While this opportunity attracted PFLE advisors interested in AI/ML and digital health technologies, there was no requirement to have prior knowledge or experience in these areas. As team members, the recruited three BHDB PFLE partners ensured PFLE engagement, representation, and participation in BHDB activities.

To kick off the BHDB PFLE engagement team, Terms of Reference were co-created to establish roles, goals, and a common understanding of contributions. Since BHDB partners were also members of their respective CAMH PFLE advisory committees, they were responsible for reporting BHDB developments to their respective committees and soliciting feedback when necessary. As engagement team members, the partners would provide leadership and expertise in planning and supporting the engagement strategies and activities; make recommendations on meaningful engagement and representation; join the working groups within and outside the governance structure; and participate in designing various projects. To ensure accountability, an activity-tracking tool was used to document all the projects, goals, and ideas stemming from the project while providing the team with the status of various project objectives. The following section highlights three major projects where the team identified and developed mechanisms for PFLE engagement.

2.2. Different types of PFLE engagement across BHDB projects

2.2.1. Project 1: consultation—research and care coordination

As a first step to developing a digital platform to support clinics with the coordination of research and care, the BHDB launched a series of five interactive virtual workshops to engage stakeholders across the hospital to gather user requirements. BHDB partners invited advisors from their respective advisory committees to participate. Advisors were encouraged to attend a pre-workshop orientation session where they were briefed with information about the context, background, technical terminology, workshop expectations and outcomes.

The PFLE advisors were invited to attend multiple workshops. Their participation accounted for 21.5% (31/144) of stakeholder engagement across all five workshops. At the virtual workshops, PFLE advisors participated in breakout sessions and discussions with clinicians, scientists, and hospital administrators. Participants provided their user requirements for the digital platform and collectively prioritized the modules identified through the workshop. The PFLE engagement was critical in advocating for a patient-facing interface, ranked in the top 5 of 12 priority areas. This finding initiated the development of a patient-facing portal to be included. The clinical and patient portals are currently under development and will be integrated into a CAMH-wide digital initiative.

2.2.2. Project 2: involvement — co-designing the patient trajectory dashboard

As the BHDB clinical treatment trajectory dashboard was integrated across various CAMH clinics to support clinician decision-making, there was a growing recognition that a patient-facing dashboard can also benefit patients and families. This initiated a project to co-design a patient journey dashboard and was brought to the BHDB PFLE engagement team to plan. A BHDB partner volunteered to be the project co-lead with a design student from a local university. As co-leads, they planned and conducted the human-centred design project. They contributed to developing questions for semi-structured interviews and interviewed seven PFLE advisors and three clinicians. The joint analysis of gathered information informed the creation of a wireframe prototype. A formal project report and presentation were co-developed and co-presented to participants and BHDB stakeholders. This project initiated the future integration of a digital patient journey tool in the next iteration of the CAMH patient portal.

2.2.3. Project 3: partnership—BHDB governance

The BHDB PFLE engagement team operated on a partnership model with a shared decision-making process for enacting PFLE engagement opportunities. This included early engagements with the BHDB steering committee and external scientific advisory committees. However, the engagement team understood that if the goal was for a meaningful engagement at all levels, it was a natural evolution for the team to advocate for its PLFE partners to be members of the BHDB steering committee. This would enable PFLE to directly influence decision-making through their interactions with senior leadership and administrators on the committee. As a result, two PFLE partners formally joined the BHDB steering committee. To ensure a smooth transition to this new responsibility, roles and expectations were communicated, and the opportunity to have additional briefing sessions before meetings. As SC members, PFLE partners raised agenda items on the team's behalf and provided updates on PFLE engagement activities during SC meetings. It has been a positive experience for the other steering committee members and PFLE partners, with positive anecdotal feedback from all parties involved. Participation at all levels in this work has also increased the visibility of PFLE engagement, which has led to additional opportunities for PFLEs, such as co-presenting to the hospital board of directors. Moreover, this approach was lauded by the external scientific advisory committee.

3. Discussion—critical success factors and lessons learned

This perspective article is intended to contribute to our understanding of PFLE engagement in digital health and AI and this dynamic field of practice. While there is increasing recognition that PFLE engagement is crucial to the acceptability of AI in healthcare, engagement is predominantly overlooked in the development of AI or has a limited presence in academic literature (31–33). A recent scoping review on patient and public involvement in AI and digital health in mental health (33) found that only 5 of the 144 articles identified focused on PFLE, whereas most studies used and accessed patient and public perspectives to inform a project or co-design technologies. This paper provides some preliminary insights into other knowledge gaps in PFLE engagement in AI by reflecting on our longitudinal experiences in the context of mental health.

This article also documents a whole system approach to PFLE engagement (34), where engagement considers the individual, team, and organizational implementation factors. Based on feedback from BHDB internal and external stakeholders, PFLE partners and advisors, and use case outcomes described above, our dynamic approach to PFLE engagement has been successful overall. Guided by the PFLE process map (Figure 1) (19), we highlight the factors that contributed to our success and identify areas for improvement.

FIGURE 1

Figure 1. BrainHealth Databank patient, family, and persons with lived experience (BHDB PFLE) engagement process map [adapted from Shen et al. (19)]. The process map was created to provide an overview of PFLE engagement processes throughout the BHDB lifecycle. Check marks (√) indicate practices that contributed to successful PFLE engagement while check boxes (□) and italicized font indicate areas for improvement moving forward. BHDB, BrainHealth Databank; PFLE, patients, families, and persons with lived experiences.

3.1. Culture and infrastructure

The commitment at the CAMH organization, BHDB leadership, and PLFE individual level led to active PFLE engagement in various degrees across the BHDB implementation. At the organizational level, CAMH's culture and PFLE engagement infrastructure immensely benefited the BHDB. At the BHDB leadership level, the genuine commitment and support to achieve meaningful PFLE engagement at all levels of the BHDB contributed to the achievements to date. These findings align with existing literature and frameworks on developing organizational capacity for PFLE engagement (35).

We were fortunate to be situated in an environment where mobilizing PFLE engagement across all CAMH activities was an organizational priority. The organizational commitment and infrastructure aligned with the BHDB's approach to building, developing, and growing partnerships with PFLEs (36). The established clinical and research patient and family advisory committees created a community of interested PFLE experts for the BHDB to draw upon with PFLE engagement to build engagement capacity by educating and training staff and PFLEs. Institutional policy and processes for providing honoraria for PLFE participation were formalized as a requirement, acknowledging time for PFLE expertise. This organizational readiness immensely benefited BHDB's PFLE engagement plans, particularly for facilitating recruitment.

3.2. Planning and preparation

Readiness across the system enabled PFLE engagement at a team level. Establishing the BHDB PLFE engagement team was critical to integrating engagement across the breadth of activities in developing and implementing the BHDB. First, it connects the various project teams, committees, and the PFLE engagement community. This connection enabled the wayfinding and identification of PFLE engagement opportunities, evaluation of feasibility, and development of plans to support the identified opportunities. Secondly, having a team of on-boarded PFLE partners, facilitators, and coordinators with diverse expertise meet monthly allows flexibility and responsiveness in planning and mobilizing for engagement activities. This was especially valuable given that opportunities were often attached to short timelines and required an understanding of the complexity and breadth of the BHDB initiative.

3.3. Identification and recruitment

The BHDB's scale and focus on cutting-edge digital health and AI technologies have attracted PLFE participation. Opportunities to engage at various levels and with varying roles allowed us to attract and recruit advisors with different experience levels and diverse backgrounds from the respective CAMH advisories. For example, those newer to the role of PFLE advisor were more comfortable in a consultation role, while those with experience were drawn to leadership roles. PFLE readiness influenced the degree to which advisors participated in this work.

PFLE recruitment has solely been from the CAMH clinical and research patient and family advisory committees. While this has allowed us to be agile in responding to engagement opportunities with tight timelines, as experienced in case study 1, this approach has shortcomings. Recruitment was from a community of advisors supporting mental health care and research at CAMH. Much of this work was during the pandemic when recruiting from the community and clinics was difficult. Recognizing the need for a greater diversity of perspectives, the team will explore additional avenues for engagement, including reaching beyond the walls of CAMH.

A robust onboarding process is necessary to achieve meaningful engagement and ensure PFLE readiness, which requires understanding project background, context, and PFLE role. There was no requirement to have prior knowledge of digital health or ML/AI to participate. The onboarding process and engagement sessions were virtual due to the COVID-19 pandemic. While this removed the need for travel and provided greater flexibility for participation, this also created the requirement for participants to have access to technology. An option to call in and receive materials via email was provided to support those without access to stable internet.

Orientation sessions involved a project overview presentation including expectations, time commitments and anticipated outcomes, followed by a question-and-answer period, were held for each case study. For use cases 1 and 2, PFLE partners helped to co-create practical and accessible project orientation and background information materials, such as a terminology cheat sheet.

3.4. Partnering with PFLEs

At the PFLE individual level, a sense of understanding and appreciation of their impact on the BHDB motivated participation. Establishing a safe environment and building trust with PFLE Partners is vital for supporting meaningful engagement. One way the BHDB accomplished this was by ensuring that there was always more than one PFLE team member present during activities. PFLE partners could also contact PFLE staff facilitators and coordinators to raise any issues and concerns. These avenues minimized the risk of power differentials impacting this work.

Fostering open dialogue in these meetings was a critical prerequisite for thoughtful and meaningful PFLE engagement. To create a safe space for open dialogue, monthly BHDB PFLE team meetings provided a comfortable space for the team to check-in, share BHDB updates, learn about related projects, put forward items and recommendations to the SC and external scientific advisory committee, and, most importantly, make decisions on matters about PFLE engagement. It also ensured a common understanding of projects and objectives and allowed the team to ask questions and work through knowledge gaps and challenges.

These regular monthly team meetings also provided a consistent cadence for sharing BHDB activity updates which were critical in enabling PFLE partners to identify engagement opportunities, fostering a sense of shared accountability for this work. While BHDB updates would frequently come from the project team, other key BDHB stakeholders were invited to present to this group. Given the enormous scope of the BHDB, sharing updates is challenging and an area to improve upon. These factors echo the experiences of other PFLE engagement initiatives (37) and the literature on meaningful engagement (34).

3.5. Evaluation and reporting

Debrief sessions were held by either BHDB or CAMH advisory committees post-PFLE engagement to assess meaningful engagement and gather feedback. Regular reporting on PFLE engagement activities to stakeholders occurred at every BHDB SC and external scientific advisory committee meeting. Any feedback received was discussed at subsequent BHDB PFLE team meetings, and an action plan was developed accordingly. The opportunity to request additional information on any topics or issues discussed during these meetings was also provided to improve understanding of the matters discussed.

In retrospect, formal evaluations of the engagement were overlooked, which is a limitation of our work to date. Moving forward, implementing standardized evaluation tools and frameworks will help systematically evaluate the impact of each engagement and inform future engagements (38, 39). Tools and frameworks such as the Public and Patient Engagement Evaluation Tool (PPEET) (40) and Guidance for Reporting Involvement of Patients and the Public (GRIPP2) (41) would allow for more consistent evaluations and reporting, thereby improving applicability across contexts. Sharing lessons learned will be critical in advancing the collective understanding on meaningfully engaging PFLE in implementing these large initiatives, documenting how we circumnavigate emerging challenges.

4. Future directions and conclusion

As the BHDB continues to grow, there is an imperative to keep improving our approach to PFLE engagement. In our future engagements, we aim to increase the diversity of PFLE perspectives in our engagements and team. Moving forward, we will explore additional areas for engagement—for example, building capacity for PFLE engagement in research through the BHDB. With the eventual integration of BHDB AI use cases, PFLE engagement becomes increasingly critical to ensure the development of responsible and ethical products that meet the community's needs, especially concerning AI's appropriateness, equity and fairness, privacy, governance, and transparency. Formal evaluations of those engagements should be a foundational activity to ensure it is increasingly meaningful and impactful for the individuals, teams, and organization. By sharing how the BHDB navigated and fostered PFLE engagement, we hope this will motivate and inspire the digital health, AI, and mental health community to continue to collectively chart this path to both support advancements and implementation of novel cutting-edge digital health and ML/AI products that will improve brain health.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

JY and NS: wrote the first draft of the manuscript. SC, BL, and RM: are BHDB patient, family, and lived experience research partners. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the CAMH BrainHealth Databank, funded by the CAMH Discovery Fund. NS was a Canadian Institutes of Health Research (CIHR) Health System Impact Fellow for part of the work discussed in this paper. The fellowship was jointly funded by CIHR's Institute of Health Services and Policy Research and CAMH.

Acknowledgments

The authors would like to acknowledge and thank all the patients, families, persons with lived experience, and the hospital-wide CAMH BrainHealth Databank team for their engagement and active participation in this initiative.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Kruse CS, Goswamy R, Raval YJ, Marawi S. Challenges and opportunities of big data in health care: a systematic review. JMIR Med Inform. (2016) 4(4):e5359. doi: 10.2196/medinform.5359

CrossRef Full Text | Google Scholar

2. D’Alfonso S, Santesteban-Echarri O, Rice S, Wadley G, Lederman R, Miles C, et al. Artificial intelligence-assisted online social therapy for youth mental health. Front Psychol. (2017) 8:796. doi: 10.3389/fpsyg.2017.00796

CrossRef Full Text | Google Scholar

3. Graham S, Depp C, Lee EE, Nebeker C, Tu X, Kim HC, et al. Artificial intelligence for mental health and mental illnesses: an overview. Curr Psychiatry Rep. (2019) 21(11):116. doi: 10.1007/s11920-019-1094-0

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Lee EE, Torous J, De Choudhury M, Depp CA, Graham SA, Kim HC, et al. Artificial intelligence for mental health care: clinical applications, barriers, facilitators, and artificial wisdom. Biol Psychiatry Cogn Neurosci Neuroimaging. (2021) 6(9):856–64. doi: 10.1016/j.bpsc.2021.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Lovejoy CA. Technology and mental health: the role of artificial intelligence. Eur Psychiatry. (2019) 55:1–3. doi: 10.1016/j.eurpsy.2018.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. (2017) 19(11):e367. doi: 10.2196/jmir.8775

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Wachter RM, Cassel CK. Sharing health care data with digital giants: overcoming obstacles and reaping benefits while protecting patients. JAMA. (2020) 323(6):507. doi: 10.1001/jama.2019.21215

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Nebeker C, Torous J, Bartlett Ellis RJ. Building the case for actionable ethics in digital health research supported by artificial intelligence. BMC Med. (2019) 17(1):1–7. doi: 10.1186/s12916-019-1377-7

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Bao L, Krause NM, Calice MN, Scheufele DA, Wirz CD, Brossard D, et al. Whose AI? How different publics think about AI and its social impacts. Comput Hum Behav. (2022) 130:107182. doi: 10.1016/j.chb.2022.107182

CrossRef Full Text | Google Scholar

10. Carr S. ‘AI gone mental’: engagement and ethics in data-driven technology for mental health. J Ment Health. (2020) 29(2):125–30. doi: 10.1080/09638237.2020.1714011

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Armstrong N, Herbert G, Aveling E, Dixon-Woods M, Martin G. Optimizing patient involvement in quality improvement. Health Expect. (2013) 16(3):e36–47. doi: 10.1111/hex.12039

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Leung K, Lu-McLean D, Kuziemsky C, Booth RG, Rossetti SC, Borycki E, et al. Using patient and family engagement strategies to improve outcomes of health information technology initiatives: scoping review. J Med Internet Res. (2019) 21(10):e14683. doi: 10.2196/14683

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Muller SHA, Kalkman S, van Thiel GJMW, Mostert M, van Delden JJM. The social licence for data-intensive health research: towards co-creation, public value and trust. BMC Med Ethics. (2021) 22(1):110. doi: 10.1186/s12910-021-00677-5

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Carter P, Laurie GT, Dixon-Woods M. The social licence for research: why care.data ran into trouble. J Med Ethics. (2015) 41(5):404–9. doi: 10.1136/medethics-2014-102374

PubMed Abstract | CrossRef Full Text | Google Scholar

15. van Staa TP, Goldacre B, Buchan I, Smeeth L. Big health data: the need to earn public trust. Br Med J. (2016) 354:i3636. doi: 10.1136/bmj.i3636

CrossRef Full Text | Google Scholar

16. Majid U, Gagliardi A. Conceptual frameworks and degrees of patient engagement in the planning and designing of health services: a scoping review of qualitative studies. Patient Exp J. (2019) 6(3):82–90. doi: 10.35680/2372-0247.1356

CrossRef Full Text | Google Scholar

17. Smits DW, van Meeteren K, Klem M, Alsem M, Ketelaar M. Designing a tool to support patient and public involvement in research projects: the involvement matrix. Res Involv Engag. (2020) 6:30. doi: 10.1186/s40900-020-00188-4

CrossRef Full Text | Google Scholar

18. Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff. (2013) 32(2):223–31. doi: 10.1377/hlthaff.2012.1133

CrossRef Full Text | Google Scholar

19. Shen N, Jankowicz D, Strudwick G. Patient and family engagement approaches for digital health initiatives: protocol for a case study. JMIR Res Protoc. (2021) 10(7):e24274. doi: 10.2196/24274

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. (2019) 22(4):785–801. doi: 10.1111/hex.12888

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Bombard Y, Baker GR, Orlando E, Fancott C, Bhatia P, Casalino S, et al. Engaging patients to improve quality of care: a systematic review. Implement Sci. (2018) 13(1):98. doi: 10.1186/s13012-018-0784-z

PubMed Abstract | CrossRef Full Text | Google Scholar

22. O’Malley S. The CAMH BrainHealth Databank. (2021). Available at: https://www.camh.ca/en/camh-news-and-stories/the-camh-brainhealth-databank (cited 2023 Jan 1).

23. Centre for Addiction and Mental Health. CAMH. BrainHealth Databank. Available at: https://www.camh.ca/en/science-and-research/discovery-fund/brainhealth-databank (cited 2023 Feb 8).

24. Sulkers H, Tajirian T, Paterson J, Mucuceanu D, MacArthur T, Strauss J, et al. Improving inpatient mental health medication safety through the process of obtaining HIMSS stage 7: a case report. JAMIA Open. (2018) 2(1):35–9. doi: 10.1093/jamiaopen/ooy044

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Rotenberg DJ, Chang Q, Potapova N, Wang A, Hon M, Sanches M, et al. The CAMH neuroinformatics platform: a hospital-focused brain-CODE implementation. Front Neuroinform. (2018) 12:77. doi: 10.3389/fninf.2018.00077

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Campbell H, Hotchkiss R, Bradshaw N, Porteous M. Integrated care pathways. Br Med J. (1998) 316(7125):133–7. doi: 10.1136/bmj.316.7125.133

CrossRef Full Text | Google Scholar

27. Chen S, Awan S, Rajji T, Abdool P, George TP, Collins A, et al. Integrated care pathways for schizophrenia: a scoping review. Adm Policy Ment Health. (2016) 43(5):760–7. doi: 10.1007/s10488-015-0696-z

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Settipani CA, Cleverley K, Hawke LD, Rice M, Henderson JL. Essential components of integrated care for youth with mental health and addiction needs: protocol for a scoping review. BMJ Open. (2017) 7(4):e015454. doi: 10.1136/bmjopen-2016-015454

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Hawley S, Yu J, Bogetic N, Potapova N, Wakefield C, Thompson M, et al. Digitization of measurement-based care pathways in mental health through REDCap and electronic health record integration: development and usability study. J Med Internet Res. (2021) 23(5):e25656. doi: 10.2196/25656

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Research AI for. A Roadmap for Patient+Family Engagement in Healthcare Practice and Research: Practical strategies for advancing engagement in healthcare—starting today. Available at: https://www.air.org/sites/default/files/Roadmap-Patient-Family-Engagement.pdf (cited 2023 Mar 31).

31. Adus S, Macklin J, Pinto A. Exploring patient perspectives on how they can and should be engaged in the development of artificial intelligence (AI) applications in health care. In Review. (2023). Available at: https://www.researchsquare.com/article/rs-2972402/v1 (cited 2023 Jul 4).

32. Camaradou JCL, Hogg HDJ. Commentary: patient perspectives on artificial intelligence; what have we learned and how should we move forward? Adv Ther. (2023) 40(6):2563–72. doi: 10.1007/s12325-023-02511-3

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Zidaru T, Morrow EM, Stockley R. Ensuring patient and public involvement in the transition to AI-assisted mental health care: a systematic scoping review and agenda for design justice. Health Expect. (2021) 24(4):1072–124. doi: 10.1111/hex.13299

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Ocloo J, Garfield S, Franklin BD, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Syst. (2021) 19(1):8. doi: 10.1186/s12961-020-00644-3

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Anderson NN, Baker GR, Moody L, Scane K, Urquhart R, Wodchis WP, et al. Organizational capacity for patient and family engagement in hospital planning and improvement: interviews with patient/family advisors, managers and clinicians. Int J Qual Health Care. (2021) 33(4):mzab147. doi: 10.1093/intqhc/mzab147

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Centre for Addiction and Mental Health. ONE CAMH—CAMH Strategic Plan 2020–2023. Available at: https://www.camh.ca/en/driving-change/about-camh/one-camh-strategic-plan-2020-2023/-/media/images/all-other-images/onecamh/one_camh-strategic_plan_2020-2023-06-23-2020.pdf (cited 2023 Feb 8).

37. Huwe M, Woolf B, David J, Seid M, Saeed S, Margolis P, et al. Conceptualizing and redefining successful patient engagement in patient advisory councils in learning health networks. Learn Health Syst. (2023) e10377. doi: 10.1002/lrh2.10377

CrossRef Full Text | Google Scholar

38. Abelson J, Tripp L, MacNeil M, Lang A, Fancott C, Ganann R, et al. Development of the engage with impact toolkit: a comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems. Health Expect. (2023) 26(3):1255–65. doi: 10.1111/hex.13742

PubMed Abstract | CrossRef Full Text | Google Scholar

39. L’Esperance A, O’Brien N, Gregoire A, Abelson J, Canfield C, Del Grande C, et al. Developing a Canadian evaluation framework for patient and public engagement in research: study protocol. Res Involv Engag. (2021) 7(1):10. doi: 10.1186/s40900-021-00255-4

CrossRef Full Text | Google Scholar

40. Abelson J, Tripp L, Kandasamy S, Burrows K. Supporting the evaluation of public and patient engagement in health system organizations: results from an implementation research study. Health Expect Int J Public Particip Health Care Health Policy. (2019) 22(5):1132–43. doi: 10.1111/hex.12949

CrossRef Full Text | Google Scholar

41. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Br Med J. (2017) 358:j3453. doi: 10.1136/bmj.j3453

CrossRef Full Text | Google Scholar