W. Scott Comulada
Catherine McQueen3
Cathy M. Lang- 1Department of Psychiatry and Biobehavioral Sciences, Semel Institute Center for Community Health, University of California, Los Angeles, Los Angeles, CA, United States
- 2Department of Health Policy and Management, University of California, Los Angeles, Los Angeles, CA, United States
- 3Department of Emergency Medicine and Informatics, Institute for Prediction Technology, University of California, Berkeley, Berkeley, CA, United States
- 4Department of Community Health Sciences, University of California, Los Angeles, Los Angeles, CA, United States
Background: Artificial Intelligence (AI)-based digital health interventions incorporating technologies like chatbots and augmented/virtual reality are reshaping the healthcare delivery landscape. The rollout of these technologies warrants updated graduate curricula to train future healthcare professionals. In response, the authors incorporated additional topics relevant to digital health intervention development into a graduate-level digital health communication course and evaluated student feedback.
Methods: The authors developed four lectures on two−/one-way digital health messaging strategies, AI/large language models, chatbots, and augmented/virtual reality, and a chatbot development tutorial as a lab. They evaluated students’ perceptions of the course and the benefits of the new content after course completion through standard and supplemental course evaluations.
Results: Eleven of 16 enrolled students completed the course evaluation, and 8 completed the supplemental survey. Most students were from the school of public health and reported female gender. One of 8 students completing the survey reported prior experience creating chatbot and AR/VR content. The overall average course rating was high (7.45 out of 9). Open-ended survey responses about the new content were mixed with enthusiasm and questions about its relevance over content on traditional communication modalities in preparation for public health work.
Conclusion: Student feedback underscored course content value, along with guidance to better emphasize how chatbots and augmented/virtual reality are relevant to clinical and public health practices. More applications relevant for diverse populations could elucidate the value of new technologies for students who will develop digital-based interventions. Applications focusing on commonalities could also solidify students’ understanding of intervention development principles that will remain, as technologies evolve.
1 Introduction
Within the past 5 years, two events have profoundly altered healthcare delivery and, in turn, the need for educational institutions to update curricula for future healthcare researchers and providers: the emergence of COVID-19 and the generative pre-trained transformer (GPT) (1), a form of generative artificial intelligence (AI). The COVID-19 pandemic accelerated the adoption of telehealth services that use digital communication technologies to overcome barriers to care imposed by social distancing, overburdened healthcare systems, and healthcare worker shortages (2, 3). Telehealth services incorporate GPTs and other AI technologies to better support healthcare delivery, e.g., to automate patient monitoring and communication (4).
Interventions designed to address health conditions through digital technology, hereafter referred to as digital health interventions, are also embedded in telehealth services and have benefited from AI advances (5–7). For example, text messages (SMS) serve as a medium for health promotion content, surveys, automated medical appointments, and medication reminders (8–10). Through AI, just-in-time adaptive interventions (JITAI) use mobile phone sensors and ecological momentary assessment to tailor SMS content and intervene in the moment when needed (11). Conversational agents, or chatbots, especially those using ChatGPT and other large language models (LLMs), enhance SMS communication by allowing patients to converse with healthcare computer systems to better meet their needs (12–14). Clinicians and researchers are also incorporating ChatGPT and LLM into augmented and virtual reality (AR/VR) scenarios to create embodied conversational agents, such as avatars that converse with patients and exhibit human expressions on their virtual faces (15). Xaia is a robot avatar and virtual health coach that patients can view and converse with through the Apple Vision Pro headset to receive mental health support between visits with a healthcare provider (16). In addition to patient-facing interventions, clinicians are developing AI-driven simulations for clinical students to bolster communication skills training and ultimately improve patient care (17, 18).
Given the rapid advances in AI-related software and AR/VR hardware, there is a growing gap between interventions that clinicians and public health researchers are developing in the field and learning how to develop and evaluate in school (19). An increasing body of literature highlights the use of AI and chatbots to improve classroom instruction (20) and a growing number of undergraduate and graduate degree programs are using AI and AR/VR tools to help students visualize anatomy and medical procedures, such as the University of California, Los Angeles (UCLA) Simulation Center and New York University’s Education Digital Experience (EDX) program. There are also programs, such as UCLA’s Master of Data Science in Health (MDSH) degree, that provide instruction on health informatics and data science and teach students how to harness AI tools and analyze data from computational systems like electronic health records. What is missing is training for future clinicians and researchers to learn about AR/VR hardware and chatbot applications and how to use them to develop and deliver AI-based digital health interventions for health education and promotion. Given the dangers and limitations of AI, e.g., those that can exacerbate biases and discriminatory practices toward under-served populations (21, 22), there is also a need for instruction on how to evaluate the appropriateness of AI technologies for healthcare and public health settings.
To date, there is a proliferation of LinkedIn and other online course content that covers chatbot and AR/VR software development. Fewer undergraduate and graduate degrees offer similar course content, especially in the health sciences, with the Cedars-Sinai Master’s Degree in Health Delivery Science (MHDS) program as an exception. As faculty in schools of medicine and public health, we have observed that our students mostly learn about intervention development through standard public health degrees, often an MPH or PhD. It’s important for school programs to provide instruction on intervention development in the context of emerging AI technologies that are becoming commonplace in clinical and public health settings. For example, healthcare organizations are using chatbots to provide remote health services to patients that would otherwise be impractical due to healthcare worker shortages (23). Research has demonstrated the benefits of VR therapy to treat mental health disorders (24) and help patients manage pain (25). Therefore, we advocate that public health curricula need health communication courses that introduce students to chatbots, AR/VR, and other AI digital health intervention tools.
To address this need, we developed new course material focused on chatbots and AR/VR as emerging digital health intervention tools within an existing health communication course that also provides foundational theories and techniques to create effective health content. The course was offered through a community health sciences department within a school of public health. We chose this course because it covered digital formats for health education and promotion. The university provides an anonymous course evaluation for students to fill out. Since the content was new, we administered an additional evaluation to gage the fit of the new content for this course, its perceived benefits to help students prepare for public health careers, and suggestions for future course modifications. We hypothesized that students would perceive the benefit of the new course content because of the growing use of chatbots and AR/VR in health interventions.
2 Materials and methods
2.1 Course overview
This study evaluated an elective four-unit graduate health communication course offered at the UCLA. This course emphasizes student mastery in understanding how the public uses news media and digital information technology to obtain health information and how public health educational materials can incorporate these delivery platforms. Students must take an introductory community health sciences course or an equivalent social sciences course as a prerequisite. No prior experience using digital information tools is necessary to enroll.
2.2 Course content and format
There are five primary objectives for students in the course: (1) design messages for health communication for different target populations, (2) learn to use new digital technologies for health communication production and distribution, (3) obtain an overview of basic principles of communication with different populations, (4) integrate theoretical perspectives into communication strategies and materials, and (5) understand access to and utilization of informatics and communication technologies among professional and general public populations. The course meets in-person weekly for 10 weeks. Instructors deliver course content through lectures over the first 2 h of each weekly session and conduct computer labs for the last hour for approximately half of the weekly sessions that introduce students to freely available software programs they can use to develop health promotion material, including GIMP, Scribus, Audacity, Shotcut, and WordPress. Table 1 summarizes lecture topics and lists software packages covered in computer labs. The course introduces students to health promotion theories and practices for developing equitable health promotion content and interventions, e.g., codesign principles for developing and pilot testing interventions with intended populations to determine feasibility before full deployment. The course also provides tutorials on the software described above, fosters class discussions about health promotion strategies and technologies, and includes assignments that guide students through the development of health education/promotion materials. Materials range from health promotion calendars/booklets to podcasts, videos, and social media campaigns. Students showcase their materials through a class presentation in the last (10th) week.
Table 1. Fifteen lecture topics and six software packages covered during lab sessions over 10-week graduate health communication course.
2.3 New course content
As illustrated in Table 2, we used a logic model to guide the addition of new course material focused on chatbots and AR/VR, which is the basis for this study (26–28). We began by assessing the need for new course content by emailing seven faculties in our professional network at different institutions who develop health interventions and have school of public health appointments. In the email, we asked them if they knew of courses on digital health intervention development and saw a need for this type of course. Correspondence established a need, best summarized by one of the email recipient’s responses. “I was a learn this on-the-fly person, but developing courses in this area sounds amazing.” Supplementary Table 1 lists email responses from all seven faculties. We delivered new content through lectures across four topics: (1) one-way versus two-way health communication in order to distinguish static interventions from those which encourages interactivity; (2) AI and LLM concepts suitable for students with limited statistical backgrounds; (3) chatbots; and (4) AR/VR. Additionally, we developed a new lab where students programmed a chatbot using Botpress, an AI chatbot development platform that lets users access basic development tools for free. The chatbot lab was based on a real-world scenario relevant to health department HIV prevention efforts. We tasked students with developing a simple version of a chatbot that would allow clients to text message the chatbot to inquire about pre-exposure prophylaxis (PrEP; Figure 1).
Table 2. Logic model used to plan and evaluate additional content added to existing health communication course on emerging digital tools (i.e., chatbots and AR/VR) that are being used to develop interventions.
Figure 1. Screenshot of Botpress graphical user interface showing conversation flow that students constructed during their chatbot lab.
2.4 Study design and population
We obtained qualitative and quantitative study data from two web-based optional surveys sent to all 16 enrolled students at the end of the Winter 2024 quarter: the university administers a course evaluation at the end of each quarter, and we administered a supplemental evaluation to students through an email invitation. The responses to the course evaluation were anonymous, while the supplemental evaluation responses were not anonymous to allow for participant compensation, i.e., $30 Amazon gift codes. The Institutional Review Board (IRB) at the UCLA determined the study to be exempt from IRB review #24-000551. The South General Institutional Review Board (IRB) at the UCLA waived the need of informed consent.
2.5 Measures
The course evaluation administered by the university contained both Likert scale questions (e.g., “To what extent do you feel that you have learned something you consider valuable?”) and a free-response question (e.g., “Please identify what you perceive to be the real strengths and weaknesses of this instructor and course.”) about the course and instructors. The course evaluation did not contain questions about sociodemographic characteristics.
The supplemental evaluation contained 25 multiple-choice questions (e.g., questions about sociodemographic characteristics and interest in the subject after taking the course) and 7 free-response questions (e.g., “What suggestions do you have for how the lectures could be improved?”) from other program evaluations we have conducted. The questions ranged from asking for feedback about the course (e.g., if there were topics students would have liked covered “in greater or less detail”) to inquiring about previous experience with various software programs, technology, and social media relevant to the field of public health.
2.6 Data analysis
We used descriptive statistics to summarize student demographic characteristics and quantitative survey responses. We calculated means and standard deviations for continuous measures and proportions and sample sizes for categorical measures. We used a mixed-methods approach to analyze quantitative and qualitative data to evaluate how quantitative course ratings aligned with qualitative responses to open-ended questions. Co-authors transcribed responses to open-ended questions and organized qualitative content using a grounded theory analysis framework to identify themes (29).
3 Results
3.1 Participant characteristics
Eleven of 16 enrolled students (68.75%) completed the course evaluation, and 8 students (50%) completed the supplemental evaluation. The number of enrolled students was comparable to class sizes for the other 17 graduate classes offered by the same department for the Winter 2024 quarter (median = 14 students; range = 3 to 51). Table 3 presents the demographic and background characteristics of the students completing the supplemental evaluation. Most students reported female gender (87.5%), with one student reporting non-binary gender. Students varied by ethnicity and race. Most students were within the Department of Community Health Sciences (75%), with one student from the social sciences and another from the school of nursing. Most students who completed the course evaluation were also in the department of community health sciences, except for two students already discussed above and a student from the school of medicine.
Table 3. Demographics and background characteristics of students who completed the supplemental evaluation (N = 8).
We did not have access to the demographic information for the 16 enrolled students. We only report that observed gender based on enrolled students’ preferred pronouns and appearances suggested the observed gender of all students to be female, except for one student identifying with non-binary gender, matching the self-identified gender composition for students completing the supplemental evaluation.
3.2 Technology experience before taking the course
Students generally had limited technical experience with digital technologies before the course, except for social media, as summarized in Table 4. Seven students (87.5%) reported no previous experience creating chatbots, while 1 (12.5%) student rated their experience level as beginner. Four students (50%) rated their experience level viewing AR/VR content through a headset as beginner, while 4 participants (50%) reported they had never viewed AR/VR through a headset before. When participants were asked about their experience level creating AR/VR content through development platforms (e.g., Unity), seven participants (87.5%) indicated they had never created virtual or augmented reality content before, and one person (12.5%) rated their experience as beginner.
Table 4. Students’ experience levels with software programs, technology, and social media before enrolling in course (N = 8).
Students came into this course with a range of experience levels and backgrounds, which allowed students to build off the knowledge of others. Some students expressed that course content was approachable and beginner-friendly, even with limited digital technology experience.
3.3 Overall course experience
Though there were differences in experience levels and interests between students before taking the course, they entered the course interested in learning about the intersection of information technology and public health. As displayed in Table 5, the average interest in the subject before taking the course was measured at 2.36 out of 3 (SD = 0.50). After completing the course, students’ interest in information technology for health promotion and communication slightly increased; participants reported an average interest in the subject at 2.45 out of 3 (SD = 0.52). The overall course rating was similarly high, with students giving it an average score of 7.45 out of 9 (SD = 1.57). For comparison, the average score of 7.45 was similar to the average of 7.51 (SD = 1.52) across the overall course rating averages of the other 17 courses offered by the same department for the Winter 2024 quarter.
Table 5. Students’ course evaluation responses to Likert scale questions (N = 11 for all questions, except N = 10 for “texts, required readings” question).
3.4 Course content feedback
Students reported feeling confident in their mastery of the course content, with their self-reported mastery of the course material averaging 2.36 out of 3 (SD = 0.50). As the course content covers different topics at the intersection of information technology and public health (e.g., health promotion social media posts and podcasts), students appreciated the breadth of topics and software programs covered in the course. This sentiment was best expressed by one student’s comment: “There was a lot of topics covered and great resources included in the lectures that I found very useful referencing later in the class as I was working on the projects.” Supplementary Tables 2–5 present all the students’ comments from the supplemental evaluation. Interest in the content extended to discussions about chatbots and AR/VR as emerging technologies. One student expanded on this, explaining they would’ve loved to learn “more about chatbots – this is an up an[d] coming area.” Students also noted these more novel technologies as interesting and exciting to learn about, particularly after seeing how they can be applied in a public health setting. Appreciation for course content aligned with course evaluation results. Students reported a high perceived value in the course’s learning outcomes, with an average rating of 7.55 out of 9 (SD = 1.29). Students’ ratings for required readings were high, with an average score of 2.55 out of 3 (SD = 0.52). Lecture presentations were rated with an average of 2.45 out of 3 (SD = 0.69).
Amidst interest in chatbots and AR/VR, some students felt that learning about these newer forms of digital technology may have taken time away from learning about other course content or strayed from the core of the class. To show, one student expressed an interest in VR but showed hesitation about its appropriateness for the course, writing, “It was exciting to see how these new tools like VR are being translated into the healthcare space. Would have really enjoyed exploring some of those tools more, though it might be out of scope for the class.” Along this same line, when providing feedback on what they would change about the content, another student stated, “Chatbot development was interesting, but maybe not the most valuable. Since there is such limited class time I would maybe offer that as a supplemental resource and not have it take up class time.” Some students wanted more class time devoted to providing more information on best practices for communicating with diverse populations, though the course touches on this topic. For example, one student noted that they were left wishing they had learned more about “forms of media for populations [that] have accessibility issues (deaf, blind, etc.) and how we can reach these populations through health communication.”
3.5 Career preparation
Students also provided insights on the applicability of aspects of the course content to their future careers. Students noted that the resources presented and exposure to digital technologies helped them develop practical skills that they can apply in the future. One student touched on this, writing, “I liked that we learned real skills, and the printed handouts and additional resources were really great to refer to, and are something I will hold onto for future use.” Additionally, some students felt that the selection of digital technologies covered in the course could be changed to prepare them for their careers better. One student touched on this, stating that they wish the course taught, “more updated and user friendly platforms. ShotCut, [A]udacity, Scribus, WordPress all felt outdated and not very easy to navigate. I think learning Adobe Photoshop, Canva, and other website platforms would have been more helpful and applicable (because i have heard of PH orgs using these platforms in the real world).” Students also expressed that they wished the course spent more time on basic design skills with conventional media to prepare them for their future roles. One student commented on this, writing, “Learning skills and principles with more conventional media make more sense for a foundational class like this, and are more useful for people looking to work on health communication materials for work. I understand that it is interesting to talk about new technologies, but learning how to make a quality brochure or website is a more marketable and applicable skill than learning about VR.”
4 Discussion
This study evaluated students perceived benefits about new course content on AI-driven health intervention technologies, including chatbots and AR/VR, that the authors added to a graduate-level health communication course in a school of public health. Mixed findings highlighted a spectrum of perceived benefits from students who were enthusiastic about the new course content to those who were not interested in it; there were students in between who wanted clarification as to its relevance for public health students. Variation in perceived benefits for learning about AI-driven health intervention technologies reflect variations in other student populations, e.g., medical students (30), and the public regarding technology adoption from “innovators” and “early adopters” to “laggards” (31).
It is important for public health educators to cultivate the enthusiasm toward technology that students who are the “innovators” and “early adopters” bring to the classroom. These students can help engage other students in new technology-focused course content and be inspired to lead technological development as future health professionals, a role that will be needed amidst a digital healthcare transformation that is underway (32, 33). Specialized courses on emerging public health innovations will be one way to accommodate “innovators” and “early adopters.” For example, the University of Michigan School of Public Health offers a “ChatGPT and Public Health” course (34).
In addition to new courses, student feedback indicated a pathway forward to incorporate new content into existing public health courses that could appeal to “innovators” and “early adopters.” Public health educators can offer supplementary course content for students who want more comprehensive coverage of certain topics. For example, we offered a single lab on chatbot development. One student suggested that the class offer an additional lab on chatbot development illustrating different software. This could have been accomplished through supplementary lab material. Between chatbots and AR/VR as the two emerging technologies we focused on, student feedback indicated that they saw greater relevance for chatbots in public health. This may be due in part to hands-on lab experience developing a chatbot that solidified public health applications we covered during lectures. We did not have a lab on AR/VR development due to the added complexities of the development process that involves visualizations, relative to the simple graphical user interface to develop text-based chatbot dialog, as depicted in Figure 1. In retrospect, we could have allocated more time to develop an introductory AR/VR lab. Some software packages have free versions suitable for introductory labs that would allow students to prototype [e.g., Figma (35)] and develop AR/VR experiences without writing code [e.g., HyperSkill (36)]. HyperSkill has a healthcare focus with pre-built avatars and scenes, including clinician and patient avatars and medical facilities that support healthcare AR/VR experiences.
A challenge in teaching an introductory course like this one is that it is intended to expose students to different digital technologies useful for health campaigns and interventions. As a result, some students received the course material better than others depending on experience levels with the digital technologies that students came into the course with, as well as the range of desired career trajectories. Students had varying opinions on which digital technologies should be prioritized and added insight into a possible solution to cover more technologies and software options by offering supplemental lab material to cover outside of the classroom. Minimizing course expenses while introducing students to industry-standard software they may use after graduating is also challenging. We utilized free programs for continual usage after course completion so that we could introduce multiple software packages to students. However, some students expressed an interest in learning to use commercial software like Adobe photograph editing and desktop publishing software versus GIMP and Scribus, respectively. Another consideration is that free versions of Botpress and other software to develop chatbots and AR/VR applications allow for exploration suitable for an introductory lab but require payment beyond that, which may be cost-prohibitive for more extensive use. Additional emphasis should be placed on understanding what these software programs can do and how learning the skills from these free programs is translatable to commercial versions. Moreover, this course highlights effective health messaging techniques, which are a cornerstone of intervention design and a skill that can be translated across all digital and non-digital media formats.
Students also provided helpful feedback for engaging students who may be later adopters of new technologies. Amidst our presentations on emerging digital technologies and software packages, we emphasized long-standing health promotion and intervention development principles. Class assignments required students to specify conceptual frameworks for health messaging, intended populations, the health needs of the population, and how chosen technologies and messaging would meet population needs. Student feedback reminded us to clarify further how new content on emerging technologies is grounded in these same health promotion development principles and can be applied in public health settings. Some students questioned how knowledge about these technologies would translate to future work, possibly considering public health settings with limited resources. As part of the “Innovator’s Dilemma” (37), early innovations offer minimal value to customers but compound in value later. Similarly, it may be that courses on innovations, as we instructed, offer greater value to students over a longer time horizon than what some students focus on, like imminent job searches.
Students also wanted to understand the application of these technologies for diverse populations, such as individuals who are deaf, blind or have other accessibility issues. For example, reception is an essential gateway for patient care from small clinics to large hospital systems. LLM chatbots have the potential to alleviate staffing shortages to coordinate patient scheduling and queries (38). AR smart glasses are a viable alternative to video remote interpreting to optimize communication for deaf patients in healthcare settings (39). By adjusting the course to include more clinical and public health case studies like these two examples, students may be able to form more connections between health communication principles, the course content, and their future careers as health professionals developing technology-driven interventions across diverse health settings and populations.
4.1 Limitations
Given the recent roll-out of the new course content, our conclusions are based on a single cohort of students. A larger sample size is needed, especially with more representation from students in clinical fields of study, to inform additional curriculum changes and if the additional content on digital health interventions should be expanded to comprise its own course. The results also need to be interpreted considering potential self-selection bias; 69 % of the students completed the course evaluation and only half of the students completed the supplemental evaluation. At least there appeared to be a comparable gender composition between all enrolled students and the subset of students who completed the supplemental evaluation. The recent roll-out of the new content also meant we could only evaluate its perceived value as a short-term outcome Table 2. We could not evaluate actual value as a long-term outcome that would best be evaluated by feedback from alumni who took the class and are developing health campaigns and interventions in the field.
4.2 Future research
The current study assessed students’ technology utilization before course enrollment to add context to the appropriateness of the course content for its intended audience. Future research should also assess students’ technology readiness (30) to help educators better evaluate how to optimize course content development and delivery based on students’ comfort and skill levels with different technologies. The current course introduced students to emerging AI-driven technologies that public health researchers are using to develop digital health interventions. Given the proliferation of these technologies, educators also need to develop course content to guide students in the selection and evaluation of digital health intervention technologies. Traditional evaluation methods for randomized controlled trials need to be supplemented with implementation science approaches to account for digital health intervention deployments beyond the confines of controlled experiments (40, 41). Proliferating AI-driven technologies also call for educators to make strategic decisions as to which technology companies they partner with that will supply technologies and software packages for educators to teach students to use in the classroom (32). Costs will continue to be a consideration but will need to be considered in concert with other factors like sustainability in an academic setting as technologies evolve.
5 Conclusion
The authors added new course content on AI-driven health intervention technology, including chatbots and AR/VR, to a graduate-level health communication course in a school of public health. The study evaluated students perceived benefits about the new course content using questionnaire survey data. Study findings were mixed as students’ excitement to learn about emerging technologies was tempered with concern. For example, some students wanted more information on chatbots, while others questioned how knowledge about chatbots and AR/VR would strengthen their future roles in public health fields. The main takeaway from the student feedback was to include more public health case studies across diverse populations in the course that incorporate chatbots, AR/VR, and other AI-driven technologies. Reinforcing the relevance of these technologies for public health professionals can increase students’ interest in the new course content. By helping students think beyond the current state of technology in healthcare and public health, instructors can better empower students to become tomorrow’s innovators and leaders in the health professions.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by South General Institutional Review Board at the University of California, Los Angeles. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants’ legal guardians/next of kin because The South General Institutional Review Board at the University of California, Los Angeles determined the study to be exempt from IRB review because the proposed study activities did not constitute a research study.
Author contributions
WC: Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. CM: Formal analysis, Writing – original draft, Writing – review & editing. CL: Conceptualization, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. Grants from the National Institute on Drug Abuse (R44DA050218), the National Cancer Institute (administrative supplement 3P30CA016042-47S3), and the National Institute of Mental Health (CHIPTS; P30MH058107) motivated course content and supported Comulada’s time.
Acknowledgments
The authors acknowledge the Harvard Macy Institute Program for Educators in the Health Professions for course content development training. The authors thank the public health students who enriched the course through their participation and feedback.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declare that no Gen AI was used in the creation of this manuscript.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2025.1577076/full#supplementary-material
References
1. Marr, B. (2023). A short history of ChatGPT: how we got to where we are today. Available online at: https://www.forbes.com/sites/bernardmarr/2023/05/19/a-short-history-of-chatgpt-how-we-got-to-where-we-are-today/ (Accessed August 7, 2024)
2. Pujolar, G, Oliver-Anglès, A, Vargas, I, and Vázquez, ML. Changes in access to health services during the COVID-19 pandemic: a scoping review. Int J Environ Res Public Health. (2022) 19:1749. doi: 10.3390/ijerph19031749
3. Shaver, J. The state of telehealth before and after the COVID-19 pandemic. Prim Care. (2022) 49:517–30. doi: 10.1016/j.pop.2022.04.002
4. Sharma, S, Rawal, R, and Shah, D. Addressing the challenges of AI-based telemedicine: best practices and lessons learned. J Educ Health Promot. (2023) 12:338. doi: 10.4103/jehp.jehp_402_23
5. Abernethy, A, Adams, L, Barrett, M, Bechtel, C, Brennan, P, Butte, A, et al. The promise of digital health: then, now, and the future. NAM Perspect. (2022) Availble at: https://nam.edu/perspectives/the-promise-of-digital-health-then-now-and-the-future/
6. Tukur, M, Saad, G, AlShagathrh, FM, Househ, M, and Agus, M. Telehealth interventions during COVID-19 pandemic: a scoping review of applications, challenges, privacy and security issues. BMJ Health Care Inform. (2023) 30:e100676. doi: 10.1136/bmjhci-2022-100676
7. Wienert, J, Jahnel, T, and Maaß, L. What are digital public health interventions? First steps toward a definition and an intervention classification framework. J Med Internet Res. (2022) 24:e31921. doi: 10.2196/31921
8. Comulada, WS, Tang, W, Swendeman, D, Cooper, A, and Wacksman, J. Development of an electronic data collection system to support a large-scale HIV behavioral intervention trial: protocol for an electronic data collection system. JMIR Res Protoc. (2018) 7:e10777. doi: 10.2196/10777
9. Ebuenyi, M, Schnoor, K, Versluis, A, Meijer, E, and Chavannes, N. Short message services interventions for chronic disease management: a systematic review. Clinical eHealth. (2021) 4. doi: 10.1016/j.ceh.2020.11.004
10. Hall, AK, Cole-Lewis, H, and Bernhardt, JM. Mobile text messaging for health: a systematic review of reviews. Annu Rev Public Health. (2015) 36:393–415. doi: 10.1146/annurev-publhealth-031914-122855
11. Wang, L, and Miller, LC. Just-in-the-moment adaptive interventions (JITAI): a meta-analytical review. Health Commun. (2020) 35:1531–44. doi: 10.1080/10410236.2019.1652388
12. Aggarwal, A, Tam, CC, Wu, D, Li, X, and Qiao, S. Artificial intelligence–based chatbots for promoting health behavioral changes: systematic review. J Med Internet Res. (2023) 25:e40789. doi: 10.2196/40789
13. Braddock, WRT, Ocasio, MA, Comulada, WS, Mandani, J, and Fernandez, MI. Increasing participation in a TelePrEP program for sexual and gender minority adolescents and young adults in Louisiana: protocol for an SMS text messaging–based chatbot. JMIR Res Protoc. (2023) 12:e42983. doi: 10.2196/42983
14. van Heerden, A, Bosman, S, Swendeman, D, and Comulada, WS. Chatbots for HIV prevention and care: a narrative review. Curr HIV/AIDS Rep. (2023) 20:481–6. doi: 10.1007/s11904-023-00681-x
15. Jiang, Z, Huang, X, Wang, Z, Liu, Y, Huang, L, and Luo, X. Embodied conversational agents for chronic diseases: scoping review. J Med Internet Res. (2024) 26:e47134. doi: 10.2196/47134
16. Spiegel, BMR, Liran, O, Clark, A, Samaan, JS, Khalil, C, Chernoff, R, et al. Feasibility of combining spatial computing and AI for mental health support in anxiety and depression. npj Digit Med. (2024) 7:1–5. doi: 10.1038/s41746-024-01011-0
17. Liaw, SY, Tan, JZ, Lim, S, Zhou, W, Yap, J, Ratan, R, et al. Artificial intelligence in virtual reality simulation for interprofessional communication training: mixed method study. Nurse Educ Today. (2023) 122:105718. doi: 10.1016/j.nedt.2023.105718
18. Weisman, D, Sugarman, A, Huang, YM, Gelberg, L, Ganz, PA, and Comulada, WS. Development of a GPT-4-powered virtual simulated patient and communication training platform for medical students to practice discussing abnormal mammogram results with patients: multiphase study. JMIR Form Res. (2025) 9:e65670. doi: 10.2196/65670
19. Li, Q, and Qin, Y. AI in medical education: medical student perception, curriculum recommendations and design suggestions. BMC Med Educ. (2023) 23:852. doi: 10.1186/s12909-023-04700-8
20. Laupichler, MC, Rother, JF, Grunwald Kadow, IC, Ahmadi, S, and Raupach, T. Large language models in medical education: comparing ChatGPT- to human-generated exam questions. Acad Med. (2024) 99:508–12. doi: 10.1097/ACM.0000000000005626
21. Chen, RJ, Wang, JJ, Williamson, DFK, Chen, TY, Lipkova, J, Lu, MY, et al. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nat Biomed Eng. (2023) 7:719–42. doi: 10.1038/s41551-023-01056-8
22. Seyyed-Kalantari, L, Zhang, H, McDermott, MBA, Chen, IY, and Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med. (2021) 27:2176–82. doi: 10.1038/s41591-021-01595-0
23. Laymouna, M, Ma, Y, Lessard, D, Schuster, T, Engler, K, and Lebouché, B. Roles, users, benefits, and limitations of chatbots in health care: rapid review. J Med Internet Res. (2024) 26:e56930. doi: 10.2196/56930
24. Spytska, L. The use of virtual reality in the treatment of mental health disorders such as phobias and post-traumatic stress disorder. SSM - Mental Health. (2024) 6:100351. doi: 10.1016/j.ssmmh.2024.100351
25. Viderman, D, Tapinova, K, Dossov, M, Seitenov, S, and Abdildin, YG. Virtual reality for pain management: an umbrella review. Front Med (Lausanne). (2023) 10:1203670. doi: 10.3389/fmed.2023.1203670
26. Dwyer, JJM, and Makin, S. Using a program logic model that focuses on performance measurement to develop a program. Can J Public Health. (1997) 88:421–5.
27. Rajashekara, S, Naik, AD, Campbell, CM, Gregory, ME, Rosen, T, Engebretson, A, et al. Using a logic model to design and evaluate a quality improvement leadership course. Acad Med. (2020) 95:1201–6. doi: 10.1097/ACM.0000000000003191
28. Rush, B, and Ogborne, A. Program logic models: expanding their role and structure for program planning and evaluation. Can J Program Eval. (1991) 6:95–106.
29. Chun Tie, Y, Birks, M, and Francis, K. Grounded theory research: a design framework for novice researchers. SAGE Open Med. (2019) 7:2050312118822927. doi: 10.1177/2050312118822927
30. MacNevin, W, Poon, E, and Skinner, TA. Technology readiness of medical students and the association of technology readiness with specialty interest. Can Med Educ J. (2021) 12:e31–41. doi: 10.36834/cmej.70624
32. Aungst, TD, and Patel, R. Integrating digital health into the curriculum – considerations on the current landscape and future developments. J Med Educat Curri Develop. (2020) 7:2382120519901275. doi: 10.1177/2382120519901275
33. Brown, TMH, and Bewick, M. Digital health education: the need for a digitally ready workforce. Arch Dis Child Educ Pract Ed. (2023) 108:214–7. doi: 10.1136/archdischild-2021-322022
34. University of Michigan School of Public Health. Students harness AI for public health with new course. (2023) Available online at: https://sph.umich.edu/news/2023posts/students-harness-ai-for-public-health-with-new-course.html (Accessed June 9, 2025)
35. Figma. Figma. Version 124.3.2. (2016). Available online at: https://www.figma.com/ (Accessed July 2024)
36. SimInsights. HyperSkill. Version 1.40.21. (2022). Available online at: https://www.siminsights.com/hyperskill/ (Accessed July 2024)
37. Christensen, CM. The innovator’s dilemma: When new technologies cause great firms to fail. Rev. ed. Boston, Mass: Harvard Business School Press (1997).
38. Wan, P, Huang, Z, Tang, W, Nie, Y, Pei, D, Deng, S, et al. Outpatient reception via collaboration between nurses and a large language model: a randomized controlled trial. Nat Med. (2024) 30:2878–85. doi: 10.1038/s41591-024-03148-7
39. Mathew, R, Dannels, WA, and Parker, AJ. “An augmented reality based approach for optimization of language access services in healthcare for deaf patients.” Paper presented at: HCI International 2023; Copenhagen, Denmark. (2023).
40. Murray, E, Hekler, EB, Andersson, G, Collins, LM, Doherty, A, Hollis, C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. (2016) 51:843–51. doi: 10.1016/j.amepre.2016.06.008
Keywords: digital health intervention, health communication, chatbot, virtual reality, artificial intelligence
Citation: Comulada WS, McQueen C and Lang CM (2025) Empowering tomorrow’s public health researchers and clinicians to develop digital health interventions using chatbots, virtual reality, and other AI technologies. Front. Public Health. 13:1577076. doi: 10.3389/fpubh.2025.1577076
Edited by:
Zubair Ahmed Ratan, University of Wollongong, AustraliaReviewed by:
Joseph Ollier, Department of Management, Technology, and Economics, ETH Zurich, SwitzerlandGillian Singleton, University of Wollongong, Australia
Copyright © 2025 Comulada, McQueen and Lang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: W. Scott Comulada, d2NvbXVsYWRhQG1lZG5ldC51Y2xhLmVkdQ==