# INTEGRATIVE LEARNING IN U.S. UNDERGRADUATE PUBLIC HEALTH EDUCATION: EFFECTIVE HIGH-IMPACT PRACTICES

EDITED BY : Andrew Harver, Katie Darby Hein and Terrel Lee Rhodes PUBLISHED IN : Frontiers in Public Health

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-426-2 DOI 10.3389/978-2-88963-426-2

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# INTEGRATIVE LEARNING IN U.S. UNDERGRADUATE PUBLIC HEALTH EDUCATION: EFFECTIVE HIGH-IMPACT PRACTICES

Topic Editors:

Andrew Harver, University of North Carolina at Charlotte, United States Katie Darby Hein, College of Public Health, University of Georgia, United States Terrel Lee Rhodes, Association of American Colleges and Universities, United States

BSPH students from UNC Charlotte participate in Global Medical Brigades program, Panama. Photo courtesy of Andrew Harver.

This current Research Topic does not derive directly from Boyer's Model of Scholarship, but nonetheless represents a well-timed exploration and example of where higher education has progressed in bringing the innovative, integrative conceptualization of higher education scholarship and practice laid out by Boyer, to realization through the growing arena of undergraduate public health programs. At the same time, the authors included here were invited to specifically address a second arena of scholarly practice associated with additional elements of Boyer's legacy, effective High-Impact Practices (HIPs) - practices that engage students, faculty and often broader communities in integrative learning that connect academic and extra-academic learning environments. Undergraduate public health programs are perfectly positioned to provide a framework for integrated learning through High-Impact Practices. Such practices encompass not only the essential learning outcomes that employers continue to demand – critical thinking, working with diverse others, written and oral communications, ethics, analysis, etc. – but also a curriculum that is scaffolded and replete with opportunities to practice and enhance performance and application of knowledge and abilities to important personal, social and global challenges and needs.

Citation: Harver, A., Hein, K. D., Rhodes, T. L., eds. (2020). Integrative Learning in U.S. Undergraduate Public Health Education: Effective High-Impact Practices. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-426-2

# Table of Contents

*04 Editorial: Integrative Learning in US Undergraduate Public Health Education: Effective High-Impact Practices* Terrel L. Rhodes, Katie Darby Hein, Donna J. Petersen and Andrew Harver

#### HIGH-IMPACT EDUCATIONAL PRACTICES


Sara L. C. Mackenzie, Deborah M. Hinchey and Kathryn P. Cornforth

*63 A Cohort Model and High Impact Practices in Undergraduate Public Health Education*

Karin Joann Opacich

#### INTEGRATED PUBLIC HEALTH CURRICULUM AND PROGRAM DEVELOPMENT


Elizabeth Armstrong-Mensah, Kim Ramsey-White and Ernest Alema-Mensah

# Editorial: Integrative Learning in US Undergraduate Public Health Education: Effective High-Impact Practices

#### Terrel L. Rhodes <sup>1</sup> , Katie Darby Hein<sup>2</sup> , Donna J. Petersen<sup>3</sup> and Andrew Harver <sup>4</sup> \*

*<sup>1</sup> Office of Quality, Curriculum and Assessment, Association of American Colleges and Universities, Washington, DC, United States, <sup>2</sup> Department of Health Promotion and Behavior, University of Georgia, Athens, GA, United States, <sup>3</sup> College of Public Health, University of South Florida, Tampa, FL, United States, <sup>4</sup> Department of Public Health Sciences, University of North Carolina Charlotte, Charlotte, NC, United States*

Keywords: Boyer's model of scholarship, curriculum development, high-impact practices, integrative learning, scholarship of teaching, undergraduate public health education

**Editorial on the Research Topic**

**Integrative Learning in US Undergraduate Public Health Education: Effective High-Impact Practices**

## INTRODUCTION

Edited and reviewed by: *Matthew Lee Smith, Texas A&M University, United States*

> \*Correspondence: *Andrew Harver arharver@uncc.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *23 October 2019* Accepted: *28 November 2019* Published: *11 December 2019*

#### Citation:

*Rhodes TL, Hein KD, Petersen DJ and Harver A (2019) Editorial: Integrative Learning in US Undergraduate Public Health Education: Effective High-Impact Practices. Front. Public Health 7:383. doi: 10.3389/fpubh.2019.00383* Nearly 30 years ago, Ernest L. Boyer introduced a model of scholarship specifically focused on how we know something is scholarship (worthy of our attention as educators); how curriculum is a form of scholarship (at its best reflective of the specific discipline); and what is the quality of the scholarship or practice (how do we know quality practice when we see it) (1). The former Association of Schools of Public Health subsequently embraced application of the model to understanding scholarship in public health practice (2). This current volume does not derive directly from those origins, but nonetheless represents a well-timed exploration and example of where higher education has progressed in bringing the innovative, integrative conceptualization of higher education scholarship and practice laid out by Boyer, to realization through the growing arena of undergraduate public health programs.

The articles that follow recognize the importance of faculty to achieve curricular, pedagogical, and hence student learning goals. One of the authors of Scholarship Assessed (3) stated in 1997:

"The goal. . . was to. . . give scholarship a. . . more efficacious meaning. . . a new paradigm of scholarship, one with four separate yet interlocking parts: the discovery of knowledge, the integration of knowledge, the application of knowledge, and the scholarship of teaching. The first two kinds of scholarship the discovery and integration of knowledge—reflect the investigative and synthesizing traditions of academic life. The third element, the application of knowledge, moves toward engagement as the scholar asks, "How can knowledge be responsibly applied to consequential problems?" Finally, the scholarship of teaching recognizes that the work of the scholar becomes consequential only as it is shared with others."

At the same time, the authors included here were invited to specifically address a second arena of scholarly practice associated with additional elements of Boyer's legacy, effective High-Impact Practices (HIPs)—practices that engage students, faculty, and often broader communities in integrative learning that connect academic and extra-academic learning environments. Research on HIPs has identified a series of criteria associated with pedagogical practices shown to be beneficial for college students from many backgrounds, especially students of color and non-traditional students who have different strengths and weaknesses compared to traditional college students (4–7). These high impact criteria include:


## CONTRIBUTIONS TO THE COLLECTION

The purpose of this Research Topic was to examine the role of effective HIPs within the curriculum of undergraduate public health programs—through original research, reports, and reviews—that promote integrative learning experiences. The final collection of curated contributions is distributed between two themes: (1) current application of high-impact educational practices in undergraduate public health education; and (2) the design and implementation of integrative undergraduate public curriculum and programs.

#### High-Impact Educational Practices

Authentic and intentional assignments are fundamental to HIPs and encourage "integrative learning," both a simple and complex approach to pedagogy. HIPs invariably target written and oral communication skills, teamwork, ethical decision-making, critical thinking, and the application of knowledge—skills uniformly valued in recent employer surveys [e.g., (8)]. Contributions that describe the development and assessment of high-impact educational practices on the authors' campuses include:

collaborative learning (interprofessional education);

diversity and global learning (study abroad);

ePortfolios (capstone coursework);

experiential leaning (research, internship, service learning, or global learning);

experiential learning (community-based learning);

experiential learning (service learning-introductory coursework);

experiential learning (service learning-capstone coursework); and

learning communities (cohort model).

### Integrated Public Health Curriculum and Program Development

A second set of contributions that address aspects of the design and implementation of authentic and intentional teaching and learning practices for students who enroll in one of the nation's fastest-growing majors include:

active learning and deliberative pedagogy; integrative curriculum reform; and student perceptions of integrative practices.

## CONCLUSION

High-Impact Practices are not high impact simply because they exhibit the criteria listed above. HIPs are highly effective when they are intentionally embedded into a curriculum—both formal and informal curricula—that brings students from an introduction to a field of study through iterative practice of knowledge and skills to demonstrated levels of quality warranting a credential.

Undergraduate public health programs are perfectly positioned to provide a framework for integrated learning that encompasses not only the essential learning outcomes that employers continue to demand—critical thinking, working with diverse others, written and oral communications, ethics, analysis, etc.—but a curriculum that is scaffolded and replete with opportunities to practice and enhance performance and application of knowledge and abilities to important personal, social, and global challenges and needs.

This Research Topic is situated, in part, within a confluence of initiatives and priorities that address demands for campuses to provide meaningful evidence of student learning that guides institutional decision-making to improve student performance. Monitoring and responding to student performance levels are not new activities for professional degree programs such as public health. As undergraduate public health degree programs continue to flourish at an unprecedented pace, however for example, "the number of undergraduate degrees awarded will probably soon meet, and likely exceed, the number of graduate degrees awarded" (9)—the need for authentic and integrative student-centered practices at the undergraduate level will similarly escalate.

We concur, and close, with Scobey's (10) recent argument,

". . . we want an educational future that draws on, and draws out, the implications of the new HIPs. Such a model would provide students with an arc of learning experiences active, collaborative, boundary-crossing, and integrative—that interweave intellectual professional, civic, and personal growth."

### AUTHOR CONTRIBUTIONS

TR, along with KH, took the lead preparing the first draft. AH edited the first draft and added additional content to the manuscript. DP critically reviewed subsequent versions of the Editorial. All authors contributed to the Editorial and approved the final version for publication.

### ACKNOWLEDGMENTS

We thank the Specialty Chief Editors (Drs. Ory and Smith) as well as the editors and reviewers for their

#### REFERENCES


expert contributions to the development of this Research Topic. We also recognize the Journal Development Specialists at Frontiers in Public Health for their expert guidance.


**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Rhodes, Hein, Petersen and Harver. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Developing a Framework for Population Health in Interprofessional Training: An Interprofessional Education Module

#### Olivia S. Anderson<sup>1</sup> , Ella August <sup>2</sup> \*, Phoebe K. Goldberg<sup>3</sup> , Emily Youatt <sup>3</sup> and Angela J. Beck <sup>3</sup>

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Iffat Elbarazi, United Arab Emirates University, United Arab Emirates Larry Kenith Olsen, Logan College of Chiropractic, United States Richard M. Schwartzstein, Harvard Medical School, United States*

> \*Correspondence: *Ella August eaugust@umich.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *15 November 2018* Accepted: *26 February 2019* Published: *21 March 2019*

#### Citation:

*Anderson OS, August E, Goldberg PK, Youatt E and Beck AJ (2019) Developing a Framework for Population Health in Interprofessional Training: An Interprofessional Education Module. Front. Public Health 7:58. doi: 10.3389/fpubh.2019.00058* *<sup>1</sup> Department of Nutritional Sciences, School of Public Health, University of Michigan, Ann Arbor, MI, United States, <sup>2</sup> Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor, MI, United States, <sup>3</sup> Department of Health Behavior and Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, United States*

Interprofessional education (IPE) is based on the concept that health professional students are best trained on the skills, knowledge, and attitudes that promote population health when they learn with and about others from diverse health science fields. Previously, IPE has focused almost exclusively on the clinical context. This study piloted and evaluated an IPE learning experience that emphasizes population health in a sample of public health undergraduate students. We hypothesized that students who completed the 2-hour online asynchronous module would better understand the value of public health's role in interprofessional teams, the benefit of interprofessional teamwork in improving health outcomes, and the value of collaborative learning with other interprofessional students. Students engaged in pre- and post-training assessments and individual reflections throughout the module. Sixty-seven undergraduate public health students completed the module and assessments. After completion, a greater proportion strongly agreed that students from different health science disciplines should be educated in the same setting to form collaborative relationships with one another (19 vs. 39% before and after completion, respectively). A greater proportion also strongly agreed that care delivered by an interprofessional team would benefit the health outcomes of a patient/client after the training (60 vs. 75% before and after, respectively). Mean scores describing how strongly students agreed with the above two statements significantly increased post-training. A greater proportion of students strongly agreed that incorporating the public health discipline as part of an interprofessional team is crucial to address the social determinants of health for individual health outcomes after taking the training (40 vs. 55% before and after, respectively). There was little change in attitudes about the importance of incorporating public health as part of an interprofessional team to address social determinants of health for population health outcomes, which were strongly positive before the training. Most students reported being satisfied with the

**7**

module presentation and felt their understanding of interprofessional practice improved. This training may be useful for students from all health disciplines to recognize the benefits of engaging with and learning from public health students and to recognize the important role of public health in interprofessional practices.

Keywords: interprofessional education, interprofessional learning, interdisciplinary, public health, professional skills, teamwork, population health, undergraduate education

### INTRODUCTION

The premise for interprofessional education (IPE) is that students from different health science backgrounds actively engage together early in their training to develop skills necessary to collaborate successfully (1, 2). Although IPE is often included within graduate-level health professional programs, mastering interprofessional competencies is important to implement early in training. This is especially the case for the health science fields, like public health, that have formalized, accredited undergraduate programs with objectives that mirror the IPE competencies. Public health education emphasizes areas like communication, teamwork, values, and health promotion (3). IPE curricular materials tailored for undergraduate education are designed to improve knowledge about interprofessional collaboration and care and to shape attitudes, behaviors, and values that support this practice in graduate training and professional practice (4–7). As the student develops professionally, these skills are poised to support all aspects of the "quadruple aim" of health care leading to (1) equitable access to care; (2) improvement in the quality of patient care; (3) enhancement of practitioner experience; and (4) improvements in population health (8).

Ample opportunities have been developed for health professional learners to engage in IPE, but most take the form of a clinically-focused case study or clinical simulated experience. The narrow emphasis on clinical learning environments prompted a movement for the Interprofessional Education Collaborative to expand their model of interprofessional care to include population health (8). Including population health enables a framework for clinical care providers, public health practitioners, and professionals from other non-clinical health fields to collaborate effectively and creatively together across disciplines to advance health at the population level. This expanded IPE model opens up the opportunity to develop interprofessional learning environments that take into careful consideration the non-clinical team members from fields like public health and their specific role in the health care process (9).

In addition to the lack of undergraduate- and nonclinicalbased IPE learning tools, IPE is often challenging to integrate into the tightly-prescribed curricula of public health programs that comply with accreditation standards (10). Online learning provides a dynamic way to offer innovative pedagogy related to IPE beyond the traditional, in-person, on-site classroom or laboratory setting. Online learning platforms are accessible, convenient, and cost-effective. These platforms offer active and engaging learning environments for diverse targeted audiences, including undergraduate health science students.

Herein, we describe a pilot and evaluation of an online, asynchronous IPE module, tested with public health undergraduate students. This module is timely, with conferred undergraduate public health degrees increasing by 11% from 2003 to 2016 (3). The module provides a didactic overview of what IPE is and how public health plays an integral role in an interprofessional team. Its availability in a portable, online format provides a practical opportunity to implement it beyond this pilot with undergraduate public health students and to engage other disciplines to learn about public health as a key component of the framework for interprofessional collaboration and care. We referenced the interprofessional learning continuum model for this study, which shows that IPE activities comprise a smaller component of foundational education initiatives early in the learning continuum, as students are being introduced to their profession (11, 12). This intervention is intended to be an initial exposure to IPE, which students build on through graduate work or on-the-job training in future years. We hypothesized that students who completed the two-hour online asynchronous module would better understand the value of public health's role in interprofessional teams, the benefit of interprofessional teamwork in improving health outcomes, and the value of collaborative learning with interprofessional students.

#### MATERIALS AND METHODS

#### Module Design

The module, Interprofessional Practice for Population Health, was developed in 2018 by the Michigan Public Health Training Center at the University of Michigan School of Public Health to address a training need among students and public health professionals. Learning objectives for the module are identified in **Table 1**.

The IPE module is a self-paced online training that takes approximately two hours to complete and is divided into five parts. The first part was developed by a University of Michigan faculty with expertise in IPE. This section provides an overview of IPE—including the four Interprofessional Education Collaborative competency domains of values/ethics, roles/responsibilities, interprofessional communication, and teams/teamwork (8). The second part was created by the Michigan Public Health Training Center's program manager and establishes the importance of interprofessional partnerships for effective population health practice and outcomes.

The remaining three parts of the module feature three realworld examples of interprofessional collaboration. Throughout the three examples, practitioners discuss benefits, challenges,


lessons learned, and recommendations for working across disciplines in order to influence community health. In Part 3, a tribal court judge and the tribal court's Healing to Wellness program manager describe how law enforcement and numerous types of health professionals work together to address opioid misuse in their community with a culturally relevant alternative sentencing program that supports participants with health and other services. In Part 4, an emergency room nurse and a peer support specialist explain the emerging recognition of human trafficking as a population health concern and several initiatives of Michigan's human trafficking task forces, which are composed of members from health, law, policy, and other fields as well as survivors themselves. Lastly, Part 5 features a public health professional and a land policy educator who explain the frameworks of Health in All Policies and Complete Streets as examples of cross-sector, team-based approaches for advancing healthier built environments. The three topics of opioid misuse, human trafficking, and Health in All Policies were selected as focus areas given their prominence in national conversations about population health outcomes and strategies, and examples in which collaboration across sectors is needed for system-level change.

#### Modality and Development

The module was created using Articulate Storyline 3, a software program used for online learning development (15). Production of the module took place across ∼ 9 months from conception to launch. The Michigan Public Health Training Center staff led the identification, recruitment, and coordination of presenters during this process. The Center's program manager worked with presenters face-to-face and by phone to shape the content in alignment with the module's learning objectives. The program manager and instructional designer then hired professional videographers to document the three practice-based vignettes. The instructional designer compiled the presentations and recordings into a single online training file, with visuals designed to have a consistent look and feel.

The first two parts of the module feature presentation slides with audio voiceover. The three remaining parts include video narrative from the practice-based professionals as well as presentation slides. Each part lasts approximately between 10 and 30 min. Brief opportunities for interaction are dispersed throughout the module in the form of multiple-choice and reflective open-ended questions. For example, after receiving information about the benefits of interprofessional and crosssector partnerships to public health practice, students are asked to write a response about how public health practitioners can work with other professions to maximize their impact in one of the 10 Essential Services of Public Health. Other interactive questions throughout the module focus on connecting the material with the concept of teamwork.

The training is hosted through the Michigan Public Health Training Center's Canvas Catalog learning management system. The final version of the training is publicly available at www.mitrainingcenter.org. The present study used a pilot version of the training module.

#### TRAINING IMPLEMENTATION

#### Participants

The module pilot was conducted with a cohort of 92 senior undergraduate students earning either a Bachelor of Arts in Community and Global Public Health or a Bachelor of Science in Public Health Sciences at the University of Michigan. In the Fall 2018 semester, Interprofessional Practice for Population Health was offered as an extra credit activity for the entire cohort of students through an academic course focused on public health practice and professional development. Incentives to participate included a certificate of completion and a seasonal snack for the class if 80% or more completed the module requirements.

Students in the academic course were invited to enroll in the training through a given link and to complete the online module requirements of a pre-survey, participation, post-survey (evaluation), and post-quiz (passing at 80%+), if interested. These are the same requirements for professionals taking the final version of the module, although the survey questions in this study were modified for the pilot undergraduate audience, as described below.

#### Evaluation Methods

Evaluation tools drew upon several existing resources. The pre- and post-surveys included one item (Q1a) adapted from the validated Student Perceptions of Physician-Pharmacist Interprofessional Clinical Education—Revised 2 (SPICE-R2) questionnaire (16). Staff from the Michigan Public Health Training Center and a University of Michigan Public Health faculty member developed three additional items to examine students' attitudes about the effect of interprofessional teamwork on patient health outcomes and the importance of public health in interprofessional teams for addressing the social determinants of health for individual and population health outcomes. Response options were a 6-point Likert scale ranging from strongly disagree to strongly agree. Post-survey questions were designed to assess participants' behavioral intentions to apply what was learned and general satisfaction with the module. The post-module quiz included questions designed to assess achievement of the learning objectives. See the **Supplementary Materials** for the specific quiz questions.

The pilot module was available to students in the academic course for 16 days, after extending the original 9-day invitation time by seven days to accommodate more student participation. Students received several reminders from their course instructor throughout that time. Students were removed from analysis if their data could not be matched across all three sources (presurvey, post-survey, post-quiz).

### Statistical Analysis

Frequencies were calculated for pre- and post-survey data. In addition, a paired t-test was used to compare mean differences in pre- and post-survey data for the items examining student attitudes. The analyses were conducted using Microsoft Excel and IBM SPSS software (version 24, IBM Corp, Armonk, NY). This study was self-determined as Not Regulated through the University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board (HUM00153475).

### RESULTS

A total of 88 public health undergraduate students enrolled in the module within the timeframe in which it was available. Of the students who enrolled, 67 (73% response rate from the 92 invited) were included in the data analysis. Over half of the students (55%; n = 37) reported spending < 2 hour to complete the module, 40% (n = 27) spent 2 hour and 5% (n = 3) spent two and a half hours or more to complete it.

Student attitudes about IPE changed after viewing the module (**Figures 1**, **2**). After participating in the module, a greater proportion of respondents strongly agreed (19 vs. 39% before and after participating, respectively) that students from different health science disciplines should be educated in the same setting to establish collaborative relationships with one another (**Figure 1**). After completing the module, a greater proportion of students strongly agreed that care delivered by an interprofessional team would benefit the health outcomes of a patient/client (60 vs. 75% before and after completing, respectively; **Figure 1**). Similarly, a greater proportion of students strongly agreed that incorporating the public health discipline as part of an interprofessional team is crucial to address the social determinants of health for individual health outcomes after taking the training (40 vs. 55% before and after taking, respectively; **Figure 2**). There was little change in attitudes that incorporating public health as part of an interprofessional team is crucial to address the social determinants of health for population health outcomes (18 vs. 19% agreed before and after completing the module, respectively, and 82 vs. 81% strongly agreed before and after completing the module, respectively; **Figure 2**). The mean scores for the first two survey items significantly increased post-training (**Table 2**). There was no change in attitudes around the importance of incorporating public health as part of an interprofessional team to address social determinants of health for population health outcomes, which were strongly positive before the training.

Most students reported being satisfied with the presentation of the module and felt they benefited from it. Sixty-seven percent reported that the presenters in the video were effective and 84% indicated that the information was presented in a clear and understandable way (**Table 3**). Three-quarters of participants reported that the module improved their understanding of interprofessional practice and 72% reported that they planned to apply the information from the module to their continuing education. Finally, 66% of students were satisfied with the module overall and 50% would recommend it to another student.

Student evaluations of the module content were very favorable (data not shown). Most students agreed or strongly agreed that the module met the four program objectives including (1) defining the four core competencies of interprofessional education (94% agreed or strongly agreed); (2) defining population health from a public health perspective (97% agreed or strongly agreed); (3) defining prevention from a public health perspective (97% agreed or strongly agreed); and (4) recognizing real-world applications of interprofessional practice (96% agreed or strongly agreed).

The knowledge check quiz consisted of ten questions and a score of 8 correct answers constituted a passing score (see **Supplementary Materials** for quiz questions). More than threequarters of students (81%; n = 54) passed the quiz on the first try and the rest of the students passed with two or more attempts. There was no limit to the number of attempts allowed to pass the quiz.

## DISCUSSION

This study represents the first online IPE intervention at this school focused on introducing to undergraduate public health students the concept IPE in practice, and specifically, how public health practitioners contribute to IPE teams. This module is unique in emphasizing a non-clinical health science discipline in an IPE learning experience. Our findings indicated a high level of interest in IPE among public health undergraduates and some understanding of the value of interprofessional practice among this student population at baseline; this understanding increased after exposure to the training. In some cases, the increase was modest, given the high level of agreement prior to the training. This finding is consistent with studies of health professions students that show positive attitudes toward IPE prior to training (17). That attitude can diminish throughout their educational program if IPE is not further cultivated within the students (17).

FIGURE 1 | Attitudes about interprofessional education before and after engaging in the module (*n* = 67). Q1a: Students from different health science disciplines should be educated in the same setting to establish collaborative relationships with one another. Q1b: Care delivered by an interprofessional team will benefit the health outcomes of a patient/client.

Overall, the module was evaluated positively by learners, with two-thirds of completers reporting satisfaction with the module and 75% reporting a better understanding of interprofessional practice. These findings are promising as this is a learning opportunity intended to provide an initial exposure to IPE, and specifically, the role that public health practitioners will have in interprofessional practice. Among professionals, trends in the field—such as Public Health 3.0—suggest the increasing importance of cross-sector partnerships in affecting change in health outcomes (18). Working together across sectors can take many forms, whether it is the integration of primary care and public health, community engagement and coalition building, or other large and small efforts that ultimately require interpersonal skills (19, 20).

Curricula in health professional degree programs are becoming more prescribed to meet changing accreditation standards and needs from the field. For instance, the Council on Education for Public Health states that undergraduate public

#### TABLE 2 | Mean differences in attitudes toward IPE pre- and post-training (*n* = 67).


*Bold values are considered statistically significant.*

TABLE 3 | Public Health undergraduate student evaluations of the IPE module (*n* = 67).


\**n* = *66.*

health students are expected to be trained in several related domains and concepts such as "teamwork and leadership," as well as characteristics of health care and public health and the "influences and responsibilities of the different agencies and branches of government," etc. (21). In addition, Master of Public Health students must be able to "perform effectively on interprofessional teams" by the end of their two-year graduate education. Given the overlap of training needs across professional and student audiences, a short 2-hour IPE module can be effective as part of a course given as an assignment or through extracurricular offerings.

#### Limitations

Although the content in this module is ultimately intended for learners across multiple health professional disciplines, the module was piloted with representative learners from only one discipline, public health. The intent in piloting this module within public health undergraduate students was to understand if the content supported knowledge acquisition of IPE and public health's role within it. The durability of learning from a single module remains unclear. Further, students self-selected into the intervention; taking the training module was encouraged though not required for the undergraduate students. We do not know if the students who elected to participate have different knowledge or familiarity with IPE compared to those who did not participate. As noted in the results, some student pre-test responses were very high for some questions before the intervention, which could either indicate prior knowledge of IPE or false perceptions of IPE knowledge and skills before the intervention. Items for the survey instrument itself were drawn from other surveys. Psychometric properties for this instrument are not available. It is possible that response bias may be a factor in the students' responses, although we do not believe that is a substantive factor in our findings.

### Future Research

Future offerings of this module will extend beyond public health students to include learners from other health professions, to determine whether similar attitudes and satisfaction with the module are evident. The evaluation of future offerings will also indicate the validity of the survey instrument used to gather post-module data. In addition, because the durability of learning from a single module is unclear, the longer-term impact of integrating IPE content into courses should be explored to determine whether exposure to foundational IPE concepts leads to more effective interprofessional practice as public health professionals.

The evaluation of this module was mainly to determine learners' attitudes, satisfaction, and perceptions toward the module and its content. While there are increasing number of IPE learning experiences across health professional students at both the undergraduate and graduate level, there is still a lack of retention of behavior and skills beyond initial IPE experiences (22). Research initiatives are currently in place at our institution to examine the application of IPE experiences within the context of professional practice of recent health professional graduates from at least four disciplines including public health.

#### DATA AVAILABILITY

The datasets generated for this study are available on request to the corresponding author.

#### AUTHOR CONTRIBUTIONS

OA (with others) developed and implemented the IPE module, generated the idea for the manuscript, co-developed the preand post-test evaluations, and drafted the introduction section and abstract. EA interpreted the results, drafted the results section and edited the final version of the manuscript. PG (with others) developed and implemented the IPE module and accompanying materials, co-developed the post-test evaluation and quiz, drafted the methods section, and assembled the appendix. EY conducted the data analysis and oversaw student recruitment. AB drafted the discussion section and implemented critical revisions to the manuscript. All authors contributed to drafting and revisions, approved the final version for publication,

#### REFERENCES


and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

### FUNDING

This module was developed by the Michigan Public Health Training Center and the Region V Public Health Training Center. It was supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number UB6HP27881 Region V Public Health Training Collaborative (total award amount \$855,000) and UB6HP31684 Public Health Training Centers (total award amount \$924,899). This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government.

#### ACKNOWLEDGMENTS

The authors would like to recognize Harmony Gmazel, Danielle Jones, Sheila Meshinski, Deborah Monroe, Seth Neeley, Patrick Shannon, Janine Sinno Janoudi, and Joseph Sowmick for their contributions to the course content. We would also like to thank Jack Wong and Adam London for their support of the pilot study.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh. 2019.00058/full#supplementary-material

students: A quasi-experimental study. Nurse Edu Today. (2017) 58:32–37. doi: 10.1016/j.nedt.2017.07.013


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Anderson, August, Goldberg, Youatt and Beck. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Integrating Diverse Study Abroad Opportunities Into Public Health Curricula: Three Distinct Strategies to Address Common Barriers

Laura Rusnak\*, J. Tory Peek, Deidre Orriola and Matawal Benjamin Makut

*Office of Undergraduate Studies, College of Public Health, University of South Florida, Tampa, FL, United States*

Background: To effectively train future leaders, undergraduate public health programs must prepare students to address challenges with cross-cultural competence and a global perspective. Study abroad programming represents a high impact practice that can be applied to any number of areas and topics within the field of public health. Infusing global learning into undergraduate curriculum, increases confidence in serving culturally diverse populations and aligns with multiple public health accreditation standards. Unfortunately, barriers often prevent integration of this high impact practice into program curriculum. This manuscript provides strategies to integrate diverse study abroad programming into public health curriculum and mitigate common barriers for students and faculty.

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Armin D. Weinberg, Baylor College of Medicine, United States Stacie Craft DeFreitas, University of Houston–Downtown, United States*

\*Correspondence:

*Laura Rusnak Lrusnak@health.usf.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *19 November 2018* Accepted: *05 February 2019* Published: *01 March 2019*

#### Citation:

*Rusnak L, Peek JT, Orriola D and Makut MB (2019) Integrating Diverse Study Abroad Opportunities Into Public Health Curricula: Three Distinct Strategies to Address Common Barriers. Front. Public Health 7:29. doi: 10.3389/fpubh.2019.00029* Methods: Faculty from the University of South Florida (USF) College of Public Health (COPH) used three strategies to improve access to global learning:


Results: A diverse array of public health-focused study abroad programs resulted from these strategies. Starting with just 12 students in the first program, the number of undergraduate participants in short-term, faculty lead courses grew to 164 students over the span of 4 academic years. This represents a 275% increase in undergraduate students participating in public health-focused study abroad courses, and a 160% increase in the percentage of public health majors participating in study abroad coursework. The primary barriers of cost, curriculum, and academic culture were addressed throughout development of each new program.

Conclusions: Infusing study abroad programs into the public health curriculum provides students with diverse opportunities to gain the skills they will need as public health practitioners. The design and implementation of the strategies that were used to successfully integrate global learning into one public health program's curriculum can help inform other schools and programs of public health on ways to increase student utilization of this high-impact practice.

Keywords: study abroad, high-impact practice, undergraduate public health education, short-term study abroad, barriers, global learning, workforce development, public health curriculum

### BACKGROUND AND RATIONALE

To effectively train the future public health leaders of tomorrow, institutions of higher education must provide students with opportunities to gain the skills needed to successfully address global challenges with cross-cultural sensitivity and an international perspective. This is especially true in the field of public health. A global health perspective is required to prepare our communities to find innovative solutions for emerging challenges. The need for additional undergraduate opportunities to engage in global learning served as the catalyst for the Association of American Colleges & Universities (AAC&U) initiative "Shared Futures: Global Learning and Social Responsibility." Through Shared Futures, AAC&U seeks to increase global learning experiences within higher education. The goal of this nation-wide initiative is to support faculty and universities in their efforts to better prepare students to remedy 21st century problems (1). One such example of global learning, study abroad, represents a high impact practice recommended by AAC&U. Students who participate in study abroad exhibit improvements in cross-cultural sensitivity (2) and confidence serving the health needs of culturally diverse populations (3). They also report a better understanding of career focus and goals (4).

Further justification for integrating study abroad into public health undergraduate curriculum can be found within the standards for Bachelor of Science in Public Health (BSPH) programs set by the Council on Education for Public Health (CEPH). This accrediting body requires students to have a cumulative and experiential learning opportunity at some point during their undergraduate public health program. Although domestic options can be both cumulative and experiential in nature, making study abroad an option to fulfill this BSPH requirement motivates students to take advantage of global opportunities. Furthermore, CEPH dictates the use of eleven foundational domains through any combination of learning experiences. Study abroad coursework addresses a minimum of two domains (although more could be easily integrated), including (1) core public health values, concepts and functions across the globe and (2) the characteristics and structure of the US health system and health systems in other countries (5). Thus, study abroad programming is well-poised to meet disciplinespecific accreditation standards for schools and programs of public health.

These national and discipline-specific calls to action have encouraged many universities to increase study abroad offerings. Indeed, the overall rate of students who currently study abroad during their undergraduate experience has tripled in the last 20 years. Despite this growth, only 10% of undergraduate students in the U.S. currently study abroad before they graduate (6). Barriers continue to prevent students, faculty, and institutions from increasing engagement in this powerful example of global learning. Faculty from the University of South Florida (USF) took practical steps to increase study abroad programming within the BSPH curriculum. The first course began as a graduatelevel offering in Panama that was changed to accommodate undergraduate students. The success of this study abroad TABLE 1 | Study abroad courses: public health topics and development strategies.


TABLE 2 | Study abroad growth.


class lead faculty to add public health coursework to an already existing, university-wide program in London, England. These two programs are described using strategy 1 below. As momentum grew, two completely new programs to Japan and Cuba were developed using academic travel companies, described in strategy 2. The Japan program continued to be offered, but shifted from using an academic travel company to being facilitated through a collaboration between university partnerships, as discussed in strategy 3. See **Table 1**.

Starting with just 12 students in the first program, the number of undergraduate participants in four different public health-focused study abroad courses grew to 164 students over the span of 4 academic years. See **Table 2**. This increased the percentage of public health majors participating in study abroad by 160%. The opportunity to engage in global learning was not limited to public health majors, however, as the courses were available to all undergraduate students on campus. From the 1st to the 4th year of implementation, there was a 275% increase in all undergraduate students participating in public health-focused study abroad courses. See **Figure 1**. The strategies described below improved access to global learning by addressing the primary barriers to engaging in this high impact practice: cost, curriculum, and culture (7). This information can help inform other schools and programs of public health on methods to increase student participation in this integrative learning experience.

### PEDAGOGICAL FRAMEWORK

The University of South Florida (USF) is a large, metropolitan university with over 50,000 students, 74% of which are undergraduate students. USF's College of Public Health has offered a Bachelor of Science in Public Health (BSPH) degree since 2011, and almost 600 undergraduate students are currently enrolled in this program. As an AAC&U member institution, committed to integrative learning, USF requires undergraduate students to complete at least two high impact practice courses before graduation. This requirement, a part of the university's general education curriculum, illustrates an example of institutional commitment to high impact practices, including study abroad coursework. In addition, USF selected the topic of global citizenship as a focus for its accreditation process by the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC). This led to the development of the Global Citizen Project (GCP), a university-wide plan to promote global citizenship through curricular and co-curricular activities. Increased funding for study abroad scholarships was made available to undergraduate students through the GCP. These initiatives set the stage for the development of new study abroad courses within the College of Public Health (COPH).

The format for each study abroad course in the COPH is similar. All courses take place within a span of 1–4 weeks, and include learning activities and assessment of learning outcomes for each phase of the course: pre-departure, in-country, and postreturn. Although the structure is similar, the delivery, educational materials, and topics are unique to each program and location (see **Table 1**). This flexibility within a standardized format ensures that learning outcomes are met. Many pre-departure and post-return activities are completed online using the university's learning management system, such as Canvas or Blackboard. This allows faculty to devote more in-country time to experiential learning rather than classroom-based didactic delivery. The extent to which online learning is used in-country, however, differs between courses depending on internet availability and restrictions of the study abroad location. For example, remote locations may lack internet, governmental organizations may restrict use of certain websites, and daily itineraries packed with site visits may limit the logistical ability to access technology. As a result, in-country assignments are adjusted to meet the needs of each program.

The selection of a study abroad destination is, in part, considered after a risk and security assessment is completed each semester a course is offered. This planning and preparation provide safety resources when traveling to areas with heightened security concerns. Before departure, all students and faculty participating in international travel through USF receive a safety orientation, health and wellness preparation, 24/7 international assistance while abroad, global health insurance, and other support services. These procedures are built into the university's travel system and are coordinated by a risk and security officer with USF's Education Abroad department.

Five faculty members at the College of Public Health have led undergraduate study abroad programs, with three additional course leaders slated to participate in 2019. What began with teaching faculty in just one department expanded to include a wider variety of faculty members within the college. Faculty begin program planning approximately 12 months before departure, followed by recruitment about 8 months in advance. Budget, course syllabus and itinerary finalization occurs around 6 months in advance. Final student selection tends to coincide with initial student payments ∼4 months pre-departure. Payments are split into installments spread throughout a semester and are completed prior to departure. While the in-country travel portion of most study abroad programs occurs during semester breaks; including spring break, May-mester, summer break, or winter break; preparation, planning, and student assessment span at least two semesters. As a result, faculty incorporate study abroad courses into their contractual teaching load. Depending on the size and location of the program, support staff, graduate assistants, or additional faculty members are included to provide assistance. As an added benefit, these individuals can take over the program the following year, which distributes workload and infuses coursework with fresh and updated material each year.

### STRATEGIES TO DEVELOP PUBLIC HEALTH-FOCUSED STUDY ABROAD PROGRAMS

#### Learning Objectives

The learning objectives include:


### Strategy #1: Add Public Health Courses to Established Study Abroad Programs

The first public health undergraduate study abroad program began as an addition to a previously established set of graduate courses. A plethora of study abroad programs to Panama were offered to graduate students within medicine, nursing, and public health. These existing programs were part of a collaborative extension between USF and Panamanian entities, offering structure around which to build a new program specifically geared toward undergraduate students.

Similarly, an established framework exists at the university's Education Abroad department, where most study abroad programs across disciplines are managed and administered. After the success of the undergraduate Public Health in Panama course, a second offering was added following an invitation to collaborate with the Education Abroad department on an existing, multi-disciplinary program set in London, England. Both programs used established frameworks to create two new public health course offerings. Each differed in program duration and collaboration with diverse stakeholders, yet were developed using the same strategy.

Adding new public health courses to an established education abroad program provided a seamless process for planning and logistics, as well as a tremendous level of security and stability. The basic program components, such as airline travel, in-country transportation, lodging, and financial management, remained consistent, minimizing administrative and logistical burden to faculty. This strategy also offers a degree of flexibility to cater specific elements to the needs of an undergraduate public health program. Depending on the program, faculty may select itinerary items from a figurative "shopping cart" of established options. Alternatively, faculty leaders can take a more active role by initiating partnerships for site visits and developing instructional options. The flexibility of developing a public health study abroad course within an established structure allows faculty to focus on pedagogical perspectives rather than detailed travel and logistical planning.

### Strategy #2: Utilize an Academic Travel Company

A variety of academic travel companies focus solely on the growing market of providing educational travel opportunities to students around the world. These organizations provide endto-end travel planning for colleges and universities seeking to integrate global experiences into their curriculum. They handle the logistics, budget, and mitigate the administrative burden of study abroad program planning. Undergraduate faculty from the College of Public Health utilized the services of academic travel companies to create two new study abroad programs The first was to Japan, one of the healthiest countries in the world (8); the second was to Cuba, a model for developing nations focusing on each citizen's right to health care with limited resources (9). The obvious differences in the two countries exemplify the diversity of study abroad programs that are possible with this option. The use of an academic travel company makes initiating public healthfocused study abroad programs achievable in almost any country around the world.

Several resources are available to assist faculty in selecting from a wide variety of travel company options. The SECUSS-L listserv (10), created by the Association of International Educators, is an open access, electronic forum for education abroad professionals. It is managed by the State University of New York at Buffalo and provides information to faculty looking for perspectives and insights into education abroad. The Forum on Education Abroad (11) also provides nine Standards of Good Practice that are recognized by the U.S. Department of Justice and the Federal Trade Commission. A review of these standards, including those for short-term programs, can assist faculty in comparing each academic travel company's ability to address academic outcomes, risk management, and other important considerations.

Most academic travel companies offer flexibility regarding which services are utilized, allowing university programs to select the level of assistance they prefer. This can include booking airfare, coordinating restaurants to accommodate diverse dietary needs, and reserving ideal group lodging. Academic travel companies are well-situated to use their local networks to meet these travel needs, as well as serve the academic needs of a study abroad course. Their in-country connections allow them to explore academic site visits that highlight course content, while faculty can provide input and ideas into suggestions that would best meet the learning outcomes of the program. Specific topics can be highlighted with powerful, hands-on experiences.

Academic travel companies also arrange for a translator to accompany students and faculty at all times. The translator greets participants at the airport upon arrival, remains with the group for the duration of the course, handles any incidents, and manages the itinerary throughout the academic experience. This individual is available to provide assistance to faculty as well, which can be especially helpful for first-time study abroad leaders. Having an in-country point of contact ensures the responsibility of orchestrating logistical needs is managed by someone best able to do so, allowing faculty to focus on pedagogy and student assessment.

### Strategy #3: Leverage Existing International and University Partnerships

Leveraging an existing international partnership presents another strategy to develop study abroad programs for undergraduate students. Existing partnerships range from formal institutional agreements between universities to informal, collegial relationships. Any level of connection could serve as the catalyst for a new global learning experience. Connections made through professional conferences, Fulbright programs, and sabbatical host institutions represent potential areas of exploration. These may include personal contacts or connections made colleague-to-colleague by faculty who have engaged in these experiences. A professional network between two colleagues led the Public Health in Japan course to transition from being managed by an academic travel company to becoming a collaborative partnership between two universities. In this instance, the Japanesebased faculty member had completed a post doctorate program at USF, through which a collegial relationship was formed with the department head of global health. The success of previous course iterations laid the groundwork for this transition. Changing the focus to one with an academic partnership facilitated more collegial and student exchange between citizens of both countries, deepening the intercultural connections. Faculty can also take advantage of available university resources in making contacts with international partners. These may include a university's Education Abroad department, global health department, or departments of international studies. At the University of South Florida, there is a dedicated education abroad office that assists faculty in formalizing international partnerships to enable smooth study abroad experiences. Formal agreements and partnerships also exist with a number of international institutions that faculty can leverage in developing new programs.

One of the highlights of this strategy is bringing together students and faculty to form professional relationships. Students learn academic content from in-country experts in the field. They also socialize with university students studying similar health topics in the unique context of their home-country environment. These structured peer-to-peer exchanges broaden cultural understanding and facilitate international collegial relationships between future public health practitioners. Faculty, along with students, benefit from the opportunity to develop academic relationships with global colleagues. These partnerships offer unique opportunities for institutions to collaborate across geographical lines.

International and university partnerships facilitate the provision of input into the planning of a study abroad program. More significant logistical needs, such as flights, accommodations, and in-country transportation, can be provided by a travel agency. International partners may be willing to provide support in areas such as language translation, in-country site visits, and other details that require familiarity with local resources. This strategy allows both universities to develop logistical arrangements tailored to the needs of their institutions.

### RESULTS

### Program Development and Student Participation

These three strategies successfully facilitated the development of undergraduate study abroad programs within the College of Public Health at USF, increasing the number of undergraduate students participating in public health-focused study abroad programs. See **Figure 1** and **Table 2**. Before the development of undergraduate, public health-focused study abroad courses, a handful of public health majors participated in study abroad programs outside of the major. A total of 15 public health majors participated in non-public health-focused study abroad programs the year before our department began employing these strategies. Public health majors participating in study abroad programs dramatically increased, however, after the employment of the strategies listed above. This number increased from this baseline of 15–21 in 2014–2015, then to 42 in 2015–2016, and 53 in 2016– 2017. The number actually decreased to 39 in 2017–2018, the reasons for which will be addressed in the Discussion section. Despite this recent decrease, the percentage of public health majors participating in study abroad increased by 160% over this 4 years period.

**Table 2** and **Figure 1** also show the total number of undergraduate students from any major participating in the public health-focused study abroad programs that were developed as a result of these strategies. The courses were available to all undergraduate majors, and tended to be attractive to students in other majors due to the interdisciplinary nature of public health and the topics covered (see **Table 1**). Students from other majors contributed to the success of these programs and their participation exposed a diverse group of undergraduates to public health content. The total number of students from any major participating in public health-focused study abroad programs grew from the first trip to Panama in 2014–2015 that hosted 12 students. This number grew to 39 undergraduate students in 2015–2016, then 68 students in 2016–2017. The academic year 2017–2018 experienced a decrease to 45 students, which will be addressed in the Discussion section below. However, the percentage of all undergraduate students who participated in public health-focused study abroad grew by 275% between the 1st and 4th year of implementation.

#### Student Outcomes

**Table 3** highlights student comments that focus on learning outcomes. These comments were obtained from course evaluation surveys and final projects in which students analyze the impact of their study abroad experiences. Students reported experiencing an increase in cross-cultural sensitivity, as well as feeling more confident in serving the health needs of culturally diverse populations. The positive impact on career focus and goals is also apparent. Students described drawing from information they learned in their public health coursework and applying it to this cumulative and experiential opportunity. The foundational domains of global public health concepts and analysis of diverse health systems are also woven throughout these comments.

### Reduction of Barriers

The strategies described above leverage strengths to minimize the barriers that often prevent students from studying abroad and faculty from developing new programs. The Institute of International Education describes the primary challenges to increasing study abroad offerings as falling within three overarching categories: cost, curriculum, and academic culture (7). This section discusses how each of the three strategies attempted to mitigate barriers for successful integration of study abroad opportunities into the undergraduate public health curriculum.

#### Cost

Of the many barriers to study abroad, program cost consistently represents the most significant deterrent to participation (12–14). In reality, however, the actual cost of a short-term international experience may not differ greatly from the resources necessary to eat, sleep, and attend school during any given semester. All the

#### TABLE 3 | Student feedback.


public health-focused study abroad programs described above are classified as short-term, with travel time spanning ∼1–4 weeks in length. While course content starts before travel begins and extends after return to the U.S., the condensed, experiential time abroad ensures program affordability. As compared to semesterlong study abroad programs, short-term options drastically reduce the cost of international education, while still meeting learning objectives (15–17). Short-term programs also reduce the time students are unable to work, earn income, take other classes, and tend to non-academic commitments. It is worth noting that faculty are also more willing to lead short-term programs for many of the same reasons.

To reduce financial burden and increase enrollment in study abroad programs, the College of Public Health waived tuition and fees for students enrolled in newly developed global courses. Once study abroad programming became an established part of the public health curriculum, tuition, and fees were reinstated, yet the popularity of these programs has continued due to positive word-of-mouth advertising. Students also receive in-depth information on seeking funding for international education. Undergraduates often fail to recognize the wealth of resources available to them, including scholarship opportunities at the national level, university sponsored funding, and organizations that support international education experiences for students. Clarifying that study abroad is not limited to those with extensive financial resources empowers students to explore all options available to them and improves the effectiveness of faculty recruitment efforts.

Careful selection of study abroad time frame, location, and learning activities can also provide a degree of variability in cost. For example, an 8 day study abroad program to Cuba over spring break creates much less financial burden than a 4 week summer program to London. Traveling during off-peak times provides an added benefit of reducing the price of flights, housing, and site visits. Learning activities also represent an area with considerable cost containment possibilities. Each year, course itineraries are examined to ensure site visits connect to learning objectives and are placed within a streamlined, cost-effective itinerary.

Courses that are part of an established study abroad program may have less control over the time frame or location, while the other two strategies provide more opportunities to lower costs. Utilizing an academic travel company allows faculty to choose locations on almost every continent at any time of the year. These organizations compete to offer cost-effective options. Planning hundreds of trips each year for a variety of groups leverages travel partnerships, resulting in competitive rates. Engaging in an existing international partnership can also present costs saving opportunities. International partners often organize a variety of in-country activities, many of which may be free of charge or moderately priced. Professional colleagues tend to be happy to provide guidance on other items such as affordable translation services, ideal location for accommodations, and inexpensive, local meal options. Each of these measures serve to reduce the most significant barrier to this high impact practice for future public health practitioners.

#### Curriculum

The perception of study abroad as a rewarding, albeit unnecessary, component of an undergraduate experience represents another barrier to expanding access to this highimpact practice. The Institute of International Education indicates that global learning should be integrated into the undergraduate experience as seamlessly as possible to overcome this barrier (7). At the University of South Florida, this occurred when the Department of Undergraduate Studies underwent a re-structuring to better align with accreditation standards. Study abroad coursework became an option that fulfills a cumulative and experiential learning requirement. This more favorably integrated study abroad coursework within the BSPH major.

In a similar manner of efficiency, curriculum design has followed a streamlined approach. All undergraduate study abroad programs within the public health major utilize the same course number. This provides a simple process for the development of classes to new locations abroad, as well as aligning competencies and curriculum across various study abroad courses. A consistent course number for all study abroad courses eases the student registration process, and burden for administration and faculty.

#### Academic Culture

Academic culture is a potential barrier that can impact faculty and students alike. It may fail to provide faculty the time and support necessary to develop study abroad programs or contribute to certain groups of students being chronically underrepresented. These barriers will be discussed, along with the strategies employed to address them.

There is no doubt that developing a new study abroad program demands a substantial time commitment on behalf of faculty. The three strategies listed above were employed to reduce this barrier. They ensured that faculty leveraged existing resources to assist in the planning process. Of all the strategies described, utilizing an academic travel company represents the most time-efficient option. It puts the in-country planning in the more capable hands of others, leaving faculty to focus on academic content. It is worth noting that recurrent programs are less time-intensive than developing new courses. In fact, some faculty lead the same study abroad program for years in order to capitalize on previous efforts. Other departments and faculty prefer a rotating schedule, giving new faculty a chance to lead programs for a few iterations before passing it on.

An important barrier, especially for junior faculty, is the misperception that leading a study abroad program is not a serious academic undertaking. It may be viewed as a distraction from time that should be devoted to research, publishing, writing, and teaching on campus. This barrier cannot be addressed by faculty alone. Leadership within departments and colleges can support faculty efforts by highlighting study abroad programming in promotion and tenure documentation. For example, both the Dean and Associate Dean of Academic Affairs at USF's College of Public Health highly value globalizing the undergraduate public health curriculum. The development of new programs is highlighted as a key strength for candidates going through the promotion process. These activities can be heralded as significant contributions to faculty service, citizenship, and workforce development. To the degree that this is possible, all effort should be made to integrate globalization of the curriculum into consideration for tenure and promotion.

Academic culture can also influence the diversity of students who study abroad. Universities can address barriers for groups that are typically underrepresented within study abroad, including racial and ethnic minorities, non-traditional students, students with disabilities, and students who identify as LGBTQ, to name just a few. Diverse faculty leaders, for example, have the potential to model international education for various student populations. When faculty leaders' racial, ethnic, linguistic, or cultural background reflects that of a particular student population, their natural social and professional networks can be utilized for recruitment and marketing. Within the study abroad programs offered at the College of Public Health, two faculty members identify as black and Hispanic. : It is interesting to note that the proportion of students who identify as black (14%) or Hispanic (23%), are higher within these public health study abroad programs than both the USF student population (11% and 20%, respectively) (18) and the national education abroad population (6% and 10%, respectively) (6). See **Figure 2**. While it is uncertain that the increases in the black and Hispanic student enrollment in the public health programs are a direct result of faculty racial and ethnic identity, it does reinforce the importance of having more faculty members of color and providing opportunities targeted to students who identify within those groups as well. When faculty strategically plan to include traditionally underrepresented groups, students of all backgrounds are more likely to participate in study abroad coursework.

#### DISCUSSION

Infusing study abroad programs into public health curricula provides students with diverse opportunities to gain the skills they will need to meet global challenges in the twenty-first century. The implementation of strategies used to successfully integrate global learning into The University of South Florida's BSPH curriculum can inform other schools of public health on ways to increase student access to this high impact practice.

There are a few components that should be considered for future practice.

As seen in **Table 2**, our programs experienced a 34% decrease in enrollment between the 3rd and 4th year of study abroad programming implementation. This was largely due to the difficulty of sustaining the exponential growth resulting from the strategies indicated above. The ability to manage the administrative side of the programs struggled to keep up with positive student response to new public health-focused study abroad courses. Although study abroad programming continues to be successful, the decrease speaks to an inability to keep up with exponential growth. A more gradual increase in courses would likely have been more sustainable. This represents an important lesson to programs and schools of public health hoping to integrate this high impact practice into undergraduate public health curriculum. The success of any study abroad program heavily relies on the availability and quality of administrative support. Even when such supports are in place, a gradual increase in offerings over the course of multiple academic years represents a more sustainable trajectory for programmatic growth.

The scope of this article, although broadly applicable, describes strategies that were available within the context of a large, metropolitan, public university. A number of characteristics may not be applicable to small or private institutions, specifically as they relate to infrastructural resources. These include access to a stand-alone Education Abroad department, and the quantity and quality of existing partnerships. Although these strategies could be used in many university settings, it is important to note the infrastructural differences among institutions may affect access to and ease of implementation of each strategy.

Although systematic and infrastructural design may influence the success of study abroad programs, these factors are constructed by leadership and organizational policy. In contrast, many institutions, specifically smaller institutions, may find limitations regarding diversity of faculty and students more challenging. USF boasts a diverse study body (see **Figure 2**) but other institutions may not experience such diversity on campus. Given this limitation, elements in this article focusing on diversity may not be broadly generalizable to smaller or majority-centric universities. To address this limitation, those planning a study abroad program can benefit from understanding their university's unique diversity profile, which populations are underrepresented, and realistically set goals for improvement. These may include expansion of marketing strategies, faculty diversification, and course content that centers on typically underrepresented groups.

## CONCLUSION

Increasing study abroad opportunities within undergraduate public health coursework need not be an insurmountable task. There are many challenges, many of which can be addressed through thoughtful and strategic program design. The three strategies provided here offer a unique starting point for study abroad courses. In addition to these, common barriers can be overcome through the following mechanism: offering short-term programs, carefully selecting the location and activities, ensuring courses fulfill a category of essential coursework, providing faculty support, and committing to student diversity. Although limitations may apply to some institutions, they should not deter from the overwhelming benefits of global learning. The high impact practice of study abroad offers a one-of-a-kind opportunity for public health students and faculty by fostering a learning experience that will last a lifetime.

### DATA AVAILABILITY

The data sets from Figure 2 from this study can be found in the USF System Facts brochure https://www.usf.edu/ods/ documents/system-facts/usf-system-facts-2017-18.pdf

### AUTHOR CONTRIBUTIONS

LR organized and lead the development of this manuscript, made substantial contributions to the conception, drafting, and revising of the work, participated in the interpretation and presentation of data that is utilized in this work. JP made substantial contributions to the conception, drafting, and revising of the work, participated in the presentation of data that is utilized in this work. DO made substantial contributions to the conception, drafting, and revising of the work, acquired the data set that is utilized in this work. MM made substantial contributions to the conception, drafting, and revising of the work.

#### ACKNOWLEDGMENTS

Dr. Kay Perrin, Ph.D., MPH, CPH, Assistant Dean of Undergraduate Studies, Associate Dean of Academic and Student Affairs, Associate Professor. In acknowledgment of unending encouragement and support of new global education efforts

#### REFERENCES


in the undergraduate public health program at the University of South Florida. USF Health Panama, Panama City, Panama. In acknowledgment of the continual fostering of collaboration among academia, international organizations, and local networks to sustain unique opportunities in international education. Drs. Mayumi Yamamoto, MD, Ph.D. and Yayoi Tajima, Ph.D. of Gifu University, Gifu Prefecture, Japan. In acknowledgment of their continued commitment to our global collaboration and support of our study abroad program through material and intellectual contributions. USF World and Education Abroad, in acknowledgement of continued opportunities and support to incorporate public health into study abroad programming across the university; and for providing the raw data on student participation.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Rusnak, Peek, Orriola and Makut. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

### NOMENCLATURE

#### Resource Identification Initiative

To take part in the Resource Identification Initiative, please use the corresponding catalog number and RRID in your current manuscript. For more information about the project and for steps on how to Additional Requirements.

# The Capstone ePortfolio in an Undergraduate Public Health Program: Accreditation, Assessment, and Audience

Andrew Harver <sup>1</sup> \*, Pilar d. Zuber <sup>1</sup> and Heather Bastian<sup>2</sup>

<sup>1</sup> Department of Public Health Sciences, University of North Carolina Charlotte, Charlotte, NC, United States, <sup>2</sup> Communication Across the Curriculum, Office of Undergraduate Education, University of North Carolina Charlotte, Charlotte, NC, United States

The Bachelor of Science in Public Health (BSPH) degree program at the University of North Carolina at Charlotte (UNC Charlotte) was launched in 2007, and was initially accredited by the Council on Education for Public Health in 2009. We admit approximately 40–45 students each fall to the upper division major, through a competitive admissions process. During the junior and senior years, BSPH majors complete a core set of required courses including internship; 18 credit hours of restricted electives; and any minor offered by the university (except public health). During 2014–2015, the Department of Public Health Sciences was one of five campus units supported by UNC General Administration to pilot the use of ePortfolios as a tool to help students integrate learning across the courses that make up the major. The pilot program continued for 2 additional years, to promote enduring faculty efforts. We subsequently outline the development and implementation of ePortfolio pedagogy in the BSPH program at UNC Charlotte, including preliminary assessment of outcomes the past 3 years. The adoption of ePortfolios has been instrumental in students' educational experiences for over 2 decades. The Association of American Colleges and Universities (AAC&U) has advocated that "ePortfolios allow faculty and other educational professionals to help students organize their learning; preserve the variety of forms in which their learning occurs; and reflect upon their learning." We have learned that effective student ePortfolios do not arise in a vacuum. In collaboration with like-minded campus colleagues including those associated with the university's Communication Across the Curriculum program, we have encountered contributing forces related to the process of "collection, selection, and reflection" including intentional assignments that yield effective student artifacts; and authentic feedback to students through adoption and modification of the AAC&U VALUE rubrics. We conclude that internal and external forces drive the development of ePortfolio content; students embrace opportunities to document learning when those opportunities are structured; the development of the ePortfolio is relational—consistent with student attributes; and ePortfolios enable evidence-based approaches to meet accreditation demands, assessment needs, and workforce expectations.

Keywords: capstone, competencies, ePortfolio, high-impact practice, reflection, undergraduate public health education

#### Edited by:

William Edson Aaronson, Temple University, United States

#### Reviewed by:

Donna Jeanne Petersen, University of South Florida, United States Katie M. Heinrich, Kansas State University, United States

> \*Correspondence: Andrew Harver arharver@uncc.edu

#### Specialty section:

This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health

Received: 04 March 2019 Accepted: 07 May 2019 Published: 31 May 2019

#### Citation:

Harver A, Zuber Pd and Bastian H (2019) The Capstone ePortfolio in an Undergraduate Public Health Program: Accreditation, Assessment, and Audience. Front. Public Health 7:125. doi: 10.3389/fpubh.2019.00125

**25**

### INTRODUCTION

In 2005, the Association of American Colleges & Universities (AAC&U) launched the Liberal Education and America's Promise (LEAP) challenge to advocate the importance and relevance of a liberal education (1). The LEAP initiative is responsive to the needs of employers for more collegeeducated workers and more engaged and informed citizens [e.g., (2)]. At the same time, the need to increase access to higher education for a broad and diverse population has informed the necessity of more diverse pedagogies and assessments to reflect student learning. High-impact educational practices (HIPs) pedagogies that have been shown to differentially engage and challenge students—include collaborative assignments, first year programs, intensive writing courses, internships, study abroad, and undergraduate research [3]. These teaching and learning practices have been widely vetted and have been shown to be beneficial for college students from many backgrounds, especially historically underserved students who often do not have equitable access to high-impact learning (3, 4). Since 2008, AAC&U has led the higher education community in promoting access to high-impact practices for all students and in testing the impact of HIPs on student learning outcomes. These efforts are especially relevant to non-traditional students who have different needs compared to traditional college students (5). In 2016, AAC&U added ePortfolios to the list of HIPs, as the eleventh high-impact practice (6).

Digital repositories of student work—ePortfolios—have addressed changes in students' educational experiences for over 2 decades (7, 8). The electronic or digital portfolio is ideal "for collecting evidence of student learning, especially for those outcomes not amenable nor appropriate for standardized measurement" <sup>1</sup> . AAC&U has advocated that "ePortfolios allow faculty and other educational professionals to help students organize their learning; preserve the variety of forms in which their learning occurs; and reflect upon their learning"<sup>1</sup> . Higher education institutions are proceeding in ways that support the ePortfolio idea. For example, faculty, other educational professionals, and institutional leaders are embracing ways to meet the capacity of students to manage their own learning and to develop their own agency and identity as they follow less predictable career paths (9, 10). The peer-reviewed International Journal of ePortfolio; recent guides, handbooks, and other publications (8, 11–13); and release of The Field Guide to ePortfolio—a publication driven by the Association for Authentic, Experiential, and Evidence-Based Learning (AAEEBL) and disseminated in fall 2017 by AAC&U involving over 50 authors from the ePortfolio field (7)—attest to an ePortfolio contagion; an epidemic of sorts. Alongside these occurrences, however, calls for rigorous ePortfolio research (e.g., do ePortfolios have an enduring value to students beyond graduation; does ePortfolio use lead to higher retention and/or graduation rates?) should not be ignored (14, 15).

During 2014–2015, the Department of Public Health Sciences was one of five campus units supported by UNC General Administration to pilot the use of ePortfolios as a tool to help students integrate learning across the courses that make up the major. The pilot program continued for 2 additional years, to promote enduring faculty efforts around not only ePortfolio pedagogy but also related contours including backward design, formative feedback, intentional assignments, and scaffolding. During spring 2016, for example, about 2 dozen faculty from across campus met bi-weekly to engage in semi-structured discourse around the recent volume by Reynolds and Patton (13). Those interactions resulted in several multidisciplinary panel presentations at professional conferences (16–19).

We subsequently outline the development and implementation of ePortfolio pedagogy in the BSPH program at UNC Charlotte, including preliminary assessment of outcomes—both qualitative and quantitative—from the past 3 years.

#### BACKGROUND AND RATIONALE

Fundamentally, the ePortfolio is an electronic collection of artifacts assembled over time intended to enhance student learning. We would be remiss to attempt, for the purposes of this paper, an adequate overview of ePortfolio pedagogy; we encourage instead that both the novice and expert consult the available guides and handbooks compiled by current thought leaders [e.g., (8, 10, 12, 13)]. The Field Guide to Eportfolio (7) in particular offers a lens—in its tone, rigor, and utility through which to view "a burgeoning and developing field of practitioners keen to explore the potential and power of ePortfolios at their institutions."

There may be no word used more frequently in the same sentence as "ePortfolio" than "reflection." In nearly every related publication, reflection—"Active, persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends" (20)—is central to ePortfolio practice and engenders the pulse of student learning [e.g., (21–23)]. At times, though, it seems that the idea of reflection is a little like fog encompassing and manifest but lacking edges or boundaries. We do not find it surprising, therefore, that there is no standalone rubric for "reflection" in the current inventory of AAC&U VALUE rubrics<sup>2</sup> .

### BSPH PROGRAM GOALS AND COMPETENCIES

The BSPH program at UNC Charlotte prepares students through didactic and practice experiences to apply core principles of public health education within a variety of community settings and to advance the public health profession. The program values professional and academic integrity and ethics, collegiality, engagement with the community, and responsiveness and innovation in its pursuit of attaining the highest possible standards of health and well-being. Practical experiences (i.e.,

<sup>1</sup>https://www.aacu.org/eportfolios.

<sup>2</sup>https://www.aacu.org/value-rubrics.

TABLE 1 | BSPH program core curriculum and elective requirements (credit hours).


internship and capstone) are designed to reinforce the knowledge and analytical skills obtained in classroom settings including study of published research.

The curriculum is designed to prepare scholar-practitioners with knowledge and skills in the core concepts of public health, including health behavior, research and statistics in health, environmental health, epidemiology, and health administration, as well as in the planning, evaluation, organization, and conduct of community and public health services. The planned course of study adopts an interdisciplinary focus and includes the development of tailored skills through the successful completion of a minor, electives, and experiential learning (see **Table 1**). Our goal is to prepare students who are particularly interested in pursuing health-related careers in health promotion, program delivery, health communication, community organization, and behavior change for entry level to mid-level positions in service and research in health departments, public health agencies, community-based organizations, outreach education programs, hospitals, private health organizations, and corporate wellness settings.

The competency-based BSPH program includes:


The UNC Charlotte Public Health Programs received its initial (5-year) accreditation from the Council on Education for Public Health in June 2009, and subsequent re-accreditation for a 7-year period, through 2021. Like all programs that meet national standards of excellence, we are continuously engaged in the assessment, review, and improvement of our competency matrices including, in our case, those specific to the undergraduate arm of our unit of accreditation. In spring 2015, for example, the BSPH Program Committee reformulated program core competencies to devise a more manageable and ultimately streamlined set of student learning outcomes to include:


#### Interdisciplinary and Cross-Cutting Competencies

Faculty advocated early for an integrative approach in the development of the BSPH program content and curriculum. For example, our institutional Student Learning Outcomes pivot on the capacity of students to respond to health-related problems; to analyze and interpret the results of studies, projects and programs related to the public's health; to design an intervention using a public health model; and to communicate public health messages to diverse audiences. Student performance in the BSPH capstone course (HLTH4600)—and completion of the capstone ePortfolio in particular—is a contributing factor in the assessment and continuous improvement of student learning. We intend for the capstone ePortfolio to document carefully selected and purposeful organization of professionally related academic accomplishments that address the required competencies of the major and—for those students who are committed to a public health practice career—the major areas of responsibility and competencies for health educators. We designed our curriculum intentionally not only around the core disciplines of public health but also around a set of focused cross-cutting competencies to support our integrative approach to the development of the next generation of public health workers.

Our BSPH program cross-cutting competencies—which are included among our CEPH accreditation reporting activities include frameworks of public health practice, communication in public health, diversity and culture, and professionalism (see **Figure 1**). Frameworks of public health practice address fundamentals that underlie public health practice including assessment of health status and application of the core functions of assessment, program and policy development, assurance, and communication in the analysis of public health problems and their solutions. Communication in public health emphasizes the ability to collect, manage, and organize data to produce information and meaning; and to gather, process, and present information to different audiences either in-person, through information technologies, or through media channels. Diversity and culture addresses the ability of public health professionals to differentiate among availability, acceptability, and accessibility of

health care across diverse populations; and to interact sensitively, effectively, and professionally with persons from diverse cultural, socioeconomic, educational, racial, ethnic, and professional backgrounds, and persons of all ages and lifestyle preferences. Finally, professionalism entails the ability to demonstrate ethical choices, values and professional practices implicit in public health decisions; to consider the effect of choices on community stewardship, equity, social justice, and accountability; and to commit to personal and institutional development.

### LEARNING ENVIRONMENT

The University of North Carolina at Charlotte (UNC Charlotte) was founded as the Charlotte Center in 1946, and joined the statewide university system in 1965. UNC Charlotte has grown to the third largest of the 17 institutions within the University of North Carolina system with a fall 2018 enrollment of 29,710 students including 5,323 graduate students distributed among 65 programs leading to Master's degrees and 24 programs leading to Doctoral degrees. UNC Charlotte enrolls more transfer students than any other North Carolina public institution and about a third of students self-report as a racial/ethnic minority.

The Department of Public Health Sciences was founded as the Department of Health Behavior and Administration on July 1, 2002 as part of the transformed College of Health and Human Services. The new Department was conceived in response to recommendations derived from UNC Charlotte's Health Commission report completed in 2000 as well as a variety of initiatives placing emphasis on population health and health behavior research. In May 2007, the Department was renamed to Public Health Sciences to better reflect the unit's larger-scale set of current and planned research programs, degree offerings, and service activities. Faculty research programs focus on individual and population health including the prevention and management of disease across the lifespan; the health status of diverse, urban communities; and population health and health care analytics. The Department values collaboration, community engagement, diversity, health equity, innovation, professionalism, and social justice.

The Bachelor of Science in Public Health (BSPH) degree program was launched in 2007, and was initially accredited by the Council on Education for Public Health (CEPH) in 2009. We admit approximately 40–45 students each fall to the upper division major, through a competitive admissions process; each year, the entering students naturally coalesce as a "cohort" (often to include establishment of "members-only" social media sites). During the junior and senior years, BSPH majors complete a core set of required courses including internship; 18 credit hours of restricted electives; and any minor offered by the university (except public health) (see **Table 1**). In 2012, the unit's BSPH program was among the case studies featured in the Successful Practices Project sponsored by the Association for Prevention

Teaching and Research in collaboration with the AAC&U<sup>3</sup> . The unit is committed to fostering a "communication enhanced curriculum" and collaborates with a renewed Communication Across the Curriculum program located in a restructured Office of Undergraduate Education<sup>4</sup> .

#### HLTH4600 CAPSTONE

The program capstone course—which occurs during the fall semester of the senior year—involves the development of a carefully selected and purposeful organization of professionally related academic accomplishments that addresses the required competencies of the major through the design, curation, and presentation of an integrated ePortfolio. We introduce the idea of a capstone ePortfolio in one of the core courses completed during the first semester of the program (HLTH 3105 Public Health Education & Promotion) to promote, at a minimum, the habit of artifact collection throughout the BSPH program as well as in all courses and co-curricular activities in which students participate.

#### Pedagogical Format

The capstone ePortfolio is designed to reflect a carefully selected and purposeful organization of professionally related curricular and co-curricular accomplishments to validate the knowledge and skills developed not only in the BSPH program but also throughout the collegiate journey. The ePortfolio becomes a digital representation of skills and accomplishments as a public health professional, as evidenced by a compilation of selected projects and activities—files, images, graphics, videos, etc. structured around the BSPH program's Interdisciplinary and Cross-Cutting Competencies. We have found value in providing students a common organizational structure that relates to the purposes of the ePortfolio. Accordingly, students organize their ePortfolio around the four Interdisciplinary and Cross-Cutting Competencies; each ePortfolio includes a section (or page) for each competency. In addition, students are required to design and develop one additional section (or page) of their choosing, to further tailor the representation of their growth and development as students and emerging professionals. We spend time exploring—through classroom discussion, review of previous student examples, and presentations of current works in progress—what the choice page could entail. Despite the talent evident among our BSPH majors, we find students are frequently challenged by the task of envisioning a self apart from the required components of the degree program. In the end, however, students uncover a facet of their individualism to complement their classroom education including, for example, career goals (e.g., health educator), international encounters (e.g., mission trips and/or study abroad), and personal qualities (e.g., leadership).

Students engage in reflective activities at two levels. First, they are required to begin each section (or page) with a oneto-two paragraph introduction. The introduction 1) summarizes the competency in their own words and otherwise introduces

<sup>4</sup>https://cxc.uncc.edu/.

the viewer to the page; 2) describes the artifacts included in the section, and how the artifacts connect to an understanding and application of BSPH program goals; and 3) integrates the collection of artifacts as indicators of preparedness for post-graduation. Each section includes a minimum of 3– 5 work samples. We remind students repeatedly "the point of the ePortfolio is to document from multiple perspectives evidence that selectively demonstrates competence to direct your own development."

Second, the final page of the ePortfolio is limited to a final 2–3 page essay designed for students to reflect on the encompassing effects of their final, integrated product. We use the following prompts to stimulate, in part, the nature and content of the final reflection:


### Course and Learning Objectives

The BSPH capstone course is configured around three objectives. First, students engage in the identification of personal and professional targets that inform short- and long-term career goals (facilitated, for example, through the conduct of informational interviews, mock interviews, and time management exercises). Second, students deliver a well-rehearsed presentation to demonstrate application of population health statistics from local, state, national, and global levels to inform a Healthy People 2020 Topic Area. Finally, students document knowledge and skills acquired throughout the BSPH program through the design, curation, and presentation of an integrated ePortfolio.

The BSPH capstone course not only contributes to CEPHrelated criteria (i.e., health communication: address the basic concepts of public health-specific communication, including technical and professional writing and the use of mass media and electronic technology) but also to institutional Student Learning Outcomes (i.e., students will be able to respond to healthrelated problems). Accordingly, the BSPH program coordinator is involved annually in the review and reporting of course objectives and outcomes.

#### Rubric and Criteria

The assessment of ePortfolios is not without challenge [e.g., (24–26)]. One approach involves the adoption of the Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics advanced by AAC&U (27, 28). During 2007–2009, teams of faculty and other educational professionals from

<sup>3</sup>https://www.aptrweb.org/page/SuccessfulPractices.

institutions across the country −2- and 4-year, private and public, research and liberal arts, large and small—developed rubrics for 16 student learning outcomes that all students need for success in work, citizenship, and life (e.g., information literacy, oral communication, quantitative literacy, teamwork, and written communication). Like at many institutions, the integrative learning VALUE rubric served as a starting point for our assessment of student learning evident in ePortfolios. Our current rubric turns on evaluation of six criteria culled and synthesized from a variety of resources (12, 13, 27–29) including artifact selection, connections to curricular and co-curricular activities, digital and visual literacy, effective communication, sense of purpose, and overall effect. We judge each criteria against the rubric, as any instructor would judge any student paper, presentation, or project. We expect as per our SLO that "90% of the students will score 80% or above on the Capstone Portfolio."

We expect that the selection of artifacts provides strong support for the claims made in the introduction to each page as well as in the final reflective analysis about student learning processes. In addition, the text associated with each individual artifact should demonstrate both technical and professional proficiencies. We have observed that the curation of artifacts is not always an easy task for students, who seem on occasion to haphazardly distribute artifacts among the requisite pages to meet the expected minimum number of work samples. We encourage students to think through as an illustration of process the relevance of the internship report, which many students include as part of the ePortfolio. We ask students to reflect, for example, if the content of the internship report best serves as a summary of the implementation of a program (frameworks), a polished piece of writing (communication), an activity involving a vulnerable population (diversity and culture), or a summary of workforce development (professionalism). In past years—especially when we were working with paperand-pencil compilations—students also sometimes struggled uncovering or recovering a critical mass of relevant artifacts (laptops had crashed; thumb drives had been lost; etc.). More recently, however, the "technology" has come into play to maximize the availability and recovery of selected material. For example, student assignments once uploaded through our learning management system (Canvas) remain available to students throughout their matriculation at the university. In addition—at least for group assignments and to some extent individual work as well—students have become adept using a variety of university-approved platforms such as Google Drive and Dropbox to archive papers, presentations, and projects during their time at UNC Charlotte.

Students readily make connections between the BSPH program goals and related curricular and co-curricular activities. Many of our students, for example, hold leadership positions in student organizations, serve as campus ambassadors, or complete study abroad programs and/or mission trips. All of our students complete a minor (or second major); and nearly all of our students work on and/or off campus at least part-time and we emphasize the value and relevance of transferable skills (e.g., assuming responsibility, customer service, interpersonal skills, interpreting policy, and negotiating).

We value as part of the building process the design, layout, and navigation of student ePortfolios to facilitate the development of digital and visual literacy; and we expect that all elements of the ePortfolio demonstrate careful editing and proofreading. It is always surprising to us that despite familiarity with social media (e.g., Facebook) the construction of a well-designed and clearly organized ePortfolio is far from intuitive for most students (even for those with a minor in graphic design). Although we align with many others who appraise the technology as secondary, some platforms afford an extensive array of design and navigational choices. We have been fortunate the past several years to implement through collaborative efforts educational software from Digication, a platform adopted by many institutions.

Because student ePortfolios are comprised mostly of text, evidence of effective communication is paramount to include organization, structure, clarity, and style. Relatedly, fully developed ePortfolios convey a sense of purpose to the reader; the writer is able to signal a strong sense of what she is trying to do or say. Our final criteria—overall effect—refers to the gestalt of the product. We confront the extent to which the entire ePortfolio is cohesive, convincing, and makes a strong case for its author's abilities, knowledge, and learning development.

#### PRELIMINARY OBSERVATIONS

In **Figure 2,** we provide screenshots of homepages for two recent BSPH student ePortfolios, from fall 2018. As is evident, in addition to the initial "welcome" or "about me" page, the top banner of our common template enables links to six additional pages. The first four are restricted to the program's four cross-cutting competencies—frameworks, communication, diversity and culture, and professionalism. As described, students also build a fifth page of their choosing, to tailor further their ePortfolio content, and are encouraged to select a theme or thread consistent with their experiences both inside and outside the BSPH program that have shaped their current goals and future aspirations. Vidhya, an aspiring MD/MPH candidate, elected to emphasize her undergraduate research experiences; Alison, an early-entry MHA student, elected to emphasize her career interests in child advocacy and policy development. The final page—reflection—serves mainly as a placeholder for students to upload the final paper at the end of the semester focused on the process of building the ePortfolio including the curation of artifacts, their organization, and their connections. Students are also required to characterize, in the final reflection, the ways in which their artifacts represent understanding and application of BSPH program goals, meet cross-cutting competency expectations, and demonstrate preparedness for post-graduation.

In each of the past 3 years we have distributed as part of the course a brief survey at the very end of the semester to collect, anonymously and voluntarily (94 of 130 total students; 72%), student ratings of outcomes (one of us—AH—was the instructor of record on all three occasions). Students rate seven questions on a five-point scale (either ranging from "strongly disagree" to "strongly agree," or ranging from "not at all" to "very much").

FIGURE 2 | Homepages for two BSPH student ePortfolios. (A) Homepage for Vidhya. Used with written informed consent of the student. (B) Homepage for Alison. Used with written informed consent of the student.

We summarize our findings from these preliminary efforts to assess ePortfolio impact in **Table 2**. Although relatively uniform, we attribute variability in the ratings evident during 2017– 2018, for example, in part to the transition to a new ePortfolio platform (like faculty, students are oftentimes frustrated with new technology). We derived our survey questions based on the Connect to Learning (C2L) Core Survey, collected from nearly 10,000 students across the campuses involved in exploring and documenting ePortfolio strategies to advance student learning (8, 30). We include for comparative purposes weighted averages from the past 3 years with equivalent values obtained from among the 24 campuses with established ePortfolio projects that participated in the C2L project [(30); **Table 2**].

As described earlier, students begin each section (or page) with a one-to-two paragraph introduction that summarizes the competency in their own words, describes the artifacts included in the section, and integrates the collection of artifacts as indicators of preparedness for post-graduation. In **Table 3,** we provide excerpts from a sample of student reflections extracted from the introduction to one cross-cutting competency (diversity and culture). We judge that there is an element of consistency among these excerpts aligned with our expectations. They demonstrate, for example, application of public health principles (e.g., "I will know how to. . . differentiate among availability, acceptability, and accessibility of health care across diverse populations;" "In order to be prosperous in public health, a person must be able to demonstrate ethical choices for the greater good of the community and to ensure social justice"). They also meet competency expectations (e.g., "Diversity is a valuable asset to any organization, but it has a unique importance in public health. A diverse workforce will be better equipped to deal with public health issues that may arise in communities") and address workforce readiness (e.g., "These experiences improved my cultural competency, cross-cultural communication, ability to navigate ambiguous situations, and creativity regarding health communication materials").

#### DISCUSSION

Although about half of all colleges report some use of ePortfolios (8), <2 dozen published papers among the over 500 ePortfolio peer-reviewed journal articles part of the PEARL (Publications on ePortfolio: Archives of the Research Landscape)<sup>5</sup> database focus on "capstone." Our report is the first to provide an overview of the development, implementation, and (preliminary) assessment of ePortfolios in a CEPH-accredited undergraduate public health capstone course. Our cumulative experiences to date—albeit within the confines of a tightly knit upper division program—compare favorably with the respected impact of the national C2L effort (8). An overwhelming proportion of undergraduate students reported particular value documenting acquired knowledge and skills through the design, curation, and presentation of an integrated capstone ePortfolio. The ePortfolio process contributed to making connections between ideas and experiences, and to thinking more deeply about the BSPH

The purpose of ePortfolios is often multi-faceted and relevant to students, agencies and institutions, and external stakeholders. There is of yet no master agreement on the key value of "ePortfolio"—both process and product is inherently adaptable, elastic, and inventive. On the other hand, few would disagree with the outline by AAC&U that the majority of work around the adoption of ePortfolios relates to student learning, institutional assessment, and professional development and employment<sup>6</sup> . We briefly address the implicit and explicit elements of these varied narratives in terms of the applicability of ePortfolios to meeting professional standards (accreditation), monitoring student learning (assessment), and cultivating student identities relevant to external stakeholders (audience).

#### Accreditation

We have found that disciplinary frameworks, program goals, and a competency-based curriculum provide a latent organizational structure for the design and layout of an ePortfolio. In this way, students selectively populate content to showcase instances of learning aligned with programmatic instructional, research, and service goals as well as articulate with clarity connections evident among course assignments, practical experiences, co-curricular activities, and transferable skills through reflective practice. Accredited degree programs of all types assure a continuous cycle of quality improvement involving students, faculty, administration, and community stakeholders. In previous years, CEPH site team members have had interest in reviewing selected paper-and-pencil portfolios. We look forward to future judgments of the perceived utility of ePortfolios to convey the proficiency of entry-level professional personnel who are able to identify, prevent, and solve community health problems.

#### Assessment

Few would argue that recent alarms over the access, cost, and quality of higher education—both imagined and real—have intensified institutional learning-outcomes assessment efforts. On the other hand, there is an increased authenticity associated with integrative assessment practices on most campuses to provide meaningful evidence of student learning that guide institutional decision-making to improve student performance. Monitoring and responding to student performance levels, however, are not new activities for professional degree programs such as public health. Student learning outcomes (e.g., students will be able to respond to health-related problems) are corroborated when assignments and rubrics are mapped to program competencies and shared for review and reflection among participating faculty.

#### Audience

We have emphasized the potential "outward facing" value of ePortfolios, especially in terms of workforce readiness. Such an emphasis is consistent with results of a recent survey involving 400 employers whose organizations have at least 25 employees

<sup>5</sup>https://eportfolio.aacu.org/

<sup>6</sup>https://www.aacu.org/eportfolios.

#### TABLE 2 | Student ratings of BSPH ePortfolio outcomes.


\*Proportion of students who rated the item as "agree" or "strongly agree." \*\*Proportion of students who rated the item as "quite a bit" or "very much." \*\*\* Eynon et al. (30).

#### TABLE 3 | Sample student reflections: culture and diversity.

Public health professionals must appreciate working alongside different organizations and agencies within the community and be able to use them as resources and for knowledge. It is important to interact with individuals with integrity and incorporate a shared understanding of the goals set in place to affect the outcome of a health issue. It is also essential to understand the availability of resources within different communities, the accessibility to health services, and the willingness of communities to take advantage of those resources in understanding how to approach working with different audiences.

Diversity is a valuable asset to any organization, but it has a unique importance in public health. A diverse workforce will be better equipped to deal with public health issues that may arise in communities. Diversity allows for the inclusion of populations often marginalized. Humans tend to be more receptive to individuals with similar cultural characteristics. The importance of diversity in public health is evident through the selection of the students in the BSPH program. BSPH students vary widely in ethnicity, age, and professional experience. We work closely with individuals within our cohort in order to foster relationships with diverse populations.

The support of the BSPH program at UNC Charlotte allowed me to take a semester abroad in Barcelona, Spain examining their public health system, followed by a summer internship in Cape Town, South Africa being directly involved with South African healthcare delivery. These experiences improved my cultural competency, cross-cultural communication, ability to navigate ambiguous situations, and creativity regarding health communication materials. My BSPH experiences only reconfirmed for me that I want to continue working with diverse and vulnerable populations.

The competencies work together to create a holistic view of a public health professional. A successful health professional encompasses strengths in each competency to produce the highest possible standard of health and well-being...Public health impacts everyone, and is not limited to one population; therefore, the ability to work with diverse populations, along with the ability to be sensitive to cultural differences, is required. In order to be prosperous in public health, a person must be able to demonstrate ethical choices for the greater good of the community and to ensure social justice. Overall, the application of each of these competencies work together to produce the outcome, which is well-being in all dimensions.

Diversity and culture address the ability of public health professionals to interact with diverse individuals and communities, with integrity and shared values, to produce or affect an intended public health outcome. As a BSPH graduate, I will know how to develop and adapt approaches to problems that take into account cultural differences and identify community assets and available resources, differentiate among availability, acceptability, and accessibility of health care across diverse populations, and appreciate the importance of working collaboratively with diverse communities and constituencies.

and report that 25% or more of their new hires hold either an associate degree from a 2-year college or a bachelor's degree from a 4-year college. Most employers (80%) say they would find an electronic portfolio of a candidate's work very (36%) or fairly (44%) useful to evaluate key skill and knowledge areas (2). We are learning that our students often now add the URL for their ePortfolio in job applications. We will be curious to track where ePortfolios might fall in the future along a line of enriched digital presences that might include, for example, LinkedIn and enhanced transcripts (31).

The ePortfolio entails an intrinsic "inward facing" value as well. The process of curating artifacts and content requires students to make organizational decisions not encountered preparing for tests or completing a 10-page research paper (32). Students often experience a new kind of discomfort when they are confronted to both discover and uncover, in a way, a mirror image (a reflection?), and the raison d'être for the ePortfolio. On the other hand, the current generation of traditional students—and we accept that they may not represent a majority of the student body at many institutions value interactive, relevant, and technology-based learning environments (33).

#### Final Thoughts

Over 20 years ago faculty throughout higher education were pressed with a rising expectation—which never gained lasting traction—that the development of effective communication skills in students, especially writing, was the responsibility of every member of the academic community. Higher education institutions are now proceeding in ways that support the ePortfolio idea, and authentic, student-centered learning. Such momentum, however, inadvertently adds to the mix of new technologies—learning management systems, degree progress verifications, and interactive learning products—that faculty are expected to master to meet new efficiencies, expedite reporting, and confront student attention spans. Although the technology that puts the ePortfolio idea into practice is secondary, the exploration and ultimate implementation of an ePortfolio course—or curriculum—competes with an ever-increasing array of advising, coaching, community engagement, mentoring, and service expectations.

We have learned that effective student ePortfolios need not arise in a vacuum, and flourish when collaborative faculty both inside and outside an academic unit establish and reinforce instructional expectations, both formally and informally. We encourage others to coordinate ePortfolio efforts with other campus initiatives and priorities to help mitigate the inevitable fatigue that accompanies early adoption of an innovation. It is important to note, however, that such endeavors—time intense, oftentimes risky, and frequently fraught with failure—are not always part of routine faculty annual evaluations, which can pose a significant challenge to the implementation and sustainability of grassroots ePortfolio initiatives. Higher levels of both faculty and student satisfaction, on the other hand, may offset the burden that often accompanies evaluation of novel pedagogical practices.

Technology choices are influenced by both "top-down" and "bottom-up" decision processes (7) and further complicate implementation and sustainability factors. As campuses scale the adoption of ePortfolios tensions can arise, as a natural sequela of complex organizations, around who fundamentally best manages ePortfolio processes or contributes most to support services and faculty development: possible players include campus equivalents of the Center for Teaching and Learning, Information Technology Services, Office of Assessment, and Writing or Communication Across the Curriculum Programs. Batson, for example, recently lamented (34):

"Institutions realized by 2003 or 2004 that student work stored on the web could be easily "mined" in aggregate for institutional assessment. In an instant, electronic portfolios became the "magic bullet" for institutional assessment. . . Nothing is wrong with that use, of course, but, as I wrote at the time, the eportfolio learning idea had been, so to speak, "hijacked" by institutional assessment needs; requiring professors to use eportfolios for institutional purposes was not the way to nurture a precious learning idea."

Despite a variety of prevailing challenges we concur, and close, with Batson's call to action (34):

"Higher education needs to lead the world, and leaders in higher education have to see that larger mission to understand what their institution must do to change. Eportfolio practice is one way to grasp the kinds of changes the entire institution must work toward."

#### CONCLUSION

We conclude that internal and external forces (e.g., institutional assessment needs and professional accreditation expectations) can drive the development of effective ePortfolio purpose and content, and that students embrace opportunities to document learning when those opportunities are structured around programmatic goals. The multi-faceted value of ePortfolios enables evidence-based approaches suitable for addressing accreditation expectations, meeting institutional assessment needs, and documenting preparedness for contributions to the workforce. Accreditation criteria and related curriculum efforts serve as ready-made organizing principles. Institutional Student Learning Outcomes reinforce programmatic goals. External stakeholders value authentic presentation of acquired skills. Unlike some high impact practices that have been shown to differentially engage and challenge students (e.g., internships and study abroad), ePortfolios are within reach of every student. The development of the ePortfolio is relational—akin to the development of creating patterns and making connections among objects and people—and, accordingly, appealing to a large portion of the current student body at institutions of higher learning. In short, we encourage the adoption of ePortfolios throughout undergraduate public health degree programs to widen the reach of effective high-impact educational practices and inclusive learning opportunities.

### DATA AVAILABILITY

All datasets analyzed for this study are included in the manuscript.

### ETHICS STATEMENT

Our study was reviewed by the Office of Research Compliance at UNC Charlotte (#19-0090) and was determined to meet the Exempt category (1. Educational setting) under 45 CFR 46.101(b).

### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

#### FUNDING

The University of North Carolina at Charlotte Atkins Library will cover the open access publication fee through the Open Access Article Publication Subvention Fund.

### ACKNOWLEDGMENTS

We acknowledge the initial financial support for our efforts provided through the Office of Undergraduate Education; the generosity of colleagues at Digication, Inc.; and like-minded faculty from across our campus who are committed to student learning. We thank Connie Rothwell for her instrumental contributions to the development of our ePortfolio program as well as comments provided on an earlier version of the manuscript.

### REFERENCES


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be interpreted as a potential conflict of interest.

Copyright © 2019 Harver, Zuber and Bastian. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Learn by Doing: A Model for Incorporating High-Impact Experiential Learning Into an Undergraduate Public Health Curriculum

#### Margaret L. Chorazy\* and Kimberly S. Klinedinst

*Undergraduate Programs, College of Public Health, University of Iowa, Iowa City, IA, United States*

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Donna Jeanne Petersen, University of South Florida, United States Charles F. Harrington, University of South Carolina Upstate, United States*

\*Correspondence:

*Margaret L. Chorazy margaret-chorazy@uiowa.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *19 November 2018* Accepted: *06 February 2019* Published: *26 February 2019*

#### Citation:

*Chorazy ML and Klinedinst KS (2019) Learn by Doing: A Model for Incorporating High-Impact Experiential Learning Into an Undergraduate Public Health Curriculum. Front. Public Health 7:31. doi: 10.3389/fpubh.2019.00031* Many accredited schools and programs of public health integrate experiential learning into the capstone experience for undergraduate public health majors; thus, the experiential learning capstone must be both cumulative and integrative. A goal of experiential learning is to foster the application of concepts and skills learned in the classroom to real-world public health situations. Students may benefit from earlier opportunities to engage in high-impact experiential learning activities. Therefore, the University of Iowa College of Public Health developed an experiential learning requirement that is separate from the capstone course. Our students' experiential learning activities do not need to be cumulative across the entire curriculum, but they should be integrative. Public health undergraduate students at the University of Iowa must successfully complete at least one of the following experiences in public health: research, internship, service learning, or global learning. This article will provide a model for the creation of an experiential learning program for undergraduate public health students that is separate from the culminating, capstone experience.

Keywords: public health education, undergraduate education, undergraduate public health, bachelors of public health, experiential learning, applied learning

### BACKGROUND AND RATIONALE

Public health education at the undergraduate level continues to grow following national recommendations to increase the proportion of 4 year colleges and universities that offer public health degree programs such that all undergraduates have access to public health education (1–3). The 2003 Institute of Medicine report also drew attention to the importance of training for the public health workforce (3).

The Council on Education for Public Health (CEPH) accredits undergraduate public health programs. An emphasis on public health training for the workplace or further education is evident in the current accreditation criteria for undergraduate programs in schools and programs of public health and standalone baccalaureate programs1,2. The criteria require that students

<sup>1</sup>https://storage.googleapis.com/media.ceph.org/wp\_assets/2016.Criteria.pdf

<sup>2</sup>https://storage.googleapis.com/media.ceph.org/wp\_assets/2018.SBPcriteria.pdf

attain two foundational competencies including (1) to communicate public health information, in both oral and written forms and through a variety of media, to diverse audiences, and (2) to locate, use, evaluate, and synthesize information. Furthermore, undergraduate public health curricula and overall undergraduate curricula should expose students to cross-cutting concepts and experiences that will contribute to success in the workplace, post-baccalaureate education, and life-long learning. Finally, students must have opportunities to integrate, synthesize, and apply knowledge through cumulative and experiential activities.

High-impact educational practices, including experiential learning activities, have been incorporated into the critical component elements identified by the Association of Schools and Programs of Public Health<sup>3</sup> and in the Liberal Education and America's Promise (LEAP)<sup>4</sup> learning objectives promoted by the Association of American Colleges & Universities. However, few published articles provide a model for experiential learning activities in public health education, and many of these focus on graduate education in public health (4–10).

Many accredited schools and programs in public health integrate experiential learning into the capstone experience for undergraduate public health majors; thus, the experiential learning capstone must be both cumulative and integrative. A goal of experiential learning is to foster the application of concepts and skills learned in the classroom to real-world public health situations. Students may benefit from earlier opportunities to engage in high-impact experiential learning activities. Especially as undergraduate education is a time of exploration, encouraging experiential activities earlier in the academic career can help students to figure out what they want to do or do not want to do after graduation. Therefore, the University of Iowa College of Public Health developed an experiential learning requirement that is separate from the capstone course. Here, we provide a model for the creation of an experiential learning program for undergraduate public health students.

### PEDAGOGICAL FRAMEWORK AND PRINCIPLES

The origins and development of experiential learning as a model for student learning can be found in the works of Dewey and Kolb among others (11, 12). As opposed to learning that occurs in the traditional classroom, experiential learning relies on experience as the source material and thoughtful reflection to facilitate learning. Experiential education is the philosophy which guides experiential learning. According to the Association for Experiential Education (AEE), experiential learning "occurs when carefully chosen experiences are supported by reflection, critical analysis, and synthesis<sup>5</sup> ". The AAE identifies 10 additional principles of experiential education practice which highlight learner independence, active learning as a process,

<sup>5</sup>https://www.aee.org/what-is-ee

intellectual and social engagement, and the personal nature of growth through experiential learning. The National Society for Experiential Education (NSEE) has also identified eight principles of good practice for experiential learning activities<sup>6</sup> . We utilized the principles of the AEE and NSEE as we developed the academic framework of our experiential learning program to support guided reflection by public health students.

### LEARNING ENVIRONMENT

### Setting

The University of Iowa is a public research university, a member of the Association of American Universities and the Big Ten Conference, and home to 33,564 students. The university's academic mission is carried out by 12 colleges, which offer undergraduate, graduate, and professional training. Threequarters of the students enrolled at the University of Iowa are undergraduates. The University of Iowa's Strategic Plan for 2016 to 2021<sup>7</sup> includes a goal to "provide a transformative educational experience that educates all University of Iowa students to be engaged citizens" and a strategy to achieve this goal by preparing students to be "experts in their disciplines and leaders in their field" through investment in high-impact practices that "promote critical thinking, problem solving, discipline-based knowledge, analysis, creativity, synthesis, and perspective taking."

The College of Public Health at the University of Iowa has a long history of outstanding graduate education in public health pre-dating the founding of our college in 1999. The College of Public Health launched two undergraduate degree programs in public health in 2016. The Bachelor of Arts and Bachelor of Science degrees in public health provide students with a basic understanding of the five core public health knowledge areas: biostatistics, social and behavioral sciences, epidemiology, health policy and management, and occupational and environmental health sciences. The program was re-accredited by CEPH in 2018.

The inaugural class of public health majors included 32 students admitted directly from high school. In 2017, we began admitting current University of Iowa students into the major. There are currently 162 undergraduate public health majors at the University of Iowa, including 50 new first-year students admitted directly from high school, 72 second-year students, and 40 third-year students. Our student population is evenly divided between the Bachelor of Arts and Bachelor of Science degree programs. We will be graduating our first cohort in Spring 2020. Our enrollment goal is approximately 500 undergraduate students in total, with 125 students per graduating cohort.

#### Supporting Personnel

The Undergraduate Program Office at the University of Iowa College of Public Health is staffed by a faculty program director (MC), a full-time professional staff academic advisor (KK), and a full-time professional staff recruiter/admissions specialist. Curricular decisions for the undergraduate program

<sup>3</sup>http://www.aspph.org/educate/framing-the-future/

<sup>4</sup>https://www.aacu.org/leap/hips

<sup>6</sup>https://www.nsee.org/8-principles

<sup>7</sup>https://provost.uiowa.edu/sites/provost.uiowa.edu/files/wysiwyg\_uploads/ StrategicPlan\_2021.pdf

are supported by the Undergraduate Program Committee which represents faculty from each department, staff, and both graduate and undergraduate students.

#### LEARNING OBJECTIVES

The experiential learning program was developed with the following programmatic objectives in mind, which map back to CEPH foundational competencies, cross-cutting concepts, and experiential activities for undergraduate public health students. Students should be able to:


### PEDAGOGICAL FORMAT

#### Overview and Expectations

Our public health majors attain experiences in public health research and/or practice by completing the experiential learning degree requirement. Students must complete at least one of the following experiential learning activities before graduation: public health research, public health internship, service learning courses related to public health, or global learning. These activities are situated outside of the course-based curriculum; however, we have assigned course numbers and titles to each experience so we are able to track student participation and completion.

It is highly recommended that students first complete CPH:2050 Second Year Undergraduate Public Health Seminar which provides an overview of experiential learning expectations and procedures as well as professional development opportunities that will help students to identify, apply for, and complete experiential learning activities. Students are encouraged to complete more than one experiential learning activity while in the degree program, when possible. Students are responsible for finding their own experiences, with support by the Undergraduate Program Office. Students must be in good academic standing during the semester in which they complete an experiential learning activity.

All students, regardless of the chosen experience, must: (1) Submit an application in order to request permission to register for academic credit, prior to the start of their experience; (2) Complete an early-term self-assessment and also be evaluated by their supervisor; (3) Submit a final written report; (4) Present their experiences in an oral or poster format; and (5) Complete an end-of-term self-assessment and also be evaluated by their supervisor. At the end of the experience, students receive of grade of "satisfactory" or "fail." Final grade assignment is made by the Undergraduate Program Director. An overview of our experiential learning program can be found online<sup>8</sup> .

#### Application and Registration

At the time of application for credit and prior to the start of the experience, students are expected to develop a personal learning plan which identifies (1) what the student will learn during the course of the experience, (2) how these learning objectives integrate with their personal educational and career goals, and (3) what specific actions, processes, and work assignments will allow the student to achieve each objective. This personal learning plan is integrated into our online application form. The application form is signed by the student's supervisor and returned to the Undergraduate Program Office for review. Student applications are reviewed and approved by the undergraduate public health academic advisor and the undergraduate program director. Once approved, the student may register their experiential learning activity. Students may register for variable credit (0–3 semester hours), however, a minimum of 80 h of activity is required for all experiences. Once registered, students gain access to an online course site in our learning management system, which we use to communicate with students regarding further assignment instructions and deadlines.

#### Student Evaluation–Skills and Professionalism

Approximately 2–4 weeks into the start of each experience, we communicate with students and their supervisors for a quick check-in via an electronic survey. The student evaluation provides an opportunity for students to evaluate their experience to-date and serves as an early check-in to address any concerns thus far about their experience. The evaluation includes questions about satisfaction with the experience and the supervisor, consistency of expectations with respect to duties as outlined in their application, and identification of unexpected challenges. Likewise, we ask the student's supervisor if the student is showing up and meeting expectations, is experiencing any significant challenges (including those related to performance and conduct), and is accepting and following guidance and supervision. We also encourage the supervisor to provide early feedback to the student. If concerns are identified on either evaluation, the Undergraduate Program Office will work with the student and supervisor to identify a plan to address these concerns.

At the conclusion of the experience, we once again communicate with students and their supervisors for a final evaluation of performance. The students are asked to evaluate their work, performance, and attitude which includes items related to learning, communication skills, problem solving skills, professional development skills, interpersonal and teamwork skills, organizational effectiveness, basic work habits, and professional conduct. This activity provides the student an opportunity to reflect on their experience holistically and to have the Undergraduate Program Office analyze their performance, learning, and professional development, which is used in assessing a grade. This evaluation also includes questions for the student to evaluate their supervisor. We encourage students

<sup>8</sup>https://www.public-health.uiowa.edu/experiential-learning-undergraduate/

to share their feedback and observations with their supervisor. Student supervisors are asked to evaluate students on the same items on work, performance, and attitude as were included on the student self-evaluation. Additionally, we ask openended questions about student strengths and weaknesses. We encourage supervisors to share their feedback and observations with the student.

#### Student Evaluation–Final Student Products

At the end of the experience, we require that students submit a 4– 5 page written report. This report includes the following sections: (1) Introduction—a background of the experience including a description of the public health issues being addressed and their significance, and how the student's experience fits with the overall research purpose (if engaged in research) or the mission of the organization (if engaged in an internship or service learning course); (2) Project Description—including but not limited to project goals, activities, student role, unanticipated challenges, and significant accomplishments; (3) Personal Assessment including how the experience has contributed to personal and professional goals (including preparation through completed coursework and skill development); understanding of public health, and attainment of experiential learning activity learning objectives; and (4) Conclusion and Future Impact—a description of the new knowledge gained through the experience and how it may impact the student in the future, including how this experience complements the student's remaining studies, campus involvement, and post-baccalaureate plans.

Students are also required to present their activities through either an oral or poster presentation in a venue pre-approved by their supervisor and/or the Undergraduate Program Office. There are numerous opportunities for students to present at the College of Public Health, the University of Iowa, statewide, or nationally. Additionally, each semester the College of Public Health Undergraduate Program Office hosts an experiential learning fair. We encourage all students who have completed an experience to present at this fair, which is attended by current undergraduate and graduate students, faculty, staff, and community practice partners.

## Overview of Experiential Learning Activities

#### Public Health Research

Engaging in public health research allows students to enrich their educational experience by integrating coursework with reallife experiences, establishing personal mentored relationships with faculty members, and applying their knowledge to make research contributions that positively impact communities. Students have two options to fulfill this experiential learning opportunity and may conduct their research at the College of Public Health, the University of Iowa, or elsewhere. CPH:3999 Undergraduate Research Experience in Public Health allows hands-on undergraduate involvement in ongoing scholarly public health research activities under the supervision of faculty, research staff, postdoctoral scholars and fellows, or graduate students (with supervision by a faculty member). Independent student research projects are not an expectation for this activity. CPH:4990 Mentored Independent Undergraduate Research in Public Health provides the framework for an independent student research project under the supervision of a faculty mentor. Undergraduate public health majors who plan to graduate with honors in the major are required to register for CPH:4990 in order satisfy the honors thesis requirement.

#### Public Health Internship

Through public health internships, students have the chance to apply classroom lessons to real-world public health issues as they work with professionals, organizations, and populations. Internships provide an opportunity to explore future careers, develop transferrable skills, establish a professional network, and gain valuable experiences and accomplishments to add to their resume. Students may complete internships locally, nationally, and globally in both traditional and non-traditional public health settings. Students register their internships under CPH:4850 Undergraduate Public Health Internship.

#### Service Learning in Public Health

Service-learning courses allow students to integrate, synthesize, and apply classroom learning by combining rigorous academic coursework with community engagement. Students register their service learning courses under CPH:3750 Undergraduate Service Learning in Public Health. To date, we have only accepted service learning courses already established at the University of Iowa which includes local, national, and global opportunities.

#### Global Learning in Public Health

Students are encouraged to pursue opportunities to work with globally diverse populations in a global (i.e., study abroad) or local context (i.e., study away). Global learning encourages students to explore cultures, life experiences, and perspectives that are different from their own. Students may pursue global research, global internships, pre-approved study abroad programs provided by third-party vendors that combine public health coursework with either internships, service learning, or independent study opportunities, or local/global opportunities with College of Public Health faculty. Completing public health coursework in an international setting (without additional experiential activities) does not meet the expectations of experiential learning. Students register these experiences under CPH:4750 Undergraduate Global Learning in Public Health.

### EXPERIENCE TO DATE

To date, 22 (13.6%) of 162 public health majors have completed their experiential learning requirement. This includes 0 of 50 first-year students, 6 (8.3%) of 72 second-year students, and 16 (40%) of 40 third-year students. At least one student has completed each of the four experiential learning activity categories: 6 students have completed internships, 15 students have completed research experiences (all of which were registered as CPH:3999), 1 student has completed a service learning course, and 1 student has completed a global learning experience. More students complete research opportunities during the academic years (14 of 15 students, 93%), as compared to the summer term. While more students complete internships during the summer term (6 of 6 students, 100%), as compared to the academic year.

When surveying second-year students who have completed CPH:2050 Second Year Undergraduate Public Health Seminar, we find that students are very interested in internships and global learning opportunities and somewhat interested in research and service learning opportunities.

Students have completed internships with local, state, and national public health agencies, as well as non-profit organizations. Students have primarily conducted research with College of Public Health faculty members. Students interested in global learning opportunities have worked closely with the university's Study Abroad office and the college's Global Public Health Coordinator to find study abroad opportunities that include experiential learning components, including internships, research, and service learning.

Current evaluation of the experiential learning program is limited to student and supervisor evaluations. Future assessment plans include a survey of graduating students and recent graduates who are approximately 1 year out of the program to determine what impact the experiential learning program has had on their professional development, short-term educational and career plans, and their level of perceived preparation for and success in their current placement.

### DISCUSSION

The goal of our experiential learning program is to foster the application of concepts and skills learned in the classroom to real-world public health situations in order to prepare students to join the public health workforce, further their education, or apply this knowledge to other areas of their future personal and professional lives within or outside of the public health field. The experiential learning program at the University of Iowa College of Public Health is separate from our cumulative capstone course. This was an intentional decision. With early experiential learning opportunities, students are able to (1) identify and try out areas of interest, (2) accumulate multiple experiences over time, (3) develop technical and transferrable skills that can be applied to their remaining coursework, and (4) speak from experience when applying for graduate and professional programs or interviewing for jobs. Additionally, this program supports CEPH foundational competencies and crosscutting concepts in a way that is adaptable to student interests and perhaps more flexible than if were intended to be both culminating and experiential.

#### Recommendations

Should other schools and programs be interested in implementing similar experiential learning programs, we offer the following recommendations.

• When possible, include a required, preparatory course in the curriculum. Our undergraduate office is small with respect to staff and other resources. In anticipation of growth in our undergraduate program, we incorporated a required seminar course in our curriculum to provide the necessary tools and skills so that our students may be confident in selfidentifying opportunities that are of interest to them, applying and competing for those opportunities, and completing them successfully.


### Final Thoughts

The purpose of this article was to share one model for the creation of an experiential learning program for undergraduate public health students. Experiential learning is a high impact educational activity that supports student success and can be tailored to meet the needs of the student and the program. We encourage other schools and programs of public health to share their experiences.

## AUTHOR CONTRIBUTIONS

MC conceived the initial concept, assisted in program design, development and implementation, drafted the manuscript, and provided critical review and revision of the manuscript. KK assisted in program design, development and implementation, and provided critical review and revision of the manuscript.

## ACKNOWLEDGMENTS

We acknowledge the contributions of the following past and current Undergraduate Program Committee members including Kathy Andrews, Grant Brown, Jason Daniel-Ulloa, Brandi Janssen, Katie McCullough, Danielle Riley, Dan Shane, Tanya Uden-Holman, Gideon Zamba, and Xi Zhu.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Chorazy and Klinedinst. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## Aloha Undergraduates: Development and Application of Local & Indigenous Topics into an Undergraduate Public Health Curriculum

#### Denise C. Nelson-Hurwitz\*, Lisa Kehl and Kathryn L. Braun

*Office of Public Health Studies, University of Hawai'i at Manoa, Honolulu, HI, United States ¯*

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Krista Mincey, Xavier University of Louisiana, United States Donna Jeanne Petersen, University of South Florida, United States*

> \*Correspondence: *Denise C. Nelson-Hurwitz denisene@hawaii.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *07 November 2018* Accepted: *21 January 2019* Published: *12 February 2019*

#### Citation:

*Nelson-Hurwitz DC, Kehl L and Braun KL (2019) Aloha Undergraduates: Development and Application of Local & Indigenous Topics into an Undergraduate Public Health Curriculum. Front. Public Health 7:16. doi: 10.3389/fpubh.2019.00016* As public health education expands to include undergraduate students, it is important to include discussion of local public health topics and issues to provide a sense of place to the educational experience. Inclusion of Native Hawaiian and indigenous issues and perspectives is also an established priority of the University of Hawai'i system. To address both needs, a required course was created during development of a new Bachelor of Arts (BA) public health program at the University of Hawai'i at Manoa to specifically focus on ¯ discussion of local and indigenous public health topics of interest. Public Health Issues in Hawai'i is an introductory course included early in the recommended undergraduate curriculum and emphasizes the application of public health skills and principles to local issues (e.g., state-level legislative awareness and local sustainability topics). The Public Health Issues in Hawai'i course further challenges students to recognize public health practice in their daily activities, and encourages them to become actively engaged in local community issues early in their public health educational careers. Among multiple advantages, improved awareness of local health challenges and early connections to community members and organizations have been instrumental in actively engaging local students in their education, and has also proved beneficial for students participating in required undergraduate applied learning capstone experiences and entry-level public health careers following graduation. Here we present insights into course development, articulation with broader program curricula, and successes and challenges in the past 4 years of implementation and instruction.

Keywords: public health education, service learning, bachelors of public health, undergraduate education, indigenous health, curriculum development, high-impact educational practices, applied learning

## BACKGROUND AND RATIONALE

During development of the Bachelor of Arts in Public Health (BAPH) program at the University of Hawai'i at Manoa (UHM), faculty sought out opportunities for students to (1) connect and ¯ apply newly acquired public health content and skills to their individual communities, and (2) gain repetitive exposure to core public health content. There was a clear need for development of a basic introduction to public health course to provide foundational exposure to core skills and concepts, however, it was limited within a semester format. To help students apply public health principles to pressing local issues, a new course was developed— PH 202: Public Health Issues in Hawai'i.

Throughout this course, students are able to apply core public health skills and concepts acquired during the prerequisite Introduction to Public Health course to local communities and gain awareness of local issues within a public health framework. In many cases, students are asked to actively engage in their local community, and participate in community events through service learning and community-based learning assignments. Inclass, oral communication skills are consistently emphasized, as is the need for students to work collaboratively with peers on in-class activities and projects.

This class also serves as a key component of our introductory core series, and as a compliment to a required global health course (1). Following a broad introduction to public health, students are asked to use and apply public health content and skills in both a local context (PH 202: Public Health Issues in Hawai'i) and a global context (PH 203: Introduction to Global Health), further providing opportunities for students to engage in issues of diversity and global learning. This dichotomy further allows students to clearly recognize the juxtaposition between application of public health in both a local and global health context.

The University of Hawai'i system has identified itself as an indigenous-serving, Hawaiian place of knowledge<sup>1</sup> PH 202 further serves as an opportunity to apply these intentions and address Native Hawaiian and indigenous health issues within the context of undergraduate public health education.

Students attending the University of Hawai'i are predominantly Hawai'i residents and frequently seek higher education with the intent of seeking employment within the state<sup>1</sup> . To best serve these students, it is helpful to support connections between students and local organizations relevant to their field of training, which may be future employers. Another key curricular component of the BAPH program at UHM is the capstone project requirement—the Applied Learning Experience (APLE) (2). Through this requirement, students engage in community internships or applied research experiences. Early exposure to local public health organizations through PH 202 also provides students with an awareness of sites with which to collaborate as relevant to their capstone projects.

#### PEDAGOGICAL FRAMEWORK

Multiple high-impact educational practices are applied throughout this course. As a highly diverse state with a large indigenous population and substantial minority and immigrant populations, diversity and global learning are of predominant importance throughout the course topics. Indigenous and immigrant health, culturally appropriate practices, and historical context of local public health issues are interwoven throughout the curriculum.

PH 202 also incorporates the critical component elements (CCEs) identified by the Association of Schools and Programs of Public Health (ASPPH)<sup>2</sup> , as well as Liberal Education and America's Promise (LEAP) learning objectives (3, 4). These objectives relate to inquiry and analysis, critical and creative thinking, written and oral communication, quantitative literacy, information literacy, teamwork, problem solving, social responsivity, and integrative learning.

To integrate and involve students actively in the local community, the course also capitalizes on service learning and community-based learning, both high impact educational practices (5). The clearest example of this is through the required 'aina (land) connection experience, where students volunteer in environmental community workdays, then reflect on their experiences. Students also engage with current topics and issues in pairs or small groups through activities including community windshield assessments, public-health-in-the-news assignments, and community challenge/success presentations. Many of these assignments also apply collaborative learning practices, another high impact educational practice, as students work in small groups to design culturally tailored public service announcements and collaborate in legislative budget balancing activities. Collaborative learning also plays a key role in this course, as an early requirement in the BAPH curriculum. Teambased experiential learning is further supported in pedagological effectiveness when applied in higher education (6, 7), and has been specifically applied in undergraduate public health coursework (8). Through group activities, students often develop a cohort effect and relationships with peers that continue through their undergraduate experience.

#### Developing the Course

Course development and design was led by formative research, including a literature review, web searches, best practices in pedagogical techniques, CEPH standards, key informant interviews with stakeholders, and review by departmental committees. Design and curriculum content were guided by recommendations from the literature (9–13) and the ASPPH Framing the Future Task Force<sup>2</sup> . The course was intended to build on topics and skills introduced during the prerequisite Introduction to Public Health course, allowing students a second exposure to key concepts in public health, as well as an opportunity to both observe and participate in local application of critical public health skills and concepts (14). Pedagogical approaches were incorporated to address variability among students in regards to learning style, with particular emphasis on application of experiential learning techniques and regular opportunities for student reflection.

Interviews were also conducted with key faculty specializing in Native Hawaiian and Indigenous Health, local community leaders, and university administration to ensure the developed course was culturally appropriate, incorporated an indigenous perspective, included strong local connections, and articulated well with existing curricular requirements and institutional learning objectives.

<sup>1</sup>http://manoa.hawaii.edu/about

<sup>2</sup>http://www.aspph.org/educate/framing-the-future/

### LEARNING ENVIRONMENT

The University of Hawai'i at Manoa (UHM) is a public ¯ research university and the flagship campus for the University of Hawai'i system. Student enrollment is 17,612 students, with undergraduate students comprising about 73% of enrollees<sup>3</sup> Demographics reflect a student body comprised primarily of residents<sup>3</sup> and high ethnic diversity (15). The University of Hawai'i system has identified itself as an indigenousserving institution and as a Hawaiian place of learning, clearly communicating the prioritization within the administration and across the multiple campuses.

Public Health Issues in Hawai'i (PH 202) is a core, required course within the BAPH curriculum at UHM. It additionally serves as a component of our introductory core course series. The two additional courses of the series includes Introduction to Public Health (PH 201) as a prerequisite, and Introduction to Global Health (PH 203) as a course taken concurrently with, or following completion of, PH 202. Three public health faculty with experience in undergraduate instruction teach each of the three introductory public health courses independently. Logistically, PH 202 is taught in two class sessions of 75 min each per week, and offered during the fall and spring semesters, with additional offerings in the summer dependent on instructor availability. Since inception, over 300 students have enrolled in PH 202, and class size ranges from 22 to 42 students per semester.

### LEARNING OBJECTIVES

The course has 12 learning objectives: (1) Develop a respect for culture and place in Hawai'i; (2) Understand health disparities caused by historical and cultural trauma among indigenous peoples, particularly those of Pacific Island descent; (3) Discuss the implications of culture on health and identify the need for culturally sensitive care; (4) Connect the need for sustainability and the importance of environmental health to human health; (5) Develop connections between students and local communities, and encourage student participation as engaged citizens; (6) Identify local community strengths and resources; (7) Promote self-reflection to identify and recognize personal skills and how they may be applied to contribute to local communities; (8) Develop an understanding of the legislative process, and an appreciation for the complexities of that process; (9) Translate social justice issues and health priorities into actionable steps; (10) Empower students to engage in health policy and advocacy; (11) Develop confidence in oral communication and public speaking; and (12) Develop skills in teamwork and collaboration among peers.

### PEDAGOGICAL FORMAT

Throughout this course, students are involved in applying core public health skills and concepts to the local community, while improving awareness of local issues within a public health framework. To achieve this, as well as to meet the learning objectives above, several key assignments serve to anchor the course.

### "Public Health in Action" Photovoice Assignment

For the first assignment of the semester, students are asked to explore their community and identify three observations of "public health in action." They take pictures of their observations, upload them into Google Slides, and caption the images to explain their observations. In class, each student reports back to the class using their Google Slides presentation and explaining their findings. This is followed by a class discussion about both similar and unique observations among students. The objective of this assignment/activity is to get the students to begin to explore their community, and to get them to realize that public health is everywhere.

#### Public Health in the News Presentations

In an effort to increase student awareness of local, contemporaneous public health issues across the state, students are asked to use local media sources to complete a brief worksheet. The students then present their stories in class as a panel guest hosting the fictitious, local evening news. This activity additionally serves as an opportunity for students to engage with university library resources, and practice accessing high quality local media sources for use in an academic setting.

#### 'Aina (Land) Connection Experience

In an effort to promote experiential learning outside of the classroom, to encourage students to engage in local community efforts, and to promote a community-grounded understanding of Hawaiian values and culture, students are required to participate in one 'aina (Land) Connection Experience at some point during the semester. Approved 'aina (land) connection experiences include a session (generally 2–4 h) of volunteer work at the University of Hawai'i at Manoa's ¯ lo'i (taro patch), located adjacent to campus, or engagement in environmental community workdays. Past events have also included organized beach clean ups and dedicated workdays at community gardens/farms or i'a loko i'a (indigenous fish ponds). Students have the opportunity to request that similar, self-identified experiences be counted with instructor approval. As evidence of participation, students are asked to provide a photo of themselves taken at the site during the activity, and to write a brief reflection of their experience.

Culturally, Native Hawaiians and other Pacific Islanders feel strong connections to their natural environment (16). By encouraging students to actively engage in their local communities, to work alongside their neighbors, and to participate in service activities centered on sustainability, students are able to develop a deeper understanding of the Pacific culture and stronger appreciation for public health topics of environmental health and community engagement.

#### Community Windshield Assessment

For this assignment, students are once again asked to explore their community. This time they use observation and assessment to complete a worksheet about the health and safety of their

<sup>3</sup>https://manoa.hawaii.edu/about/

Frontiers in Public Health | www.frontiersin.org

community in regard to food availability, opportunities for physical activity, sidewalk existence and conditions, health care facilities, and overall built environment. The objective of this assignment is for students to see their community through the lenses of public health and built environment by identifying elements that facilitate and hinder health within their community.

#### Challenge/Success Presentations

Public Health Challenge and Success Presentations are assigned in lieu of a final exam. For this assignment, students work in assigned pairs to develop and deliver an 8- to 10-min presentation regarding a public health challenge and a public health success in Hawai'i. The presentations must include the following elements for the challenge and the success: (1) brief background; (2) description; (3) key stakeholders and interests; and (4) public health significance. For the challenge, students also present potential solutions, and for the success they also speculate how the public health community beyond Hawai'i can learn from it. A brief 2–3 min question and answer session follows each pair presentation. This assignment requires students to identify strengths and challenges of local public health and public health issues, and also allows for assessment of student oral presentation skills early in their public health degree program. Past selected challenges in local public health have included issues such as homelessness and invasive miconia plants, which prevents the growth of native plants and increases the risk of erosion with its shallow root systems. Past successes have included implementation of smoking bans on local public beaches, and increased nutritional access through acceptance of EBT cards at local farmers markets.

### Weekly Blog Reflections

Students are required to write weekly blogs/reflections and post them on the course website. Each blog is a critical synthesis of that week's topics, discussions, and activities written as student responses to reflection questions (see **Table 1**). Additionally, students are encouraged to respond to other student's blog entries to promote group discussion and learning. For the course instructor, weekly blogs also provide an opportunity to assess student mastery of content and skill development (17–19).

### OUTCOMES AND ASSESSMENT RESULTS

Rubrics are used to assess mastery of assignments and learning objectives, and these data suggest that >88% of students leave the course with solid mastery (either excellent or good), of the learning objectives (see **Table 2**).

Data from course evaluations and exit questionnaires additionally suggest that the students appreciate the approach of the course and the challenge to apply concepts from introductory public health to the local level through this subsequent course. In global appraisal, 97–100% of students rate the course overall as "very good" or "good" on a 5-point likert scale in end-of-semester evaluations (Spring 2016, Fall 2017).

Student feedback from end-of-semester evaluations and periodic mid-semester assessments, are used to refine course TABLE 1 | Sample blog prompts.


materials and improve articulation of the course with the broader undergraduate public health curriculum. Informal assessment is also conducted during grading and review of weekly blog reflections, where students express concrete and actionable responses to prompts and demonstrate competence in the course material. Anecdotal evidence from instructors in subsequent courses suggest students are mastering course content and are able to apply developed skills in later coursework.

### DISCUSSION

Since initial development, implementation has been challenged by the need to scale the course to accommodate a larger class size and concerns about overburdening community guest speakers. Early course sizes (roughly 12–20 students per semester) accommodated seminar-style discussions and activities. However, as the BAPH program rapidly expanded, there was a need to increase the capacity of the course to accommodate 30–40 students per semester. To address this need, activities were scaled back slightly in scope, and some activities initially completed in pairs are now completed by groups of 3–5 students. Application of technological TABLE 2 | Mastery of learning objectives as assessed by the instructor through completion of assignments—most current semester (Spring 2018).


support further assisted in expansion. Course management software helped to track class attendance, and was later used to administer short quizzes on assigned readings and videos to ensure assignments were read and videos viewed prior to class. Google Slides was later utilized to minimize transition time between challenge/success presentations, as students were required to share their completed slides with the instructor prior to class and upload them to the shared drive.

The PH 202 class has been consistently taught two to three times per year for the last 4 years. Though the topics and curriculum have remained consistent, the regularity of the course has led to a general rotation of local guest speakers. For example, a guest lecture focused on local legislative advocacy has featured, in rotation, representatives from the Hawai'i Public Health Institute, the Hawai'i-Pacific affiliate of the American Cancer Society Cancer Action Network, and a local non-profit organization, Healthy Mothers, Healthy Babies. This practice ensures local community agencies (particularly grassroots organizations with limited staff) are not overtaxed in their donation of time and also keeps the course fresh each semester as new perspectives are added.

Through implementation of this course, students demonstrate an improved awareness of local health challenges and actively engage in local community practices. The course helps students to connect newly acquired public health content and skills to issues many have seen first-hand in their home communities. Developing connections with local public health organizations and practitioners also help students to identify potential sites for required capstone experiences and future employment. Course implementation further fosters collaboration with local public health organizations and practitioners, improving community awareness of our program and students.

Public health undergraduate programs may benefit from development of coursework reflecting local and indigenous public health topics, or from the integration and application of community engagement into existing coursework. To begin development of such a course, institutions may consider outreach to local public health organizations (e.g., non-profit organizations), indigenous health-serving institutions, and the local department of health. Current public health faculty with local community connections (e.g., grant-related collaborations), may be helpful in facilitating development of these early relationships. Discussion with local organizations may support in identification of public health challenges most relevant to the local community, which may then be incorporated into the newly developed course.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the University of Hawai'i (UH) Human Studies Program as exempt from federal regulations pertaining to the protection of human research participants. Authority for the exemption applicable is documented in the Code of Federal Regulations at 45 CFR 46.101(b) 4. The protocol was approved by the Office of Research Compliance, University of Hawai'i system (Protocol Number 2018-00751).

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

DN-H and LK contributed conception and design of the course and all associated assignments and activities. DN-H wrote the first draft of the manuscript. All authors wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

#### ACKNOWLEDGMENTS

We acknowledge that without the commitment of the current B.A. Public Health students at the University of Hawai'i at Manoa, as well as the support of past undergraduate chair, ¯ Dr. Robert Cooney, and excellent student service support staff, Ms. Michelle Tagorda, Ms. Kimberly Inouye, and Ms. Kirsten Sensano, this program would not be possible. We gratefully acknowledge their contributions.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Nelson-Hurwitz, Kehl and Braun. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Service-Learning as a Practical Introduction to Undergraduate Public Health: Benefits for Student Outcomes and Accreditation

Meghan R. Mason<sup>1</sup> \* and Elizabeth Dunens <sup>2</sup>

*<sup>1</sup> Public Health Department, St. Catherine University, St. Paul, MN, United States, <sup>2</sup> The Center for Community Work and Learning, St. Catherine University, St. Paul, MN, United States*

Since the mid-1980s, service-learning has gained recognition as a pedagogical model in higher education with exciting potential for students' academic, civic, and professional development (1). Deemed a high-impact educational practice by the American Association of Colleges and Universities (AAC&U), extant research points to student learning, engagement, and retention benefits from community-based experiences integrated into curriculum (2, 3). Numerous studies have examined best practices for service-learning from varying stakeholder perspectives (faculty, student, and community partner) and disciplines, however, due to the recent development of public health as a major offering in U.S. undergraduate education, the value of service-learning within the discipline should be further explored. While recommendations for service-learning in undergraduate public health programs have been provided, no evaluation of the impact on student learning outcomes has been conducted (4). This study presents one university's model of service-learning in introductory public health courses, and results from the analysis of two datasets representing students' experience with service-learning in undergraduate public health curriculum. Findings provide empirical support of the effectiveness of this pedagogy for advancing student learning and the achievement of foundational accreditation domains outlined by the Council on Education for Public Health (CEPH).

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Diane D. Stadler, Oregon Health & Science University, United States Krista Mincey, Xavier University of Louisiana, United States*

> \*Correspondence: *Meghan R. Mason*

*mrmason@stkate.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *15 January 2019* Accepted: *05 March 2019* Published: *02 April 2019*

#### Citation:

*Mason MR and Dunens E (2019) Service-Learning as a Practical Introduction to Undergraduate Public Health: Benefits for Student Outcomes and Accreditation. Front. Public Health 7:63. doi: 10.3389/fpubh.2019.00063* Keywords: undergraduate, public health, service-learning, student learning outcomes, accreditation

## INTRODUCTION

The start of undergraduate public health education in the United States can be traced back to 2003 when the American Schools of Public Health (now American Schools and Programs of Public Health) launched a taskforce for undergraduate education (5). This taskforce was a response to an Institute of Medicine report that recommended all undergraduates, regardless of their discipline, have access to public health education (6). Interest in public health grew, and the development of majors followed. In 2002, there were 1,438 degree conferrals at the undergraduate level for Classification of Instruction (CIP) codes in the area of public health, increasing 5-fold to 6,464 by 2012 (7) and to 12,895 conferrals in 2016 (8). While monitoring and accreditation of graduate programs has been in place since 1946 (6), it was not until 2003 (9) that the Council for Education in Public Health (CEPH) first accredited undergraduate programs under already-accredited graduate schools of public health. Only recently, in 2013, has CEPH opened accreditation to stand-alone undergraduate programs (9). As demand for undergraduate majors in public health has increased, so too has interest from colleges and universities in seeking accreditation.

St. Catherine University's undergraduate public health major was founded in 2009 and is currently seeking accreditation under the CEPH 2016 criteria (10). St. Catherine's is a private, Catholic, liberal arts university located in St. Paul, Minnesota with 4,724 students enrolled across associate, undergraduate, and graduate degree programs. The undergraduate degree in public health is the first of its kind in Minnesota and includes four concentrations: Health Sciences, Public Policy, Health Education, and Community Health Worker. In <10 years, this degree has become the second most popular major for St. Catherine undergraduates. It is also one of the most diverse majors when it comes to student identities represented. Given the social justice orientation of the University and the community-based nature of the discipline, the department chose to integrate servicelearning into the undergraduate public health curriculum in fall of 2015. Inclusion of community engagement in the first course that majors encounter (Foundations in Public Health) also addresses two of the CEPH 2016 accreditation criteria: (1) Students should have learning experiences that address the "socioeconomic, behavioral, biological, environmental, and other factors that impact human health and contribute to health disparities" (Foundational Domains); and (2) Students are required to have experiential learning activities that support their didactic education (Cumulative and Experiential Activities). In this paper we analyze student data collected since the inception of the service-learning course to examine the effectiveness of using this pedagogical model for introductory, undergraduate public health coursework.

### SERVICE-LEARNING

Service-learning is an experiential learning model involving the incorporation of community engagement into course curriculum to enhance student learning outcomes and advance community efforts. Service-learning is not simply volunteer activity tied to a course, but an intentional pedagogy that "ensures that both the service enhances the learning and the learning enhances the service" [(11), p. 12]. In order to achieve this, courses should be grounded in reciprocity, reflection, and social change. At St. Catherine University, faculty work with staff from the university's Center for Community Work and Learning (CWL) to identify student engagement opportunities at community organizations with which CWL has an ongoing relationship. Faculty, community partners and CWL staff co-design servicelearning on a course-by-course basis to meet learning objectives and the expressed needs of the community partner. This focus on relationship is informed by community engagement theory and praxis, and an aspiration to achieve "transformational" over "transactional" collaborations (12). The scaffolding of in-class and assignment-based reflection activities before, during and after service-learning is essential to students "mak[ing] meaning of their experiences in and with communities and enhancing the quality of thinking, of learning, of service, and of partnerships" (13). This intentional bridging of course and community learning also prevents the phenomenon of students perceiving their engagement as an unnecessary "add on" to the course (14, 15).

Regarding social change, in our service-learning experience we see this happen in two ways. First, our students are contributing to and allying with local, justice-oriented community efforts. Secondly, through engagement in critical service-learning, our students advance toward becoming public health practitioners who understand "the systemic and institutionalized nature of oppression [and]...personal and institutional contributions to social problems and measures that may lead to social change" [(16), p. 54]. Developing students' social justice literacy aligns with institutional values and prepares them to be effective public health practitioners. As undergraduate public health programs continue to emerge and evolve, we present one model that has worked in our Foundations of Public Health class to meet student, community, and accreditation outcomes.

## FOUNDATIONS IN PUBLIC HEALTH

St. Catherine's Foundations of Public Health course is intended to introduce students to the population perspective and ecological nature of public health. Additionally, students examine how health is distributed unequally across race, gender, class, and environment, with a focus on health disparities in our own communities. Broadly, the learning objectives for the Foundations of Public Health course fall under three primary themes: (1) foundational concepts of health, public health, and health systems; (2) social justice implications of public health; (3) beginning level professional attitudes, behavior, and communication. These themes provide students with the opportunity to explore public health as a major while reflecting on and integrating their learning with the broader St. Catherine University liberal arts and Catholic social teaching traditions.

To tie the three primary learning themes together, the Foundations of Public Health course has a cumulative Community Health Assessment project that includes the following components: (1) community definition and description; (2) quantitative data and statistics; (3) walking survey; (4) key informant interviews; (5) reflection on servicelearning; (6) opportunities for improving the health of the community (**Appendix A**). This project is grounded in students' service-learning experience. For service-learning, students engage in small teams at a non-profit organization on three occasions for a total of 6 h of community-based work. Servicelearning activities range from working on an urban farm, to preparing and eating meals with seniors, to assisting with an allgirls open gym program for Muslim-identified youth. While the total number service-learning hours is on the lower end of ranges recommended (17), it reflects the introductory level of the course and an amount that is realistic for the student population (many of whom work 10–20 h per week to finance their education). The faculty rely on CWL to identify community partners that benefit from engagement of volunteers in short-term experiences. For those organizations interested in a St. Catherine student presence for more than 3 weeks, CWL places students from other service-learning courses at the same organization on alternating weeks, resulting in consistent St. Catherine's partnership with the organization across the semester.

During the final week of class, students present their community health assessment as a written paper and oral presentation to their classmates. Before service-learning was introduced to the course in fall 2015, students selected a community within the Twin Cities arbitrarily, often resulting in superficial understanding of the community and complications for even introductory-level assessment. Thus, the Public Health program contacted CWL to determine how service-learning might facilitate deeper community engagement and student learning in this course. data suggests the pedagogy is meeting this need. Furthermore, encountering service-learning at the start of the major ensures that all students have a foundational understanding of best practices for community engagement in public health settings, and are prepared for more advanced community-based activity scaffolded throughout the major (i.e., project-based service-learning and a 150 h practicum at a nonprofit, government, or public health organization).

### METHODOLOGY

In order to assess the effectiveness of service-learning as a tool to advance student outcomes related to 2016 CEPH criteria, we submitted an IRB application to gain access to two extant sources of data from the course: student evaluations of service-learning in Foundations for Public Health, and students' written reflections from the course.

At the end of each semester, CWL collects evaluation data from service-learning students via a Qualtrics survey with items related to students' experience and outcomes. Between fall 2015 and spring 2018, there have been 10 sections of the Foundations of Public Health course taught by six different faculty. In these 10 sections, there were a total of 206 students (average of 21 per class) of which, 84 students (41%) responded to the CWL evaluation survey. After removing student identifiers from the data, we examined student responses to four survey items contributing to assessment of the extent to which service-learning advances achievement of the two CEPH criteria selected.

In Foundations of Public Health, students complete a reflection assignment as part of their final Community Health Assessment project which includes addressing how service learning contributed to their educational experience in the course. For this study, the researchers accessed a total of 154 reflections submitted from nine sections of the course between fall 2015 and spring 2018. Reflections were coded for text supporting student achievement of CEPH criteria through service-learning. The researchers also noted exemplars to demonstrate the optimal student outcomes when using this model (**Tables 1**, **2**).

### FINDINGS

In reviewing CWL evaluation data, students from Foundations in Public Health valued their service-learning experiences. Eightysix percent of students reported that they would recommend service-learning to other students, 9% reported they might recommend, and only 5% reported they would not recommend service-learning. Of the 5%, most did not express concerns about the value of the pedagogy, but instead, their ability to meet the requirement due to time, work, or life constraints. As multiple factors can impact students' experience, we view this data point as a positive indicator for use of service-learning in public health introductory courses. In addition to students' appreciation for the experience, the following data supports service-learning as an effective tool for meeting CEPH criteria.

**Criterion 1: Students should have learning experiences that address the "socioeconomic, behavioral, biological, environmental, and other factors that impact human health and contribute to health disparities" (Foundational Domains)**

The foundational domains outlined in the CEPH criteria are heavily supported by the inclusion of service-learning in the Foundations of Public Health course. On the CWL evaluation survey, 88% of students agreed or strongly agreed that service-learning increased their understanding of diverse and global perspectives, which equips students with new lenses for understanding health and health behaviors. Additionally, 88% of students agreed or strongly agreed that service-learning increased their understanding of social justice, which aids students in identifying and addressing health disparities.

The contributions of socioeconomic, biological, behavioral, environmental, and other factors to the persistence and–ideally– elimination of health disparities were articulated by students in their end of semester reflections for the Foundations of Public Health course (**Table 1**). The most commonly reported factors mentioned in student reflections were focused on social support and a sense of community or belonging, food access, language and culture, housing/shelter, education, and age/ability (**Table 1**). The direct links to service-learning sites for many of these factors is often straightforward. For example, students were placed at organizations where there were food shelves and community gardens (food access), they tutored English-language learners (language and culture), and distributed hygiene packs and bedding at homeless shelters (housing/shelter). However, the students most frequently reflected on the role of their service-learning organization in contributing to a sense of community and belonging for the individuals it served. This factor was present across all semesters, suggesting that, regardless of community placement, students recognized the importance of social supports to individual and community health. This was most profoundly stated by one student who wrote that her organization

"was all about creating a non-judgmental environment for people in stressful situation to be apart [sic] of so they could experience what it feels like to have a positive community to belong to. . . If we are able to attempt to understand where people are coming from and how someones [sic] life experience can shape everything that they participate in life, it will become easier to provide help for people in disadvantaged situations."

In spite of some grammatical errors (not uncommon as many of our students are non-native English speakers), nearly all Factor Frequency (%) Exemplars Social support and sense of community or belonging 34 (22.1%) *"...was all about creating a non judgmental environment for people in stressful situation to be apart of so they could experience what it feels like to have a positive community to belong to…If we are able to attempt to understand where people are coming from and how someones life experience can shape everything that they participate in life, it will become easier to provide help for people in disadvantaged situations." "Some elderly do not have a family, so they need a company. They are very happy to see volunteers to come, so they can have a good time to share about life experiences and stories. Because of the needs of care and support, I make sure that they are happy and comfort when I am with them."* Food access 30 (19.5%) *"...lessens the problem of food inequity. When [sic] great thing that [the organization] does is that the vegetables and fruits are not weighted like other foods. Each family can get as many fruits and vegetables as they want…" "This farm gives its people...a chance at healthy food. In other neighborhoods, people don't get much fruits and vegetables...This place has shown that with people working together, they can grow more than fresh produce, but they can grow a community."* Language and culture 26 (16.9%) *"It was designed in a way that approached language through lessons that delivered specific language education encapsulated into information in the following areas: public health, safety, personal health and nutrition...This type of education and community commitment can breakdown barriers in access to health care, which is a key social determinant in health outcomes." "Language barrier puts limitations on an individual not living in their native land. It can cause misunderstanding can result in them being taken advantage of."* Housing/shelter 22 (14.3%) *"...those without homes don't have a lot of resources, and the resources they do have are lacking resources themselves...they carry a heavy stigma…" "Jane Doe had not eaten all day, and she passed out in line while waiting to get in...Sitting down with talking with her, I found out that she was living in a hotel; Jane Doe was told she couldn't stay at the hotel anymore due to lack of payment…"* Education 20 (13.0%) *"I feel like the areas of social justice that need to be worked on in academic situations are above my control. Things like admission rates for minorities and affordable education for all students are all things I cannot change by myself." "It seemed to me that we were able to help the kids who really needed it in order to close the education gap."* Age and ability 17 (11.0%) *"The class is designed for all levels and focuses on engaging you as a whole and interacting with one another. I think that's a huge part of healing and coping with a chronic and debilitating illness." "It gave me new perspective on the population with developmental and intellectual disability and understanding of how this social group are excluded from the large population."* Incarceration 14 (9.1%) *"Where many women are in prison are there due to some kind/type of social injustice. Most women that are in prison come from median to low income families/communities where violence is higher and the chances to enter a circle of violence is even higher." "After finding out that the statistic of women in prison is due to domestic violence, and misdemeanor crimes such as theft, and drug possession, I came to realize that many of the women were there because of self-defense, or were trying to make a living within the circumstances they were in."* Gender 14 (9.1%) *"This program is for young Muslim and Somali girls. It empowers and unites them through sportsmanship...I felt I could relate to it...I was once a young girl with plenty of insecurities...Playing sports was one of the few things that established confidence in me as a child." "These women's stories showed me that sometimes you do not realize you are in a violent relationship until after you leave and look back. They hurt themselves without ever realizing that they were hurting."* Employment 9 (5.8%) *"The business also helps them gain some work experience which ultimately helps them in their future…" "...in service-learning by promoting the welfare of individuals in poverty (all pertaining to immigrant/refugee populations)...to increase the job opportunities available to these individuals, as well as general necessary knowledge...These services are necessary for the students to gain independence within the community and provide opportunities for their families."* Violence/trauma 9 (5.8%) *"...opened my eyes into the world of sexual violence and the issues that arise from this injustice." "I learned so much about what sexual violence was and how many people it really affected. I also learned that it wasn't just physical, it was also mental and emotional…"* Access to medical care 8 (5.2%) *"...Americans must be able to access healthcare as a right. It is almost impossible to get the healthcare you need without a state or any type of picture ID." "It seemed like he was a young kid stuck in a grown man's body. It made me realize he might not have been getting the health resources he needed."* Race 8 (5.2%) *"To think that some people are at a disadvantage because of their race, the color of their skin hurts. I realized how we all say we have come a long way and how racism isn't prevalent yet some people are experiencing racism and are directly affected by that fact." "They unite communities by community services and fight racism through training and institutional change."* Environment 6 (3.9%) *"Having green space and safe environment supports people getting outside and becoming more active, getting to know other people and building a community that keeps an eye out and improves the overall safety of the community."* Income 2 (1.3%) *"The programs aims to reduce the causes of poverty, rather than just financial assistance."* Genetics 1 (0.6%) *"Genetic risk factors and deep brain stimulation are of the most interest to me because of how many opportunities for understanding the disease better they both offer…"* None 10 (6.5%) N/A

TABLE 1 | Frequency and exemplar quotes of socioeconomic, behavioral, biological, environmental, and other factors that impact human health and contribute to health disparities as described in student reflections from *Foundations of Public Health* (Fall 2015–Spring 2018).

TABLE 2 | Frequency and exemplar quotes of themes and sub-themes that demonstrate the support of experiential learning to didactic public health education in *Foundations of Public Health* (Fall 2015–Spring 2018).


reflection papers mentioned at least one factor related to human health and health disparities (93.6%, n = 145).

**Criterion 2: Students should be involved in experiential learning activity to support didactic education (Cumulative and Experiential Activities)**

Another undergraduate CEPH criterion states that students should be involved in experiential learning activities that support didactic education. According to evaluation data from students who completed service-learning as part of Foundations in Public Health between fall 2015 and spring 2018, students found service-learning to be a valuable part of their course learning. Of the 198 students who completed service-learning in the Foundations of Public Health course between fall 2015 and spring 2018, 84 responded to the evaluation survey from the CWL, for a response rate of 42%. Eighty-two percent of students agreed/strongly agreed that service-learning enhanced what they learned in the course, and 86% of students agreed/strongly agreed that course activities and assignments connected their servicelearning to their classroom learning.

While there are several specific learning objectives in Foundations of Public Health, the learning objectives can be grouped into three primary themes: (1) foundational concepts of health, public health, and health systems; (2) social justice implications of public health; (3) beginning level professional attitudes, behavior, and communication. Through student reflections, it was clear that these didactic elements were supported through service learning in the course (**Table 2**). Only ten (6.5%) students did not mention at least one of these learning objectives in their reflections. Within the first theme, examples included in the reflections focused on describing the social determinants of health, health disparities, definitions and aspects of health, health systems and access to healthcare, the social-ecological framework, and community-based interventions. Succinctly put, one student described social determinants in the homeless community as, "The homeless community needs support, education, healthcare and jobs."

The most frequent theme across the student reflections focused on social justice in the work of the community organization. Elements of social justice were identified in 64 (41.6%) of student reflections. One student wrote in her reflection, "Service plays a huge role in social justice; you are taking action and as a result, moving toward a solution/something better."

Notably, students also described specific skills and interests at their sites that would serve them well in future public health work including communication skills, advocacy, and problem-solving, and future public health roles. Many students expressed a desire to share their new knowledge with others. "This experience gave me a stronger sense of connection to...the disabled community It strengthened my desire to support and advocate for these communities."

One aspect of St. Catherine University's Catholic mission is the importance of reflection, and most importantly, selfreflection. Although not evaluated as a learning-outcome, many students explored relationships between their service-learning site and their own lived experiences in their reflection papers without prompting. In our undergraduate public health program, two-thirds of our traditional students and one-third of students in our adult learner program identify as students of color. More than one in four undergraduate public health students are multilingual. Throughout their reflections, students shared how they had either benefitted from similar service-learning organizations when they were younger, and what barriers they and/or their parents faced as new immigrants to the United States:

"I learned English when I was 10 years old and have experienced the struggle of learning a language and how it feels to have a language barrier. . . Language barrier puts limitations on an individual not living in their native land. It can cause misunderstanding can result in them being taken advantage of."

Others mentioned having received food from food shelves because their family did not have enough food, and still others shared their experiences with domestic violence. Many described their appreciation of service-learning as a way to give back to the community.

## DISCUSSION

Data supports the continued use of service-learning in St. Catherine University's undergraduate public health program to meet CEPH student outcomes related to determinants of health and experiential learning. The foundational domains of CEPH accreditation criteria state that students should have learning experiences that address the "socioeconomic, behavioral, biological, environmental, and other factors that impact human health and contribute to health disparities." Certainly the definitions and examples of these health factors need to be introduced in a classroom setting, and servicelearning does not replace the required knowledge acquisition. Rather, service-learning does reinforce concepts discussed in the classroom by provided students with examples of determinants of health as they appear in their community context. These connections were well-articulated by students in their servicelearning reflections. We have also found it important to provide students with service-learning in a foundational course so that they are able to better conceptualize the breadth of what constitutes working in public health. This helps to meet the CEPH accreditation criterion related to cumulative and experiential activities that support didactic education. As described in their reflections, students reported demonstrating beginning professional attitudes, behavior, and communication skills in public health. The St. Catherine University Public Health program's application was accepted for accreditation, and we are actively completing the self-study process. Feedback from our CEPH consultation visit was overwhelmingly positive with respect to the service-learning elements of our undergraduate program.

As we recommend service-learning as a pedagogy for undergraduate public health, we recognize it may not be feasible or beneficial for all programs; as Enos and Trope (18) observe, while "it is possible to incorporate [service-learning] in any discipline," service-learning is "not destined to be used in every course" (p. 159). Furthermore, we do not advocate that our exact model be replicated. Rather, we encourage departments to consider how this pedagogy might be applied to their program in a way that that reflects the unique needs of their student, faculty, and community stakeholders. Understanding best practices of community engagement along with the history, power and privilege at play in university-community partnerships is key to successful service-learning design and implementation. As undergraduate public health programs contemplate utilizing service-learning, the following lessons from our own experience may also be helpful.

#### Timeline

Service-learning requires intentionality in design and relationship development, which take time. We recommend starting to plan at least 1 year in advance.

### Capacity

If your institution does not have a community engagement office able to cultivate community partnerships and advise on service-learning, consider whether faculty workload allows for the additional time required for relationship development with community partner organizations and researching servicelearning pedagogy. Because of the commitment to partnerships and pedagogy, as well as to ensure continuity of messaging around the importance of service-learning to our public health program, we have found that primary instructional faculty are best-suited to incorporate service-learning into their courses.

### Design and Implementation

Prior to considering your course or program design, we recommend reaching out to colleagues on campus with servicelearning experience; consulting literature such as Service-learning Essentials (19) or The Cambridge Handbook of Service Learning and Community Engagement (13); and connecting with your regional Campus Compact office. (20) draws a distinction between service-learning that is "placed in the syllabus as something that is supplemental and periodic," and servicelearning that acts as the "experiential base of the course with a full array of activities and exercises to support it and to amplify course material" (p. 33). To achieve this latter model in a way that simultaneously seeks reciprocity for community partners, consult with extant theory and practice-based resources available; sharing on lessons learned can be particularly valuable.

As faculty approach implementation, it is important to note that service-learning may require new or unfamiliar approaches to instruction. Critical reflection should be integrated throughout the course and can occur in different formats, such as speaking, writing and classroom activities (19). We cannot stress enough the importance of thoughtful reflection design, and there is empirical evidence that the "degree of written and oral reflection influence[s] the effectiveness of service learning" [(21), p. 224]. Although journaling and written reflection are popular methods, "other classroom teaching and learning activities can help students make connections as well" [(22), p. 51].

The pedagogy is also not conducive to exclusively lecture-style courses, and "challenges faculty to reconsider their constructions of control and authority in the classroom and beyond" [(18), p. 159]. Bringle et al. (14) describe this as a "paradigm shift in higher education because it heightens the role that students can assume as constructors of knowledge" (p. 7). Finally, servicelearning may require professional development coaching from faculty as students work on communication and relationshipdevelopment skills in environments outside of the classroom. As one undergraduate public health program reported from their model of community-engaged learning in a capstone course, "faculty serve not only as instructors and mentors but also as conveners, resources, coaches, mediators, managers, assessors, and motivators" [(23), p. 97].

### Evaluation

As with most curriculum, refining service-learning in a course is an ongoing process. Our evaluation data from students and community partners informs course adjustments and is one more avenue for understanding and maintaining relationships. While we were able to retroactively draw from our evaluation data for identifying the meeting of CEPH criteria, programs just beginning to consider service-learning have the opportunity to design evaluation in a way that more explicitly assesses how service-learning advances student outcomes required by accreditation.

## CONCLUSION

In completing this study, our findings support servicelearning as an effective pedagogical model for introductory undergraduate public health coursework. In St. Catherine University's Foundations in Public Health course, servicelearning facilitates achievement of student learning outcomes related to understanding the "socioeconomic, behavioral, biological, environmental, and other factors that impact human health and contribute to health disparities" (CEPH Foundational Domains). Furthermore, by nature of its design, the pedagogy provides students with experiential learning that enhances their didactic learning (CEPH Cumulative and Experiential Activities requirement). Although the use of extant data sources limited the opportunity to compare achievement of the first criteria with a control group of students without service-learning, our data suggests that students with service-learning achieve learning outcomes desired while also meeting the second CEPH criteria around experiential learning (something that would not happen without service-learning). The original motivation for including service-learning in St. Catherine's Foundations of Public Health course was informed by student and community partner interest in combination with the University's value for social justice and community engagement. However, we have come to realize the pedagogy offers an additional benefit of meeting multiple CEPH criteria essential to the success and accreditation of our undergraduate public health major—something other undergraduate public health programs may want to consider as well.

## DATA AVAILABILITY

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.

## ETHICS STATEMENT

This study was granted exempt level approval from the St. Catherine University Institutional Review Board because it used previously collected, de-identified, evaluation data.

## AUTHOR CONTRIBUTIONS

MM and ED both contributed to the design and implementation of the research, to the analysis of the results, and to the writing of the manuscript.

### ACKNOWLEDGMENTS

Special thanks to our community partner organizations for your collaboration; D'Ann Urbaniak Lesch (St. Catherine University's Center for Community Work and Learning) and Dr. Angela

### REFERENCES


Ekwonye (St. Catherine University's Public Health Department) for encouraging this project; and the GHR Foundation's Academic Excellence Grant to the St. Catherine University Henrietta Schmoll School of Health for financially supporting publication of this study.

#### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh. 2019.00063/full#supplementary-material

Handbook of Service Learning and Community Engagement. Cambridge: Cambridge University Press (2017). p. 168–82.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Mason and Dunens. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Public Health Service-Learning Capstone: Ideal for Students, Academia and Community

#### Sara L. C. Mackenzie<sup>1</sup> \*, Deborah M. Hinchey <sup>2</sup> and Kathryn P. Cornforth<sup>3</sup>

*<sup>1</sup> Department of Health Services, University of Washington, Seattle, WA, United States, <sup>2</sup> Department of Biomedical and Health Sciences, University of Vermont, Burlington, VT, United States, <sup>3</sup> Carlson Leadership and Public Service Center, University of Washington, Seattle, WA, United States*

Undergraduate public health degree programs strive to educate students to improve the health of communities. As such we have an obligation to develop curricula that push students to think critically about their perspectives, examine assumptions, and provide supported opportunities to apply their academic learning. In addition, curricula ideally develop and nurture students' sense of civic responsibility. Community-engaged learning provides opportunities for students to interact with populations with a range of needs and different perspectives. Students need to be prepared to engage ethically and respectfully, while thinking critically about and reflecting on their roles in these communities. Service-learning is a high-impact practice that combines community service with structured academic learning, including preparation, and reflection. In line with public health community-based work, a key aspect of service-learning is the intentional development of community partnerships to ensure that students are filling the needs defined by the communities themselves. Accreditation criteria may guide what is taught but say little about how it should be taught. However, how we teach matters. Service-learning is a high impact practice that not only aligns well with the goals and objectives of an accreditation required culminating senior experience but shares many of the values of the discipline of public health. This paper analyzes the use of service-learning in the development and delivery of the University of Washington School of Public Health undergraduate Public Health-Global Health majors' culminating experience. We describe the course learning objectives, structure, and assessment tools. In addition, we present quantitative and qualitative results on the impact of the course. We argue that it is feasible, sustainable, and beneficial to students and communities when the high impact practice of service-learning is used in delivery of a culminating senior experience.

Keywords: high-impact practice, curriculum, service-learning, culminating experience, capstone, undergraduate public health education

### INTRODUCTION

Undergraduate public health degree programs strive to develop students who are competent to improve the health of communities. The Council on Education in Public Health (CEPH) developed requirements for accredited undergraduate baccalaureate programs (1). The domain areas and competencies guide what programs should teach students in order to ensure that they

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Nilesh Chandrakant Gawde, Tata Institute of Social Sciences, India Krista Mincey, Xavier University of Louisiana, United States*

> \*Correspondence: *Sara L. C. Mackenzie saramack@uw.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *20 November 2018* Accepted: *11 January 2019* Published: *29 January 2019*

#### Citation:

*Mackenzie SLC, Hinchey DM and Cornforth KP (2019) A Public Health Service-Learning Capstone: Ideal for Students, Academia and Community. Front. Public Health 7:10. doi: 10.3389/fpubh.2019.00010* are adequately prepared to enter the workforce or to begin graduate education upon degree completion (1). However, the domains and competencies say little about how to structure curricula or deliver course content to achieve the intended results. Intentional consideration of how teaching methodologies are integrated into courses and across curricula provides an opportunity to provide purposeful student progression to achieve intended results (2–4).

High impact practices are approaches to content delivery that have been shown to increase educational outcomes at the college level (3, 4). While utilization of high impact practices benefits all students, the positive results in regards to GPA, probability of retention and completion are especially significant for students from communities historically underserved in higher education (3–6). High impact practices can be implemented at the course level—such as collaborative projects, service-learning, and study abroad. Other practices are at the curricular design level—such as learning communities, common intellectual experiences, and writing intensive courses.

The Bachelor of Arts and Bachelor of Science Public Health-Global Health majors at the University of Washington incorporate multiple high impact practices across the curriculum. After a twice-a-year application process, we welcome a diverse group of students who progress, in general, as a cohort through a 2-year, sequential, required series of courses. The purposeful pathway allows intentional development of competencies and allows scaffolded and repeated practice opportunities for students to progress to higher levels of challenge and engagement. For example, competence in public health writing is developed across the curriculum with all required courses building on prior course assignments and skills (7). In this paper, we present arguments for why the high impact practice of service-learning aligns with the intended learning outcomes and goals of an accreditation required culminating capstone experience.

### BACKGROUND AND RATIONALE

A senior culminating experience or capstone course requires seniors to apply what they have learned throughout their educational program. A culminating experience is, in and of itself, a high impact practice because it requires students to integrate and apply their learning (3, 4). Given the complexity of public health issues and the fact that public health work occurs in community, the service-learning model, when implemented well, is a practice designed to foster transformative student and community development (8). Students engage with community partners and are challenged to critically self-reflect, synthesize, and apply their public health learning, analyze ethical and civic situations, and, in partnership, work toward community action.

### SERVICE-LEARNING FRAMEWORK AND PRINCIPLES

"**Service-learning** should be about social change, not just filling a gap in services. It should be about questioning the conditions in society that create the need for service in the first place and seeking to alter those conditions" (8).

Service-learning is a pedagogy that aligns with public health discipline values. In fact, one can almost substitute the words "public health" for "service-learning" in this definition. Traditional service-learning emphasizes reciprocal and intentional work focused on community-identified needs. Incorporating components that require students to analyze and address power dynamics and adopt a social justice lens moves the service-learning pedagogy to what is referred to as "critical service-learning." The reframing pushes students to do more—to analyze and challenge the power structures and systems of oppression that contribute to ongoing community needs. Students are encouraged to consider their role as agents of change (9–11).

Regardless of student background (prior lived and field experience), critical service-learning methods enhance cultural humility and students' abilities to work within and across communities (12). Service-learning balances academic and field experience creating a triad relationship between student, faculty and community agency. Teaching faculty frame the experience as a partnership and ask students to work with community partners to build on the strengths and knowledge present within the community. Intentional academic preparation requires students to draw and reflect on their prior academic learning and consider how it might apply in their field site. While the communities served and specific projects differ, academic exploration pushes students to critically reflect on and analyze the structures and power systems that create a need for service in the first place—an essential component of "critical" service-learning (9, 11, 13).

This pedagogy allows students to work in partnership with communities while simultaneously engaging in a structured academic and reflective classroom experience. Critical servicelearning pedagogy develops the personal, civic, and academic capacities of public health students while directly benefiting the communities with whom they work (12, 14, 15). The integrative and applied learning aspects of critical service-learning make this an ideal high impact practice for achieving the culminating experience competencies.

### CAPSTONE COMPETENCIES

The service-learning capstone is required for all majors and fulfills the CEPH culminating experience requirement necessary for program accreditation. In addition, the capstone learning objectives map to domain and competency areas identified as essential in the 2016 CEPH standards (1) for program accreditation. The learning objectives are high Bloom's level learning as students are expected to apply what they have progressively practiced through their required courses. Synthesis of learning is necessary to successfully work with agencies. By the end of the capstone, students are able to:


### LEARNING ENVIRONMENT

The University of Washington is on the quarter system and our capstone course spans two sequential quarters or 20 weeks. There are three offerings each year of the capstone (Fall/Winter, Winter/Spring and a Summer intensive). The pedagogical format is in person. The course was developed by a core faculty member in collaboration with the program director and is led now by a single instructor with in-classroom support of graduate teaching assistants. The course instructor works year round to develop and maintain relationships with community partners in coordination with the University of Washington Carlson Leadership & Public Service Center. The center provides support for community-based learning pedagogy and supports campuscommunity partnerships.

During the course, students meet weekly in class for a 110 min session with faculty. The course includes three key components:

1) **Academics:** The structured academic component ensures students are synthesizing and applying their academic learning. In addition, it creates a structured environment to support students in examining and reflecting on identity development and issues of power, privilege, and oppression. Assignments and in-class activities encourage student reflection on their implicit biases, backgrounds, and the lens through which they see the world. **Table 1** illustrates the assignments and how they are structured across the 20 week curriculum. Weekly in-class sessions and scaffolded assignments develop students understanding of context (agency), population, and assets in and problems experienced by the population. Students develop an evidence base for the analysis of the systemic causes of a public health issue facing the populations with whom they are working. Their work culminates with a concrete and actionable project.

Students produce an agency overview, including looking at the mission, vision, organizational chart, budget, and funding models for the agency. Students are regularly faced with situations on-site that require ethical decision-making. The pre-field work presents scenarios for students to work through. In both the service positions and the academic work, students are challenged to work independently and as part of a team to assess, analyze, and address pertinent TABLE 1 | Academic activities and goals assessed.


health issues facing the populations with whom they are working. Professional development is embedded throughout the duration of the capstone course, and includes guest lectures by career specialists, resume workshopping, and professional communication. Academic assignments align with and assess student progress in achieving the identified course competencies. Assignments are graded using rubrics and ensure that, regardless of the site placement, students make connections to public health issues and broader course competencies. Students must demonstrate a thorough understanding of the material in order to pass the course.

2) **Community Service**: In the first 2 weeks of capstone, community partners present an overview of their work, populations served, community need, and field opportunity. Students select a placement that aligns with their personal goals and interests. A sampling of community agencies and projects are presented in **Table 2**. Students spend ∼5 h per week over the course of 10 weeks at their field site for a minimum of 50 h of service. The first weeks of the capstone experience are focused on preparation and the last weeks reserved for culminating reflection and production of academic work related to the service experience. Students often continue working with their service site beyond the required number of weeks.

Students are placed in small teams within the organization so that each site has at minimum three students. Students are not required to do their service at the same time as their peers but the team approach creates a group experience. Work within the organization is designed and supervised by

#### TABLE 2 | A sampling of the range of service positions.


the community partner organization. Students are supported to focus on community-articulated needs in their service hours. The service experience is framed as an opportunity to network with professionals working in the field and students have experiences in class and field which prepare them to network successfully. Site supervisors provide feedback to course instructor on student performance in particular as relates to professionalism and ethical behavior.

3) **Reflection:** Reflection is a critical component of servicelearning and occurs throughout. Early reflection activities are designed to prepare students to enter the community. Our academic responsibility to our community is to ensure students enter communities aware of their own roles in power and privilege dynamics. During the field experience, reflection activities focus on supporting students in analyzing what they are observing and experiencing in the community. The closing reflective experiences as students to consider how the experiences has impacted them. The "what, so what, now what" experiential learning cycle is emphasized throughout (16, 17). Reflective activities and writings are not graded for content but are required components of the course.

#### RESULTS AND IMPACT

At this time, 899 students have completed the service-learning capstone. Students must complete all service hours, participate actively in-class (through attendance and group engagement), and demonstrate a thorough understanding of the material in order to pass the course. Students have provided upwards of 45,000 h of service to the community and produced multiple resources for partner organizations–from academic literature reviews of best practices to training videos for staff to community asset maps that are usable by clients of the organizations.

A 2018 student survey assessed student perception of the culminating experience. The online survey was administered to 150 students in class during the last week of the quarter. The response rate was 90 percent. **Table 3** shows the questions and response rates from the survey. The majority of students felt that the capstone experience required them to apply prior learning, analyze systemic causes contributing to public health problems and apply this to formulate an approach to problem solving. In addition, to be successful the majority of students felt they had to evaluate their personal attitudes and approaches to working in communities and they were challenged to be professional and ethical in their behavior. The majority of students also felt they developed skills in teamwork, oral and written communication, accessing and interpreting literature, and systems thinking.

A key function of service-learning is the development of civic responsibility and community engagement in students. **Table 4** shows results of a survey question which asks students to consider how the capstone impacted their sense of commitment and responsibility to communities. Twenty-five percent of students felt the capstone instilled this sense of commitment and responsibility, whereas sixty-six percent felt it strengthened an existing sense of commitment and responsibility. Only eight percent of students felt the experience did not change their sense of commitment and responsibility.

We have used informal, regular meetings to assess the impact on community. The majority of our agencies have continued to request public health students once they partner with us for capstone. Many of our partner agencies have been with us since 2013. These ongoing relationships and partner feedback on student employability are important ways for us to assess student preparation for the workforce and program structure. Sample quotes from partners include:


The theme of public health students being able to extend the reach and hours of community agencies is common. For example, in an ongoing partnership since 2013, students at one capstone site have assisted emergency responders in educating 12,000 limited English proficient residents on 9-1-1 and CPR through workshops and knocked on over 2,000 doors in door-to-door outreach. Another site requested student engagement during the summer in order to meet their agency needs. In response, we began offering a summer intensive capstone so the organization could provide summer hours for clients.

#### DISCUSSION

The service-learning model is a sustainable, high impact practice that aligns well with the instructional needs of a culminating senior capstone course and the values of the public health

#### TABLE 3 | Student perception of capstone learning and application.


TABLE 4 | Capstone impact on student commitment and responsibility to make a difference in communities.

#### Which of the following best describes your experience. participation in capstone…


discipline. The practical implications of our model is that it is flexible enough to serve thirty students at program inception 6 years ago and expandable to now meet the needs of over 500 students. Our results show that the practice is effective at engaging students in high level work. In addition, for the majority of students, the experience instilled or strengthened their sense of commitment and responsibility to the community.

While students appreciate experiential opportunities and service-learning is well-received, we have had to balance the time, flexibility, and effort components of academic and community work due to demands on and complexity of student lives. For example, for several years a service site involved students teaching in local high schools about public health. However, the rigidity of high school class times and shifting schedules during quarter changes created ongoing challenges for our students that could not be overcome and we had to let go of this service site. We have found being realistic and transparent about travel time, schedule flexibility, and student obligation at sites is critical for ongoing mutual benefits.

A challenge that sometimes arises is a student requesting to develop their own service site instead of working with an agency with whom we have a relationship. The motivation behind these requests are almost always student driven with different goals than intended for capstone. We have chosen to not allow this for several reasons. A critical component of capstone is the balancing of student needs with community needs. In setting up their own sites, students tend to focus on their own needs. Additionally, the professional and applied skills incorporated into the team-based course structure are essential to preparing students for post-graduation success. We do encourage these students to set up a credit-bearing internship with alternative sites as an internship is typically more focused on student driven objectives. If they choose this option, it is in addition to capstone.

Another powerful aspect of our model is that we utilize a team-based approach to the assignment of students to sites. Each site has at minimum three students working at it. The

initial goal of the structure was to support students building skills working on site projects together and conflict resolution. However, further benefits to the team/group model quickly became clear. The shared student experience emulated a realworld work experience, and students were able to create a community of practice. Students appreciated having peers with whom they could discuss experiences, examine their reactions, and support each other to push their comfort zones. Additionally, the team-based approach helps with site coordination as we need less sites to accommodate our large number of students.

The identification, development and maintenance of community partnerships is critical to the success of servicelearning. Because of this, we invest in designated faculty time for relationship building. We are fortunate that community agencies will often approach the UW Carlson Center or our faculty when they identify a need for students. As in many communities, we must be sensitive to not overburdening community partners as undergraduate and graduate students from many U.W. schools and colleges, in addition to other programs in the area, regularly perform community work through service-learning, internships, and pratica.

We tend to select service-learning agencies addressing a broad range of social determinants of health instead of governmental public health agencies. Additionally, it is imperative to provide students with a choice of sites that serve varied populations and offer diverse experiences. The maintenance of sites requires significant facetime between instructor and community partner (14). We have found this to be true and have regular meetings between faculty, agencies, and the Carlson Center staff to develop shared expectations around academic and field work. Agency partner engagement in planning supports the entire experience. When sites are aware of the academic components, not only can they help to guide students in topic areas but they benefit from the work. For example, student literature reviews can be guided and used by agencies. On many occasions, agencies will use student developed work in the professional development of agency staff.

Another challenge arose with the rapid growth of our majors and consideration of a culminating service experience. Our school also graduates approximately one hundred and fifty Masters in Public Health students a year. The majority seek local placements for their required 120-h practicum experience. Unlike undergraduates, MPH students are expected to contribute

### REFERENCES


substantial knowledge and skills while working with agencies and require significant on-site mentorship. When constructing experiences for undergraduates, we needed to ensure that we were not increasing the already great supervisory burden on partner agencies. Additionally, it was essential to work with partners clarify how their needs aligned with the skill sets of different learners to create best fit scenarios. We find that the needs of non-governmental social service organizations that address public health issues are often more in line with the skill sets of undergraduates and offer robust opportunities to engage students in big public health questions.

### CONCLUSION

Service-Learning, and in particular "critical" service-learning is a high impact practice that serves well as a method of delivery for the culminating senior experience in public health major degree programs. The approach requires students to synthesize and apply their learning in the community. In addition, students analyze and challenge the power structures and systems of oppression that contribute to ongoing community needs. Students consider their role as agents of change. The intentional engagement with community agencies creates a sustainable and action-oriented model for education.

### AUTHOR CONTRIBUTIONS

All authors contributed to the conception of the course. DH developed the course content. SM and DH co-wrote early drafts. All authors provided content expertise, contributed to manuscript revision, read, and approved the submitted version.

### FUNDING

The University of Washington School of Public Health will cover the open access publication fee.

### ACKNOWLEDGMENTS

The authors wish to acknowledge the insight and feedback from community partners and students over the past 6 years.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Mackenzie, Hinchey and Cornforth. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Cohort Model and High Impact Practices in Undergraduate Public Health Education

#### Karin Joann Opacich\*

*School of Public Health, University of Illinois at Chicago, Chicago, IL, United States*

Developing curriculum that is more than a collection of courses necessitates articulating philosophy and principles that undergird curricular decisions. While faculty are accustomed to expressing ideas within their realms of content expertise, building consensus around educational philosophy and pedagogy may be less common but equally important to assure coherent curriculum. Such discussions lead to intentional curriculum. When attuned to intent and combined with high impact practices, curriculum is likelier to result in student success and engagement. Since public health by nature entails community interaction, opportunities to think and work in a variety of communities reflects the work in the public health arena. Building a community of learners in the context of a highly diverse urban campus requires very deliberate curriculum planning and design. The likelihood that learning communities will emerge spontaneously is reduced when only a small proportion of students live on or near campus, and most spend considerable time commuting. Virtually all undergraduate public health students have responsibilities beyond academics, including employment, family caregiving, religious obligations, etc. Since most undergraduate students in this setting are first generation to higher education, learning communities and other high impact practices become even more important to provide meaningful baccalaureate education. Such communities evolve most efficiently when integrated into the curriculum design. By implementing a cohort model, not only can faculty participate and facilitate the evolution of a community of learners, they can employ other high impact practices designed to enhance and compound public health content and processes. Undergraduate public health students in this setting take all of their core courses (32 semester hours) together in a prescribed sequence. Faculty have clear understanding about what preceded a course and what follows. Every course entails both individual work and group collaboration. Students come to understand each other's strengths and needs, and with rare exception, they support each other on the journey and share some mutual successes. Both expected and unintended outcomes of this approach are conveyed in this article along with a few cautions for those considering these strategies for undergraduate public health education.

Keywords: learning communities, cohort model, undergradaute curriculum, public health education, high impact practices

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Darcell P. Scharff, Saint Louis University, United States Elizabeth Gouaux Levitzky, Louisiana State University, United States*

> \*Correspondence: *Karin Joann Opacich kopacich@uic.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *01 December 2018* Accepted: *09 May 2019* Published: *29 May 2019*

#### Citation:

*Opacich KJ (2019) A Cohort Model and High Impact Practices in Undergraduate Public Health Education. Front. Public Health 7:132. doi: 10.3389/fpubh.2019.00132*

**63**

#### INTRODUCTION

The public health enterprise can best be described as an arena to which many different professionals representing a wide array of disciplines and perspectives come to solve health challenges faced by communities and populations. Subsequently the public health enterprise leads to myriad options for educating those who aspire to resolving public health challenges. Even for programs accredited by the Council for Education in Public Health (CEPH), curriculum development requires many choices about what we teach and how we take into consideration industry standards and expectations, institutional missions, faculty expertise, student audiences, and curriculum aspirations. Ultimately, curriculum is an expression of what faculty chooses to profess to assure that learners achieve a set of clearly articulated curricular learner outcomes that exceed the mere acquisition of knowledge.

Developing curriculum that amounts to more than a loose collection of courses necessitates clear communication of philosophy and principles undergirding curricular decisions. While faculty are accustomed to expressing ideas within their realms of content expertise, building consensus around educational philosophy and pedagogy is less common but equally important to assure coherent curriculum. Such discussions lead to intentional curriculum. When attuned to educational intent and combined with high impact practices, curriculum is likelier to result in student engagement and success, especially for historically underserved students. Since public health by nature entails community interaction, most often with vulnerable populations, opportunities to prepare students to think critically, collaborate successfully, and solve vexing problems is central to public health education. The cohort model represents one expression of intentional learning communities, a high-impact practice known to be powerful and highly compatible with education and practice in public health.

This paper will address the cohort expression of learning communities that has been implemented in the baccalaureate program at UIC since its inception. It comports with the learning community as described by Gabelnick et al. (1) to be:

the purposeful restructuring of the curriculum by linking or clustering courses that enroll a common cohort of students. This represents an intentional structuring of the student's time, credit, and learning experiences to build community, and foster more explicit connections among students, faculty, and disciplines (p. 6/7).

#### BRIEF HISTORY AND EVOLUTION OF THOUGHT ABOUT LEARNING COMMUNITIES

According to Barbara Leigh Smith, who spearheaded a project to chronicle the national history of learning communities (2), interest in learning communities advanced in three waves beginning in 1920. Early experimental efforts exemplified by Meiklejohn's approach in the late 1920s at the University of Wisconsin, employed a pedagogy that integrated active learning with theory and practice in the interest of exploring democracy and issues challenging society (3). His experiment, although perceived as having tremendous impact on students, was abandoned by the university after 5 years. The second wave emerged in the 1960's and focused on alternatives to traditional segmented structures in the academy, and these attempts resulted in innovations but also met with resistance. Some of those innovations, (e.g., student-centered learning, writing across the curriculum, active learning, and interdisciplinary programs) managed to survive (2). In the mid-1980s the notion of learning communities was again resurrected, and research emerged attesting to the tremendous value and potential of an array of learning communities. Educational researchers began to document the power of learning communities to engage students and to engender profound learning, sometimes referred to as deep learning (4, 5) Wharburton (6) contrasts surface learning with deep learning as, "...paying attention to underlying meaning. It is associated with the use of analytic skills, cross-referencing, imaginative reconstruction and independent thinking." In contemporary literature, proponents of deep learning note qualitative differences in learning that share attributes with learning communities. Most studies are qualitative, although a few quantitative studies do numerically measure the power of learning communities. Some literature does specifically point to the compatibility of learning communities with baccalaureate public health education (7, 8).

### BACCALAUREATE PUBLIC HEALTH PROGRAMMING IN THE UIC CONTEXT<sup>1</sup>

"With the 9th highest overall score on U.S News and World Report's ethnic diversity index, UIC's student body is one of the most diverse in the nation"2Over 30,000 students are enrolled at UIC, most commuting from the surrounding metropolitan area. More than 19,000 of those enrolled are undergraduates with nearly 90 undergraduate majors from which to choose. Although the School of Public Health was established almost half a century ago, baccalaureate programming is a newer phenomenon introduced on this campus in 2012. At present, more than 65% of UIC undergraduate public health students identify with underrepresented racial and ethnic minorities<sup>3</sup> reflective of even greater diversity than the campus as a whole.

The UIC Bachelor of Arts in Public Health has embraced four theoretical planks in its approach to teaching and learning in public health. The first of these is a commitment to principles of liberal education. Liberal Education & America's Promise (LEAP), an initiative launched in 2005 by the American Association of Colleges and Universities (9), has permeated higher education and is deemed to be central to the development of "an educated citizenry," a major impetus for the undergraduate public health

<sup>1</sup>The University of Illinois at Chicago (UIC) is Chicago's only public research university, and it includes 30,000 students, 15 colleges, a hospital and a health sciences system. It is one of three campuses in the University of Illinois system.

<sup>2</sup>The diversity of the UIC student body has been recognized by many agencies and sources over the last decade. Current diversity information can be found at, https:// www.hr.uic.edu/diversity\_at\_uic/.

<sup>3</sup>Current institutional data can be found at, https://oir.uic.edu/data/student-data/

movement (10–13). The second plank, the notion of confluent education, first described in 1971 by Brown (14) and since elaborated by others (15, 16), was selected to give voice to student experience bridging a gap between cognitive and affective understandings, particularly since the student body is largely comprised of first generation college learners whose family stories are replete with health challenges and health disparities of concern to public health. The third plank of the educational philosophy is action learning about which much has been written (17–20). Students are taught to be agents in the production of their own learning to assure that learning continues well-beyond the classroom experience.

Last and at the crux of this paper, the fourth plank, community engaged participatory learning, shares many features with community-based participatory action research (21). Students quickly come to realize that they both contribute to and benefit from the understandings of their cohort members as well as from people in the communities who experience public health challenges. Public health by nature is collaborative, so experience in community and learning in community is highly compatible with public health practice. When students of public health function within a cohort, they experience many of the same phenomena that they will encounter working with and within a variety of communities from vulnerable populations to professional associations (22). Undergraduate public health students at UIC complete 60 semester hours of general education including three public health pre-requisites (nine semester hours fulfilling general education requirements) and a newly added two semester hour Foundations for Public Health course in their first 2 years of study after which they matriculate into the upper division major. Students can declare the pre-public health program as incoming first year students, internal transfer students, or external transfer students before matriculating to the full major in the junior year. The upper division major begins once a year in the fall although students may be admitted to the premajor phase throughout the academic year. Students in the major move through all 13 upper division public health core courses (35 semester hours) as a cohort<sup>4</sup> . They plan their remaining 25 semester hours of selectives<sup>5</sup> and electives in accordance with their individual interests and pathways. To date, a single cohort has been as large as 43 students, but sub-cohorts will be necessary as the program grows. This particular cohort model can be described as a long-term (2 year), cross-curricular, face-to-face, public health student community that is supplemented with a digital learning platform (Blackboard).

The overarching curricular themes, the curricular goals, and the curricular learner outcomes appear in **Table 1**<sup>6</sup> . The program was developed as the Undergraduate Learner Outcomes were emerging in the Framework for the Future initiative<sup>7</sup> . The baccalaureate program is situated in a School of Public Health and was judged by CEPH<sup>8</sup> to be in full compliance with standards set forth for undergraduate public health programs. Ongoing program evaluation is organized by a fluid Input-Process-Output model predicated on program evaluation concepts introduced in the literature by Stake (23–26). These concepts, adopted in turn by Opacich (27) for application to academic program evaluation ultimately informs our current model of program evaluation that allows for multi-dimensional assessment of all aspects of the program including but not limited to the effectiveness of the cohort model. See **Figure 1**<sup>9</sup> . Evaluation priorities change from year to year, and a sub-committee of the Undergraduate Oversight Committee determines the focus of assessment crafting new questions, selecting methods, and developing tools as needed.

### THE UIC EXPERIENCE: CHALLENGES AND OPPORTUNITIES ASSOCIATED WITH THE COHORT MODEL

#### Designing, Facilitating, and Nurturing Learning Communities

Designing, facilitating, and nurturing learning communities in any setting takes skill and effort. Building a community of learners in the context of a highly diverse urban campus requires deliberate curriculum planning and design. The likelihood that learning communities will emerge spontaneously is reduced when only a small proportion of students live on or near campus, and most spend considerable time commuting. Today's undergraduate public health students commonly have responsibilities beyond academics, including employment, family caregiving, religious obligations, etc. For first generation college students, learning communities and other high impact practices become even more important to provide meaningful baccalaureate education (28–31). The learning communities evolve most efficiently when integral to curriculum design.

Although proponents of learning communities (32–41) express some variation in characteristics, there is general agreement about the intent and the potential outcomes of true learning communities asserted by Lenning et al. (42) in Powerful Learning Communities:

In a powerful LC, there is optimally effective and ongoing interplay and collaboration among the community's members as they strive for specified common learning goals, and the result is deep learning

<sup>4</sup> Students can petition to replace PUBH393 Fieldwork in Public Health with another selective course, in which case they will complete 12 core courses for a total of 32 semester hours.

<sup>5</sup> Selectives refers to courses that are part of a specific menu of courses from which students chose.

<sup>6</sup>The curriculum is in full compliance with the CEPH undergraduate learner outcomes. Due to the integrated rather than discipline-specific nature of the curriculum, CEPH competencies for baccalaureate level education tend to map to multiple rather than singular courses.

<sup>7</sup>The Framing the Future initiative was launched in 2014 and spearheaded by ASPPH to re-examine public health education.

<sup>8</sup>UIC School of Public Health is accredited by the Council for Education in Public Health through 2022.

<sup>9</sup>The matrix model is predicated on the work of Stake, Stufflebeam, and Quinn-Patton. It entails goal directed program evaluation and accommodates both qualitative and quantitative indicators reflecting the intersection of planning, implementation, and outcomes relative to faculty, content, and students. Evaluation priorities change and the model is designed to capture data and information pertaining to the questions raised.

#### TABLE 1 | UIC baccalaureate in public health-overarching curricular themes, goals, and learning outcomes.

#### Overarching Curricular Themes

Health as a Moral Endeavor *exploring the moral importance of health and healthcare considering individual and societal commitments and obligations including the use of limited resources*

Health Equity *having equitable access and the means and resources to attain one's full life potential*

Life Course Perspective *the cumulative evolutionary, pre-generational, pre-natal, and life events and circumstances that influence health at any one point in time* One Health [Human-Animal-Environment] *the inextricable relationship among animal, human, and environmental health as determined by, e.g., evolutionary biology, human behavior, and environmental phenomena*

Cultural Relevance *the lens through which life events are experiences and interpreted and through which meaning is ascribed*

Local/Global Impact *appreciation for the global systems that influences the processes, dynamics, and activities of the world's populations; health as a multi-faceted state shaped within, e.g., biological, socio-cultural, geographic, economic, and political contexts*


that is maximally insightful and useful as it pertains to those goals. The members of the group express mutual trust and loyalty, share ideas, and support one another (p. 9).

Facilitating effective learning communities necessitates specific attention to faculty development as it aligns with chosen models for student learning and compatible instructional strategies.

#### Relevant Faculty Development

As is true of any best practice in education, faculty invested in fostering engagement in and beyond the classroom need to be developed and supported. A Master Teacher Mentoring (MTM) program was implemented at the inception of the UIC program, and it aims to acculturate potential faculty to the curriculum as a whole and a single course in particular. Mentees practice instructional strategies that are consistent with the curriculum philosophy under the auspices of faculty who have demonstrated teaching success. Mentees have included advanced public health practitioners, doctoral students, postdoctoral fellows, and academic professionals whose regular scope of work is not teaching. Participants are paired with a master teacher in a course that aligns with their expertise, and a learning contract is negotiated. Additional resources for faculty development are available through the campus.

The literature indicates that there is a modicum of faculty skepticism about the use of cohort models in education, largely because sub-groups can tend to ban together to target faculty or to make unreasonable demands. Conflict of any kind, whether in the classroom, on the campus, or in the

community requires insight and skilled management. Public health faculty might reframe these inevitable occurrences as opportunities to teach mediation and negotiation! Issues and ideas are exchanged in faculty meetings and the annual faculty retreat. Most assuredly, considerable attention must be paid to planning learning activities, classroom management, and classroom culture. It is also imperative that faculty collaborate across the curriculum to foster and maintain that culture. When expectations are clear about (e.g., class attendance, preparation for class, active participation in discussions, and contribution to group assignments), it is likelier that students will form the habits that bode well for their individual success as well as success perceived by both teachers and cohort peers. Within the cohort model, faculty can participate and facilitate the evolution of a community of learners using other high impact practices (43) to enhance and compound public health content and processes.

#### (re)Shaping Student Learning

Faculty are urged to re-examine their assumptions about college students and teaching/learning. Secondary education continues to be scrutinized and found to be inconsistent in preparing young students for college and beyond. Federal programs such as Race To the Top<sup>10</sup> were intended to incentivize improvements and innovations especially for underperforming school systems often challenged with inadequate resources. Adding to the complexity of the modern college environment, literature has been emerging about the impact of No Child Left Behind<sup>11</sup> on the learning habits and outcomes of a generation of students (44, 45) Alumni of NCLB are now on college campuses, and they are unlikely to mirror the academic experiences of the professoriate. While most students come to college eager to learn, some arrive with less than robust portfolios of academic skills necessary for college success.

In the Chicago metropolitan area, school funding, school violence, and school closures most certainly impact secondary education. The University of Illinois at Chicago (UIC), like many colleges and universities across the country, has implemented student success initiatives aimed largely at supporting first-year students and bolstering their eventual success in achieving a college degree. Intentionally designed to engage and enhance learning LCs can be effective success strategies. However, students accustomed to more traditional surface learning and knowledge transfer approaches may need to be acculturated to a different paradigm of engagement and accountability. The close learning community may very well be an important factor in student persistence and success (46, 47).

Of the high-impact educational practices described by Kuh (43), the UIC baccalaureate curriculum in public health routinely incorporates nine of these: common intellectual experiences; learning communities; writing-intensive courses; collaborative assignments and projects; undergraduate research; diversity/global learning; service learning/community-based learning, internships; and capstone courses and projects. While the literature shows that each alone can add value to the learning experience, the cohort model seems to magnify all the others. When high-impact educational practices are implemented, processed, and distilled in the cohort context, all members benefit by compounding what they might have learned and experienced individually and leading to richer understandings. At this time, midway through the seventh year of operation, 6 year graduation rates for public health hover around 95%, well above the campus average of 6 year graduation rates of 59.7% for first-time, fulltime attendees according to 2017 institutional data<sup>12</sup> The graduation rate for public health students is no doubt influenced by a 3.0 GPA graduation requirement in the major13, a higher standard than most programs on campus, but with rare exception, even students who have been on academic probation or have experienced other challenges have managed to graduate and have been celebrated by their peers. Approximately 20% of UIC baccalaureate public health graduates matriculate to graduate and professional programs immediately, and the remaining 80% of graduates enter the workforce in a variety of public health related settings and jobs. Yet another wave of working graduates seem to be entering graduate school within a few years of degree completion, and more precise data will be forthcoming as alumni surveys are disseminated. Establishing and nurturing learning communities affords both challenges and opportunities. Both expected and unintended outcomes in this highly diverse urban public university are discussed below.

### DISCUSSION

### Advantages of the Cohort Model

Since the program has required students to progress through the core courses together in the same sequence, faculty are well-apprised of where and how their respective courses are situated in the curriculum. They know what learner outcomes students will have been expected to achieve. Unlike courses that may host students at different entry points and widely different levels of understanding and skill, faculty know what students will have achieved so that they can set expectations scaffolding new learning and opportunities accordingly. Faculty carefully communicate and coordinate across the curriculum during the academic year, at the annual faculty retreat, and most certainly when changing any major assignment or component of a course. This practice minimizes unplanned redundancy and renders a clearer picture of how the curriculum unfolds. Attention to priorities, sequencing, and purposeful repetition allows for more focused program evaluation and illuminates needed revisions. When student performance does not meet expectations or when students express a need for further instruction, it is somewhat easier to diagnose where we have fallen short or where we might accommodate their requests in an informed, systematic fashion.

Virtually all classes entail both group assignments and individual assignments, and both the proportion and assessment of group work is informed by the education literature. Students in a closed learning community have the opportunity to collaborate with different sets of peers exploring their particular talents and working toward common goals. The cohort provides a level of familiarity yet allows for both faculty and peer feedback in the execution of assignments. Unlike approaching

<sup>10</sup>Race to the Top was a \$4 billion fund launched by the federal government in 2009 to incentivize states and school districts to innovate and improve education and student outcomes.

<sup>11</sup>No Child Left Behind was a 2002 reauthorization of the Elementary and Secondary Education Act enacted by Congress to "close student achievement gaps by providing all children with a fair, equal, and significant opportunity to obtain a high-quality education."

<sup>12</sup>Current institutional data, including graduation rates, can be found at, https:// oir.uic.edu/data/student-data/.

<sup>13</sup>The major GPA for graduation will be adjusted to 2.5 in AY2019/20, which may impact future graduation rates.

course assignments in groups of strangers, students are more accountable to each other. Since public health students take 12 of 13 of their core courses (32 semester hours) together in a prescribed sequence, both students and faculty have clear understanding about what preceded and what follows. In both individual work and collaborative efforts, students come to understand each other's strengths and needs, and with rare exception, they support each other on the journey sharing mutual successes and individual achievements. They develop work skills, patterns, and habits that can be applied in the public health arena with the understanding that they can be productive even if they might not like everyone in the group.

Direct feedback during the junior year professional topics seminars, exit surveys, and alumni focus groups have yielded positive feedback about the cohort model. There are always a few students who dislike some of the necessary features, e.g., the schedule, because it does not always comport with individual preferences, but they do not seem to link the scheduling restrictions with the cohort model. It appears that students form a camaraderie that often extends beyond the classroom and curriculum that bolsters them through the life events and stressors that they experience. For the most part, students encourage and support each other, but occasionally, faculty mediate conflicts or intervene when a student is underperforming affecting the cohort. If students have issues with faculty that cannot be satisfactorily resolved, the matter is referred to the Office of Student Affairs for mediation, but these incidents have been rare. Students generally appreciate their ready access to faculty and seek their counsel. Most cohorts have sustained contact with each other through digital media long after graduation, and most keep the program abreast of their jobs and achievements.

#### Challenges

In this particular urban setting, students who are navigating higher education for the first time in their family histories seem to express more stress and anxiety than there are resources available to support them. Some of these stressors are related to academic performance, but most pertain to finances14, living situations, and family and relationship circumstances beyond the classroom. Professors receive many letters of accommodation from the Disability Resource Center, and in the classroom one can expect students who are diagnosed and those who have yet to be diagnosed with conditions that impact learning and group dynamics. For some, the cohort model provides a peer group that serves as a surrogate family providing safety, acceptance, and some measure of predictability. Occasionally students behave in ways that are disruptive to the community of learners bringing their own histories and issues to the forefront. Managing dissonance requires clarity and cohesion on the part of the faculty and support from administrators. A thorough understanding of campus resources can be very useful in meeting both student and faculty needs.

### INSTITUTIONAL CHALLENGES TO COHORT MODEL/LEARNING COMMUNITIES AND OTHER BEST PRACTICES IN EDUCATION

A staunch proponent of learning communities, Smith (2, 48), observes three challenges in traditional university structures that are bettered through the formation of learning communities: the challenge of diversity, the challenge of institutional change, and the challenge of purpose. These are certainly relevant in today's academies and reflective of the challenges at UIC.

### Diversity of Students

Along with diversity comes a range of perspectives, cultures, customs, values, and life experiences. Student expectations are shaped by their families and traditions added to their own lived experience and aspirations. Most students are working to support themselves or their families, and others have family caregiving responsibilities. Their satisfaction with the curriculum can be influenced by all of these factors and the roles and demands that they juggle. Some students are explicitly directed by their family's wishes, while other students persist in spite of grim family circumstances to achieve baccalaureate degrees. It is important to recognize that student satisfaction measures can be influenced by any number of things including performance expectations or even how well-course schedules accommodate their commutes and work schedules.

It is not uncommon for students to dwell on future employment prospects more than the educational process at hand. Particularly for students who do not come from privileged backgrounds, remunerable employment is central to their motivation to attend college. Although education is a vehicle for obtaining knowledge and skills that can be applied in job settings, academic institutions set different priorities than employment agencies. Entry level job opportunities in public health are not always obvious to students. To prepare students to search for jobs and to present themselves in the job market, students need additional guidance and reassurance. Because these skills are important but non-academic in nature, they are presented through a mandatory, no-credit professional topics seminar and individual coaching through the Career Services Office in Student Affairs. Students find encouragement in the stories of credible informants, namely recent graduates who have found satisfying jobs. For a proportion of undergraduate students, the focus is on the next leg of their educational journeys. They, too, want to hear from students who have successfully matriculated into the graduate and professional programs to which they aspire.

### Institutional Change

#### Education as a Business and Financial Performance of Programs

Most campus budgets are driven by tuition revenue, and few institutions these days are brimming with reserves. Since undergraduate education enrollments are the largest proportion of tuition income, there is constant pressure to grow enrollments to support more costly graduate programs and other institutional

<sup>14</sup>A very high proportion of students on campus are PELL eligible, reported as 57% in AY2017/18.

needs and priorities. Cohort models have historically been associated with high impact but lower enrollment (2). Resnick et al. (49) review of undergraduate public health programs confirms the wide variability of programs from curriculum to enrollment, even among the subset of programs accredited by CEPH (49). Like the proliferation of many other health related programs, baccalaureate programs in public health have been viewed as revenue producing opportunities and as such are subject to being regarded as a means to an end. Consequently, curriculum decisions may be driven by economics rather than best practices.

A notable academic who, in his frustration to respond to the constant demand for institutional and accreditation metrics, coined the phrase, "the tyranny of metrics" (50). While his commentary pertained largely to measurement of student outcomes and faculty productivity, the tyrannical metrics can easily be expanded to include enrollment targets, tuition revenue data, retention rates, and graduation statistics. Education incurs costs, but the value of that education must also be considered. Acknowledging that educational institutions (and departments and programs) need to stay afloat, a purely business/accounting approach to educational endeavors can seem antithetical to the values we aim to engender through public health education. That is to say, the dollars, income and expenditures, don't tell the whole story. Some public health programs have boasted of hundreds of students enrolled in their programs or hundreds of graduates annually, and new programs continue to proliferate with the hope of the sacred cash cow arriving to ameliorate whatever financial wows a campus might be experiencing. This, however, does not seem to be a very mindful approach to designing meaningful education nor to developing communities of conscience. Employing high impact practices in education and especially implementing the learning community/cohort approach may not yield the profit margins of less intentional and more convenient and economical knowledge transfer approaches, but lasting value... now that's another matter that raises Smith's question of purpose (48).

#### The Challenge of Purpose

Reflecting on the purpose of the academic institution might well be the greatest determinant of the choices made about curricular approaches and pedagogies. The options are circumscribed by the priorities, so just what are the priorities? If an academic institution is genuinely devoted to best practices and maximal educational benefit, then strategies like learning communities and cohort models are viable choices. The question then becomes, how can we implement these approaches given the resources at hand? If, on the other hand, a more utilitarian view of education prevails, some good disperse across the greatest number, then the focus shifts to the economics of education, e.g., how can limited resources be distributed to be of some benefit to the greatest numbers of students? Depending on the philosophical positions and values of an institution, a college, or a department, different options will be considered. Invoking the old adage, we value what we measure may be appropriate to these considerations. If the metrics we choose are limited to enrollments, graduation statistics, and job placement figures, we risk bypassing and missing the impacts of learning communities and cohort models that are not so easily reduced. Somehow, we must reconcile the economics with curricular integrity.

#### Clarifying the Purpose of Undergraduate Public Health Education

The Institute of Medicine Report (10) that stimulated the rapid growth of undergraduate public health programs asserts that the point of education is to yield an educated citizenry. That is a broad educational objective that has been infused into many different endeavors some of which may not be compatible with very intentional curriculum, learning communities, and cohort models. Nevertheless, learning communities are valuable for solidifying values and mirroring collaboration in the public health arena affording students opportunities to fail and succeed in a learning environment while preparing for the ambiguities of the real world. At this point in the evolution of undergraduate programs, the landscape is just coming into focus (49). Some programs represent very intentional curriculum intended to prepare students to enter the public health work force. Others offer foundational public health degrees via a menu of public health courses culminating in baccalaureate degrees that serve as pipelines to other health related career trajectories (e.g., MD, DDS, DVM). Students in pre-health science programs will likely form primary professional identities outside public health even if they are informed by public health understandings. These different ends are likely to invoke different means, and cohort models and intentional learning communities may not be compatible with all undergraduate initiatives. Especially for programs intentionally preparing graduates to fulfill entry level roles in the public health workforce, learning communities remain a highly relevant educational strategy.

### QUESTIONS FOR FURTHER CONSIDERATION

As the undergraduate public health movement evolves, the full picture will become clearer. As has occurred within other disciplines and academic programs, there will likely be some adjustment in both the number and the nature of programs. Among the questions to be answered are:


To answer the questions, it is imperative that we re-examine and affirm the philosophical commitments of our academic institutions and of public health education. We must take into account our public health values and ethical commitments remembering that we are preparing students to address complex problems of vulnerable populations. The AACU endorsed best educational practices are likely to endure in any analysis, but we are challenged to find better ways to measure the extent of their impact. Given that the power and impact of learning communities/cohort models continue to resurface, it is likely that

#### REFERENCES


this strategy will hold a key to meaningful education and "deep learning" in undergraduate public health education.

#### AUTHOR CONTRIBUTIONS

KO conceptualized and wrote this article reflecting the collaboratively developed undergraduate program at the University of Illinois, School of Public Health.


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Opacich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Using Deliberative Pedagogy as a Tool for Critical Thinking and Career Preparation Among Undergraduate Public Health Students

#### Denise C. Nelson-Hurwitz\* and Opal Vanessa Buchthal

*Office of Public Health Studies, University of Hawai'i at Manoa, Honolulu, HI, United States ¯*

Engagement of undergraduate public health students in active learning pedagogy has been identified as critical for recruitment, retention, and career preparation efforts. One such tool for engagement that has proven successful in STEM programs is deliberative pedagogy, where it is used to stimulate student interest in research and policy applications of technical course content. Broadly applied, deliberative pedagogy is a consensus model of decision-making, applied as an in-class exercise, where students work in small groups and model a community task force with assigned group roles. In these groups, students collect evidence from literature and media sources, and prepare a consensus response to an assigned question. Here we present an adaptation of this pedagogy to provide undergraduates with the tools needed to actively engage in public health policy and planning work groups. This adaptation is first applied during an introductory public health course, where it is used as a tool for engagement and excitement, and as a critical thinking exercise. It additionally serves as an opportunity for students to apply information literacy skills and engage with research and policy initiatives discussed in class. The same tool is reintroduced prior to graduation in a capstone course, where the emphasis shifts to application of research skills and analytical concepts. The activity is also an opportunity for students to apply professional skills needed for engagement in program development, program evaluation, institutional policy, and legislative advocacy. Through application of this pedagogical tool at two critical time points in an undergraduate curriculum, students develop skills necessary for early career professionals and are better prepared to actively engage in policy and planning as it relates to critical public health initiatives, both locally and globally.

Keywords: public health education, bachelors of public health, undergraduate public health, undergraduate education, curriculum development, high-impact educational practices, career preparation, critical thinking

#### BACKGROUND AND RATIONALE

Education of undergraduate students involves a need to introduce, reinforce, and apply basic professional skills, in addition to imparting core content knowledge. Undergraduate education traditionally includes foundational general education courses for this purpose; it is also important for the student's specific degree program to facilitate application of professional skills within the framework of major-specific graduation criteria.

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Iffat Elbarazi, United Arab Emirates University, United Arab Emirates Edward J. Trapido, Louisiana State University Health Sciences Center New Orleans, United States*

\*Correspondence:

*Denise C. Nelson-Hurwitz denisene@hawaii.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *07 November 2018* Accepted: *13 February 2019* Published: *04 March 2019*

#### Citation:

*Nelson-Hurwitz DC and Buchthal OV (2019) Using Deliberative Pedagogy as a Tool for Critical Thinking and Career Preparation Among Undergraduate Public Health Students. Front. Public Health 7:37. doi: 10.3389/fpubh.2019.00037*

**73**

In the last decade there has been a dramatic expansion of public health education from it's historic focus on graduate education and curriculum design into undergraduate programs1,2 As a result, public health programs and faculty have needed to gain a better understanding of undergraduate student pedagogical needs beyond mastery of introductory public health-specific skills and content knowledge.

Employer assessments of college graduates further supports the need for graduates to demonstrate professional skills in critical thinking, team work and collaboration, and in the ability to apply quality information and data to solve real-world challenges. In a job outlook report conducted and published by the National Association of Colleges and Employers, 82.9% of surveyed respondents reported a need for problem-solving, teamwork, and leadership, with an equivalent 82.9% reporting a demand for graduates to demonstrate an ability to work as a team (1). In a similar study compiled at the request of the Association of American Colleges and Universities (AAC&U), employers reported asking for increased college-level emphasis on integrative learning (73% of respondents), teamwork skills (76%), critical thinking and analytical reasoning skills (73%), and the ability to locate, organize, and evaluate information from multiple sources (70%) (2).

The literature further supports the need to incorporate active learning pedagogy for increased student engagement (3, 4) and performance (5), which may have further implications for student recruitment and retention efforts.

Deliberative pedagogy is a consensus model of decisionmaking intended to model the function of a community task force (6). Applied in an educational setting as an inclass exercise, students work in small groups in the capacity of assigned group roles. In these groups, students work collaboratively to collect evidence from literature and media sources, then critically appraise and apply this evidence to develop a consensus response to an assigned question. Through employment of this pedagogical practice at two strategic time points in an undergraduate public health curriculum, students develop, and practice workplace-oriented skills in critical thinking, information appraisal and literacy, and teamwork/collaboration. Students are additionally prepared to actively engage in real-world public health challenges and apply quality information to professional engagement in program planning and policy development.

#### PEDAGOGICAL FRAMEWORK

Three high-impact educational practices are predominantly reflected in application of deliberative pedagogy. By nature, deliberative pedagogy is a collaborative project, conducted in small student groups, and requires undergraduate students to engage in research practices to find and evaluate credible evidence relevant to their assigned prompt questions. Application of both collaborative projects and undergraduate research are high impact educational practices (7). Focus topics of assigned group prompt questions further reflect values of diversity and global learning, another high-impact educational practice (7). Prompts assigned often focus on public health issues of global importance or implication, or local issues with broad implications. Diversity of opinion is promoted during discussion, most notably represented by the student designated as the "devil's advocate" of the group, as is the practice of applying high quality evidence to support the group's final consensus statement.

Deliberative democracy pedagogy applies several critical component elements (CCEs) identified by the Association of Schools and Programs of Public Health (ASPPH)<sup>3</sup> , and additionally utilizes Liberal Education and America's Promise (LEAP) learning objectives (8, 9). Relevant objectives relate to inquiry and analysis, critical thinking, information literacy, teamwork, problem solving, and social responsivity.

Team-based experiential learning approaches, as applied through deliberative pedagogy, have been shown to be effective in higher education broadly (10, 11), and specifically in application within public health education (12). Peer group learning, applied in small, semi-independent group settings, has been effective in promoting higher order skills in higher education (13), and inquiry-based activities applied in small groups have promoted the mastery of complex material among students (14). Undergraduate programs have been successful in application of a deliberative pedagogy to increase student ability to evaluate and synthesize information, and to actively engage in civic issues engagement (6, 15). The literature further provides support of the effectiveness of deliberative pedagogy in comparison to standard lecture format as evidenced by changes in student self-reported understanding of course content and increases in assessmentbased content knowledge (15).

### LEARNING ENVIRONMENT

The University of Hawai'i at Manoa (UHM) is a public research ¯ university with enrollment of 17,612 students, and undergraduate students comprising about 73% of enrollees<sup>4</sup> Public health education began at UHM in 1962 with the offering of graduatelevel public health degrees. The Bachelor of Arts in public health (BAPH) degree was added in January 2014, has produced more than 140 graduates, and currently supports approximately 170 declared majors.

Deliberative pedagogy was introduced into public health courses at UHM in fall 2015, and is currently applied at two strategic time points throughout the BAPH curriculum at UHM. It is first introduced during our PH 201 (Introduction to Public Health) course. In this course, the target audience centers on new and prospective undergraduates. As a large lecture format general education course, many, but not all, enrolled students are interested in further pursuing public health as a future field of study. Enrollment in the courses often ranges from 70 to 100 students per class. As an introductory course, the students

<sup>1</sup>https://bigfuture.collegeboard.org/majors/health-professions-related-clinicalsciences-public-health-public-health

<sup>2</sup>https://www.aspph.org/connect/data-center/

<sup>3</sup>http://www.aspph.org/educate/framing-the-future/

<sup>4</sup>https://manoa.hawaii.edu/about/

frequently have no prior exposure to public health concepts and the emphasis of the pedagogy centers on introducing and applying basic public health concepts and skills, including critical thinking skills, collaborative learning, and both collection, and application, of high-quality evidence. The application of this pedagogy further serves to promote awareness among new public health students of current issues of public health concern (e.g., climate change).

Early offerings of the PH 201 course had a small class size, ranging from 30 to 40 students per semester. In-class student debates on public health topics were employed as critical thinking exercises for students when class sizes were small. However, as enrollment in PH 201 increased in subsequent semesters, application of deliberative pedagogy replaced the student debate activities as an approach to scale the activity, while meeting similar critical thinking objectives.

This pedagogy is again applied during our PH 489 (Public Health Undergraduate Capstone Seminar) course. This is often the last public health course students enroll in prior to graduation. As the capstone course, it is intended to integrate students' prior classroom learning with the exposure to public health practice attained through their capstone service-learning or research experience, and to prepare students to enter the workforce or graduate study. As such, the target audience is advanced public health majors who have completed their required public health capstone projects and most, if not all, required coursework in public health. In this course, the emphasis of deliberative pedagogy shifts to center on career preparation and real-world application of public health skills and content. As a university designated writing intensive seminar format course, enrollment is set at a maximum of 20 students per class.

### LEARNING OBJECTIVES

While the deliberative pedagogy and approach is consistently applied throughout the curriculum, it has slightly different goals depending on the course, and subsequently the target audience of students, in which it is being implemented.

Learning objectives of the deliberative pedagogy in PH 201 (Introduction to Public Health) include the following:


Learning objectives of the deliberative pedagogy in PH 489 (Public Health Undergraduate Capstone Seminar) include the following:

1. Reinforcing students' critical thinking skills in public health practice


### PEDAGOGICAL FORMAT

Deliberative Pedagogy is a consensus decision-making activity modeled on task force protocol. Students work collaboratively in small groups to practice using critical thinking and teamwork to identify information needs and collect evidence, then develop, present, and defend a consensus position on an ethical issue in public health.

Preparation activities require 30–45 min of time allotted inclass to organize the group and plan work. Groups of 4–6 students are pre-assigned into designated group roles by the instructor, primarily for efficiency, but also to challenge students in roles they may not ordinarily have selected for themselves. Each group is assigned a focus question to investigate, and given a worksheet that prompts them to assign team roles, identify the underlying public health issues, develop the group's initial stance on the focus question, identify sub-questions to investigate and potential sources of evidence. Following in-class small group discussion, students are allotted 1–2 weeks for group members to investigate questions identified by the group discussion and collect data.

After the allotted time for research and collection of evidence, groups reconvene in class to review data and develop, present and defend their team's evidence-based consensus statement on the focus question. A final group worksheet documents consensus statement, team member roles, key pieces of evidence with American Psychological Association (APA) citations, and criteria for validity. A full class session is dedicated to this effort and includes 45–60 min for discussion and statement development, as well as 15 min for presentation/large group discussions.

In PH 489, where the deliberative democracy activities also are intended to reinforce teamwork as a critical public health job skill, an additional element is added between the two deliberative democracy activities. After the completion of the first deliberative democracy project, students are guided through an in-class discussion to reflect on the strengths and weaknesses of their teamwork, and uncover for themselves core concepts in team functioning. This understanding is then reinforced through a lecture and guided small-group activities on teamwork. Students are then organized into groups for the second deliberative democracy assignment, and each group is required to develop a team contract that addresses individual responsibilities, team rules and consequences.

Deliberative democracy pedagogy is implemented twice during PH 201- once centered on a topic of environmental health importance, and later centered on a topic of health policy TABLE 1 | Deliberative democracy implementation overview and sample student group statements.


source documentation, and brief reflection on group process

TABLE 1 | Continued


relevance. During PH 489, it is implemented twice- once centered on a topic of legislative advocacy, and later centered on a topic of community health and institutional policy. More specific details describing application of deliberative pedagogy in each PH 201 (Introduction to Public Health) and PH 489 (Public Health Undergraduate Capstone Seminar) are described in **Table 1**.

### OUTCOMES AND ASSESSMENT RESULTS

Since the deliberative democracy pedagogy is applied in slightly different ways, with variations in learning objectives, during two distinct courses within the UHM public health curriculum, outcome and assessment data are presented here separately.

### PH 201 (Introduction to Public Health) Outcomes and Assessment

When applied in PH 201 (Introduction to Public Health), deliberative democracy pedagogy is a emphasized as a tool through which college freshmen and sophomores with minimal background in public health apply basic public health concepts and skills, including critical thinking skills, collaborative learning, and application of high-quality evidence. The pedagogy also helps to encourage awareness of key public health issues, and promote student enthusiasm in the undergraduate degree program and in public health careers. PH 201 end-of-semester course evaluation data corresponding to the most recent semester, Spring 2018, is provided in **Table 2** as compared to aggregated data from the full UHM campus.

In coding of open-ended responses on the most current PH 201 course evaluation (Spring 2018, n = 64 responses), the top two coded reponses to "what do you feel was the most valuable aspect of this course?" was interactions or discussions (8 responses) and specifically named deliberative democracy discussions (7 responses). In a free comments section, the most common feedback (8 responses) indicated students wanted deliberative democracy activities to be expanded or utilized more frequently in the course.

Anecdotal evidence from end-of-semester interviews conducted with instructors of public health courses taken by students following PH 201 suggests students are able to better distinguish between high and low quality evidence, and TABLE 2 | PH 201 end-of-semester course evaluation data—Spring 2018\*.


\**Questions reported were optional additions to end-of-semester course evaluations. UHM Aggregate data reported reflect the UHM respondents among courses where the question was selected and asked of students.*

are better able to recognize when data are needed to support claims, compared to student demonstration of skills prior to implementation of deliberative democracy pedagogy.

### PH 489 (Undergraduate Public Health Capstone) Outcomes and Assessment

When applied in PH 489 (Undergraduate Public Health Capstone), deliberative democracy pedagogy is a career development tool promoting application of professional skills, and encouraging engagement of early career professionals in policy development and analysis. The pedagogy is intended to strengthen students' awareness of the role of policy advocacy in public health practice, to build students' familiarity with common data sources used in policy analysis, and to reinforce critical-thinking, professional teamwork and professional communication skills related to public health policy advocacy. PH 489 end-of-semester course evaluation data corresponding to the most recent semester, Spring 2018, is provided in **Table 3** as compared to aggregated data from the full UHM campus.

Open-ended responses to a question asking students to identify the most valuable parts of the course were coded. More than half (9 out of 14) of the comments identified in-class group activities like the deliberative democracy activities as the most valuable part of the class. They felt that these activities were valuable because they "helped me remember key points about public health," "involved practice of real-world skills," or "involved preparation for public health careers."


\**Questions reported were optional additions to end-of-semester course evaluations. UHM Aggregate data reported reflect the UHM respondents among courses where the question was selected and asked of students.*

Since the BAPH program is relatively new, data gathered from employers of graduates is pending, however, an employer survey is currently being developed with plans for deployment within the academic year.

#### DISCUSSION

Deliberative Democracy activities can foster critical thinking and help students make the link between policy and the concrete public health practice experiences (6, 15). Deliberative Democracy activities are closely related to task force or team activities routinely performed in public health practice, and can be an effective tool for career preparation. Learning objectives associated with Deliberative Democracy, also integrate well with public health Bachelor's degree program competencies as outlined by CEPH (Council on Education for Public Health) accreditation criteria (16) D10 (Public Health Bachelor's Degree Foundational Domains), D11 (Public Health Bachelor's Degree Foundational Competencies), and D13 (Public Health Bachelor's Degree Cross-Cutting Concepts and Experiences) as outlined in **Tables 4**, **5**.

TABLE 4 | Learning Objectives of Deliberative Pedagogy in PH 201 (Introduction to Public Health) Mapped with CEPH (Council on Education for Public Health) Accreditation Criteria\*.


\**Numbered values correspond to specific criteria on CEPH Data Templates available https://ceph.org/documents/32/2016templates.xlsx*

Throughout application of deliberative pedagogy at two strategic curriculum time points, student feedback and assessment activities lead to adaptations and evolution of pedagogical application. In working with a large, lectureformat course of introductory public health students within the framework of PH 201, it was clear that careful instruction and repetition was important, as was continuous monitoring during discussions. Preparation for deliberative pedagogy class sessions must include curricula focused on critical thinking, finding quality resources, and evaluating evidence, which were found to be essential in providing an adequate foundation of skills to apply during deliberative pedagogy sessions. The use of multiple focus, or prompt, questions was necessary to expose students to a range of issues related to a central topic, and avoid redundancy during report-back. Logistically, advanced assignment of student groups and designated roles was found to maximize time spent on discussion, rather than logistics. Grading of preparation activities and final products separately was also identified as necessary to address student absences during one of the two designated class sessions.

In applying deliberative pedagogy in a seminar-format course with advanced public health students within the framework of PH 489, we found students required prompting and specific guidance from the instructor to link the in-class deliberative pedagogy policy activities to related content introduced during prior courses. This is further supported by literature articulating the benefits of repetition throughout an undergraduate degree curriculum (17). Logistically, adding a class session between the first and second deliberative pedagogy sessions where students reflect on collaborative team function and discussed professional teamwork skills was found to enhance student skill development in the second session. Specifying a broad range of information sources to consult for policy questions supported advanced students in understanding the range of practical factors agency staff and leadership may weigh in developing an agency response to a public health concern. Finally, deliberative pedagogy sessions were found to have greater salience for advanced students when the connection to public health career skills was made explicit throughout the activity.

TABLE 5 | Learning objectives of deliberative pedagogy in PH 489 (Undergraduate Public Health Capstone) Mapped with CEPH (Council on Education for Public Health) Accreditation Criteria\*.


\**Numbered values correspond to specific criteria on CEPH Data Templates available https://ceph.org/documents/32/2016templates.xlsx*

Deliberative Democracy pedagogy is successfully applied at UHM during two critical time points in the undergraduate public health curriculum. Data suggests it is successful in meeting intended learning objectives, including the application of critical thinking skills and promoting the linkage between policy and the concrete public health practice experiences. Deliberative Democracy activities are closely related to task force or team activities routinely performed in public health practice, and can be an effective tool for career preparation.

### DATA AVAILABILITY

The datasets generated for this study are available on request to the corresponding author.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the University of Hawai'i (UH) Human Studies Program as exempt from federal regulations pertaining to the protection of human research participants. Authority for the exemption applicable is documented in the Code of Federal Regulations at 45 CFR 46.101(b) 4. The protocol was approved by the Office of Research Compliance, University of Hawai'i system (Protocol Number 2018-00751).

#### REFERENCES


#### AUTHOR CONTRIBUTIONS

DN-H: initial conception and adaptation of the pedagogy to both courses; DN-H and OB: contributed substantial reformatting of the pedagogy; DN-H: wrote the first draft of the manuscript. Both authors wrote sections of the manuscript, contributed to manuscript revision, read, and approved the submitted version.

#### FUNDING

Contributions by DN-H partially funded in relation to Building Infrastructure Leading to Diversity (BUILD): Enhancing Cross Disciplinary Infrastructure and Training at Oregon (EXITO), funded by the National Institute on Minority Health and Health Disparities (1RL5MD009591-01).

#### ACKNOWLEDGMENTS

We acknowledge initial introduction to this pedagogy provided through project-related discussions as a part of Building Infrastructure Leading to Diversity (BUILD): Enhancing Cross Disciplinary Infrastructure and Training at Oregon (EXITO) project funded by the National Institute on Minority Health and Health Disparities (1RL5MD009591-01). Curriculum development support related to the PH 489 course provided by Ms. Michelle Tagorda. The authors gratefully acknowledge their contributions.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Nelson-Hurwitz and Buchthal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Integrative Approaches to the Undergraduate Public Health Major Curriculum: Strengths, Challenges, and Examples

#### Marc T. Kiviniemi <sup>1</sup> \* and Sarahmona M. Przybyla<sup>2</sup>

*<sup>1</sup> Department of Health, Behavior and Society, College of Public Health, University of Kentucky, Lexington, KY, United States, <sup>2</sup> Department of Community Health and Health Behavior, School of Public Health and Health Professions, University at Buffalo, Buffalo, NY, United States*

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Darcell P. Scharff, Saint Louis University, United States Donna Jeanne Petersen, University of South Florida, United States Terrel Lee Rhodes, Association of American Colleges and Universities, United States*

> \*Correspondence: *Marc T. Kiviniemi marc.kiviniemi@uky.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *09 November 2018* Accepted: *11 April 2019* Published: *30 April 2019*

#### Citation:

*Kiviniemi MT and Przybyla SM (2019) Integrative Approaches to the Undergraduate Public Health Major Curriculum: Strengths, Challenges, and Examples. Front. Public Health 7:106. doi: 10.3389/fpubh.2019.00106* Many "first generation" undergraduate public health degree programs were designed based on "siloed" course structures centered around subunits in the discipline (e.g., Introduction to Epidemiology, Introduction to Environmental Health) that may be meaningful primarily to experts in the field. An alternative to the siloed approach is an integrative curricular design, in which courses are designed around meaningful thematic units (e.g., explaining public health problems, asking and answering scientific questions in public health), with an emphasis on drawing connections between knowledge from different but complementary disciplinary areas as a means to improve student learning and retention. The integrative approach shifts the curriculum conversation to capitalize on the interdisciplinary roots of the public health profession. This approach is consistent with the learning outcome recommendations in the Framing the Future Task Force report and in the CEPH requirements for the undergraduate public health major. We explore integrative approaches to developing curricular models for undergraduate public health programs and discuss both pedagogical and career preparation arguments supporting an integrative curriculum approach. These include facilitating the often-challenging task for students of seeing how concepts interrelate, making transparent how "basic" knowledge in the discipline relates to "real world" applications of the content, and better mirroring how professionals in the discipline actually use knowledge in practice. Finally, we review examples of core concepts and features in an integrative curriculum approach to the undergraduate public health major as an effective educational program with high-quality, learner-centered educational experiences.

Keywords: undergraduate public health, BSPH, integrative public health curriculum, curriculum designed, student learning

## INTRODUCTION

The goal of any undergraduate curriculum, including public health, we would argue, is to accomplish two objectives. First, it should teach students the "habits of thought" that characterize the discipline—how do public health professionals think about problems, analyze them, and go about solving them? Second, it should equip students with the essential skills necessary to function as entry-level public health professionals [for discussion of these two goals and their possible rapprochement see (1)].

As a way of considering what is involved in meeting these two goals, consider a classic problem in public health education the "potato salad" problem (2). A group of people goes on a picnic and they eat several foods, including potato salad. Later, some of them get sick (and, in true public health fashion, some of them do not). They provide some data on what foods they ate at the picnic, and the task for our budding public health professional is to decide whether they got sick because they ate the potato salad. As it has been traditionally done in class, this is epidemiology content pure and simple—students craft out a table calculating the disease rate for those who ate the potato salad relative to those who did not and, if they want to get really fancy, compare that to the disease rate for people who ate other foods vs. those who did not. A few simple calculations later and, voilà, the potato salad is identified as the cause of the sickness and students have a piece of knowledge on how to use an epidemiological principle to answer a public health question.

As a class example to teach and test a piece of knowledge from epidemiology, this works well as is. But going back to the articulated goals for a public health curriculum, we need to consider phenomena as they would occur in the "real world." In the real world of public health, the problem that would present itself to a member of the public health workforce wouldn't be friends on a picnic, it would be multiple people in the community buying vats of potato salad at the grocery store's deli counter to serve at their picnics and a set of ambiguous, incomplete case reports trickling in from around the community and needing to be assembled into a coherent whole. Also, in the real world a data table wouldn't magically appear from which students could calculate disease rates. The public health worker would have to use community engagement skills to go out, collect relevant data, assemble information from people's memory, and overcome community suspicions. Then, the data analytic skills above would surface. Even then, though, the real world work wouldn't be done. The public health worker would have to consult with a legal team to decide what steps could be taken to address the situation within the bounds of the city or county health department's authority. Health communications work would then need to come into play to develop the optimal strategy for getting the information disseminated to the public in a way that not only informs but also leads to needed action. The contrast between the class example and the real world is telling. Real world public health requires the integration of knowledge and skills from across the spectrum of traditional public health disciplines in order to understand and address the problem. It also requires a more complex set of thinking and analytic skills than simply the calculation of an odds ratio from a 2 × 2 table.

The primary question for public health education, and the question we seek to address in this paper is: how do we best define and develop undergraduate public health education programming in a way that allows us to simultaneously meet two critical goals—to both best educate students to understand and analyze the complexities involved and to ensure that students leave our programs with skills that allow them to do things to address these complex problems when they engage in the "real world" work of public health practice? This real world of public health involves the kinds of complexities in the examples above, and such complexities are growing over time as we shift toward a public health paradigm that explicitly addresses social determinants (3), pushes for an approach of health in all policies (4), and shifts focus toward one health (5) and other explicitly interdisciplinary, multilevel approaches. In doing so, we ask a key question at the level of the curriculum for the undergraduate public health major—what high impact, learner-center practices and approaches should be used in developing and implementing a curriculum in order to maximize student learning and student outcomes?

In this paper, we make the argument that achieving the goals of undergraduate education within the interdisciplinary context of public health is best done by designing curricula using an intentional, integrative approach. We begin by discussing what is meant by an integrative approach. We then advance arguments for the effectiveness of an integrative approach within the context of the goals of undergraduate education and why we would characterize it as a high impact approach to the undergraduate public health curriculum. Finally, we describe an exemplar of an undergraduate public health (UGPH) curriculum developed using principles of integrative curriculum design.

### WHAT IS AN INTEGRATIVE CURRICULUM MODEL?

To develop an integrative Bachelor of Science in Public Health (BSPH) curriculum, one's team must first examine existing definitions. Jacobs defines interdisciplinary learning as "a knowledge view and curriculum approach that consciously applies methodology and language from more than one discipline to examine a central theme, issue, problem, topic, or experience" [(6), p. 8]. Public health curriculum developers must also recognize the discipline field as a "specific body of teachable knowledge with its own background of education, training, procedures, methods, and content areas" [(6), p. 7].

The key feature of an integrative curriculum model is taking a conscious approach to integrating in coursework and drawing explicit educational connections between pieces of knowledge from particular separate disciplinary backgrounds (7). Thus, one does not define the curriculum in terms of a set of courses that individually cover particular disciplinary domains, but rather defines in terms of common questions, problems, ideas, skills or other meaningful thematic units built from the connections drawn as described above. Consequently, integrative curricula can have a variety of definitions [for examples see (8, 9)]. A common approach in medical education has been to replace separate and distinct "disciplinary" courses in the first 2 years of the curriculum with interdisciplinary courses organized around organ systems (10). In our UGPH program, the organization of a curriculum around meaningful thematic units was designed with both attention to what thematic organization best meets student learning objectives and best captures the "real world" uses of the curriculum material.

### WHY AN INTEGRATIVE CURRICULUM APPROACH?

With the definition of an integrative curriculum model in mind, we now turn to a brief discussion of why a unit or department would want to consider an integrative approach to developing an undergraduate public health curriculum and the advantages that it may have over the more traditional, siloed approach. Why would one design or redesign an undergraduate public health curriculum from an integrative standpoint?

### What Are the Goals of an Undergraduate Curriculum?

Central to our arguments about the advantages of an integrative approach relative to the traditional, siloed model is a set of basic assumptions about the desired outcomes of an undergraduate degree in public health. We assume that any undergraduate public health program has as its goal meeting at least four key outcomes.

First, we want to develop and implement a curriculum that prepares undergraduate students with the knowledge, skills, and values necessary to enter the public health workforce (broadly defined) ready to successfully begin their careers or further their education and contribute to addressing the public health needs of the population from their first jobs and continuing through the remainder of their careers (11, 12).

Second, we want to offer a curriculum that introduces students to how public health thinks about the world, the ideas and lenses through which one takes a public health perspective in explaining and preparing to change and improve the world, that provides reasoning and problem-solving skills and that introduces students to the values and underlying principles that inform the public health perspective (1). Although programs and curricula can vary in the relative weighting they place on goals one and two (13) both are, we would argue, necessary for a high-quality curricular approach.

Third, for undergraduate programs situated within schools or programs of public health or for those that have pursued standalone baccalaureate program status from the Council on Education in Public Health, a key goal is to ensure that the curriculum allows full satisfaction of the required learning outcomes for an accredited undergraduate degree in public health. The accreditation requirements for schools and programs offering the undergraduate degree articulate both content knowledge (e.g., "the socioeconomic, behavioral, biological, environmental, and other factors that impact human health and contribute to health disparities") and demonstrable skills (e.g., "the ability to locate, use, evaluate and synthesize public health information") that must be addressed in the curriculum (14).

The preceding goals don't occur unless we achieve a fourth, final goal—a goal of undergraduate education (and truthfully, all higher education), is to produce student learning and especially to produce high-quality learning that a student retains over a period of time—in other words, although the task we undertake in our classrooms is providing instruction, the ultimate goal in our courses and overall programs is to produce learning in our students (15, 16). Although this seems like a statement of the blindingly obvious, it needs to be explicitly stated because designing the curriculum in a way that best produces longterm learning needs to be an explicit goal when one engages in curriculum design.

### How and Why Does an Integrative Curriculum Help With Meeting These Goals?

So, from the viewpoint of these four goals for undergraduate education, why is an integrative curriculum approach potentially a better bet than the traditional, siloed approach? Let's consider potential integrative curriculum advantages from the vantage point of the four goals of undergraduate education outlined above.

#### The Nature of the Field of Public Health

For the first goal, preparing students for the workforce, the core argument for the integrative curriculum approach is that the work that is done in public health is inherently integrated. Very few of the tasks a public health professional undertakes in her daily life, and virtually none of the problems that public health professionals tackle, neatly fit into a single one of the silos defining the traditional curriculum approach.

The potato salad example above illustrates this point nicely. Identifying, developing plans to address, intervening to address, and evaluating whether the intervention was effective involves knowledge and skills that span multiple knowledge domains within public health. Virtually every public health example is similar. A group of residents from a particular neighborhood are concerned about what they perceive as high rates of respiratory problems in their children. Is something going on and if so, how is the local health department going to respond? Even answering the "is there a disease cluster?" in this community question involves the combined knowledge set of epidemiology and biostatistics, and the skills to gather the requisite exposure information from community members and health outcome knowledge requires skills from both community health and health services perspectives. We're only one step into fully addressing the problem and already addressing it involves skills in four of the traditional knowledge domains of public health.

We leave it to the reader to conduct additional thought experiments here if additional illustrations are needed. Suffice it to say that we take as a given that "real world" public health work has an inherent and inescapable requirement for integrated knowledge and skills across public health disciplines. We also take it as a given that as the public health challenges faced by the population shift over time, the need for integrated knowledge and skills will only grow. Emerging perspectives shaping the field all require more integrated knowledge and skills than did John Snow's work with the Broad Street Pump in 1854 (17) (and we would argue that there was some integrated knowledge at work even there).

A core principle in education and curriculum design is that one provides a learner with mastery of a set of program-level learning outcomes by providing instruction, practice, and the ability to demonstrate mastery of the types of skills that the learner will need to perform to be successful moving forward. To that end, if the reader buys our argument that the "real world" skills our undergraduate public health students will be expected to perform are integrated skills, then the work we do in our classrooms must be to teach and allow the opportunity to master those skills. If we teach a series of siloed courses in which students learn and practice skills in epidemiology in an epidemiology course and skills in biostatistics in a biostatistics course and so on, when and where do they get the opportunity to practice the integration of those skills?

In a siloed curriculum model, the student is most often left to master the integration of skills on her or his own. This approach rests on two rather tenuous assumptions–that the student should master them and that the student can master them. The should part comes down to a belief about educational philosophy. While we hope that the reader will be convinced by the end of this paper that students should not be left to master integration of siloed content knowledge on their own, that has to be a judgment made by the curriculum designers.

The question of whether studentscan master siloed knowledge integration on their own is an evidence-based question. The predominance of evidence suggests that the answer to this question is more often than not no. Evidence on retention of knowledge and ability to integrate skills across domains shows that often students are unable to integrate unaided (18, 19). Although integrating information and applying it to novel situations is a key educational outcome, students often do not integrate and extend application successfully on their own [e.g., (19)]. In fact, some attempts to accomplish integration by maintaining siloed courses but using common themes and topics to draw connections have found that students didn't even notice the integrated material until it was explicitly pointed out by faculty members.

#### The Nature of Student Learning

As discussed above, the central goal of education is not teaching a particular set of content; it is creating student learning, particularly learning that is long-lasting and able to be effectively retrieved and used when needed in a student's post-collegiate life and work (15, 16). Curriculum designs focused on ideas, themes, and answering public health questions can be more effective than designs focused on delivering information and leaving it to students to integrate (16, 20, 21). As such, improved learning outcomes and improved student motivation are core arguments for the integrative approach (22–24).

Public health is an evidence-based discipline–we (rightly) ask about the evidence base for interventions and policy approaches to address public health problems, we recognize assessment and evaluation as core functions of public health, and at all levels of the curriculum, we take as a core goal providing students the skills to assess, interpret, and evaluate the evidence for an approach. It behooves us, then, to apply evidence-based practices to the design of our curricula and courses. There are a variety of articulations of these best practices [e.g., (25)] and, notably, siloed curriculum approaches are not among them. Indeed, an overview of teaching best practices for long-term learning argued that ". . . it would be difficult to design an educational model that is more at odds with current research on human cognition than the one that is used in most colleges and universities" [(25), p. 4]. Within the context of these best practices, integrative learning is frequently cited as a high-quality educational practice and as a means of producing high-quality, long-lasting learning. Integrating allows focus on the "big ideas"—often cited as a best practice in curriculum design (16). The American Association of Colleges and Universities (AACU) has argued that the ability to integrate and synthesize knowledge is one of the four essential outcomes of an undergraduate educational experience (26).

Accomplishing integrative learning involves several shifts from "surface level" taking in of facts to "deeper level" thinking about connections across time and across contexts (27), from thinking about disciplinary and course contexts as isolated from one another to thinking about the webs of connection that tie things together to solve "real world" issues (19).

Unfortunately, typical undergraduate students are unlikely to learn very real and necessary connections on their own. There are two types of integration involved—first, students may not see how concepts from one course/discipline/module connect to related concepts from another course/discipline/module (typically termed horizontal integration). Second, students may not see how "basic" or "conceptual" material concepts relate to applications and professional lives (vertical integration) (10). Noticing, appreciating, and accomplishing integration is often one of the most challenging cognitive tasks we ask students to undertake. By making explicit the connections between ideas, and indeed framing the curriculum around those connections, the integrative curriculum approach addresses directly the difficult cognitive task that, in more traditional siloed approaches, is left to students to accomplish on their own.

In addition to the direct cognitive task of integrating knowledge, a second learning-based argument for the integrative approach is that learning is only effective if it is long-term learning that can be retrieved and used in the future. In terms of teaching and learning best practices, the argument has been made that "We need to provide an education that lasts a lifetime, which means thinking beyond the end of the semester, and let the learning principles for long-term retention and flexible recall guide our teaching practices" [(25), p. 4].

In the context of long-term retention and retrieval of information, some key principles of human memory and how humans learn highlight advantages of the integrative approach. In order to be remembered, retrieved, and used, delivered course content has to be tied to other, pre-existing memory structures (28). An item in isolation is much less likely to be effectively learned and remembered than one that is tied to preexisting knowledge. Organization and association of information in memory is key to long term retention and undemanding. It aids initial understanding by contextualizing the information in light of other information, but also aids the likelihood of long-term recall (29, 30). Full mastery of learning content involves multiple steps—acquiring skills/knowledge; integrating the individual skills and knowledge with others; and then applying those skills and knowledge appropriately to address and understand problems, including extending the domains for that application past the specific domain in which the skill was acquired (18).

Finally, not only can integrative curriculum designs aid students in seeing and understanding critical interconnections between content domains and in learning course material in ways conducive to long-term information retrieval, such design also aids in developing critical higher-order thinking and reasoning skills. One example of this is creative generation of new ideas. One conceptual approaching and fostering creativity is a "Janusian approach" in which creative thinking involves creating connection between ideas that might otherwise not seem connected at all (31). To the extent that we want our students to be both creative thinkers and problem solvers, arguably a core professional skill for public health where problem solving and intervention development involves working within systems and with multiple constraints to solve health problems that may not have existed when the student was in college, developing the ability to see connections between seemingly disparate ideas is critical. As discussed above, most students are unlikely to see or form such connections on their own and therefore a goal of education has to be teaching those connections as a core, concrete skill to be introduced practiced, and mastered in the course of the undergraduate program (7).

As another thinking skills domain, consider the need to take ideas and skills developed in the classroom and apply those skills to solving a problem that may be in a novel domain. Such an ability to transfer across domains is not inherent or automatic and therefore must be learned, and curriculum design needs to take this necessary learning into account. We know that repeated practice with different applications and different spheres is advantageous—organizing around the skill/topic may help (32). In fact, some have even argued that course content should be framed by laypeople because laypeople tend to think about things in terms of actual problems to be solved whereas professors tend to focus on subject matter (33).

For all of these student learning-focused reasons, we argue that an integrative curriculum approach offers advantages in engaging student learning. We are not the only ones to support this strategy. Many curriculum reform efforts advocate for integrative learning as a core feature of highquality educational programs across various disciplines? (20, 34, 35). The AACU argues for creating "intentional learners" and defines that, among other things, "Intentional learners are integrative thinkers who can see connections in seemingly disparate information and draw on a wide range of knowledge to make decisions. They adapt the skills learned in one situation to problems encountered in another" [(36), p. 21].

In the context of undergraduate degree programs in public health, where there are accreditation requirements specified by the Council on Education in Public Health, one can also consider the learning advantages of the integrative curriculum approach from the perspective of meeting accreditation requirements. Although the CEPH accreditation requirements (14) do not specify any particular curricular approach, we would argue that they do implicitly articulate a vision for undergraduate public health education that has integrative themes woven throughout. Themes like multilevel, socialecological approaches to understanding and addressing problems, systems thinking, and synthesis of information are all inherently integrative. Moreover, regardless of the curricular approach taken to address the knowledge and skill domains specified by CEPH, one must demonstrate student learning and outcomes relevant to those domains. To that extent, the articulated advantages of the integrative approach for student learning are also advantageous for satisfying the accreditation requirements of ensuring student learning of what is inherently an integrative, interdisciplinary field of study.

#### Curriculum Synergies

As a final argument for the advantages of an integrative undergraduate public health curriculum, our experience of developing the integrative curriculum approach described in detail below is that there were interesting, unexpected synergies that occurred when considering the curriculum from an integrative perspective. The first of those synergies is that we discovered there were some topics that don't have a natural "home" in any of the siloed, disciplinary courses typical of nonintegrated curricula but that emerge and fit quite nicely in an integrative approach. For example, our integrative curriculum includes a course on public health intervention approaches that uses the social-ecological model as an organizing framework. When developing content for the course, we realized that food fortification was an ideal public health strategy to talk about as a basic biological intervention in the ecological framework. We then realized that food fortification wouldn't likely come up naturally in any of the coverage of intervention approaches in a siloed model—health education and individual interventions would be covered in health behavior, policy approaches would be addressed in a health systems/services course, and screening would be covered in epidemiology, but food fortification wouldn't fit neatly into any of the necessary disciplinary "bins" and therefore likely wouldn't be covered.

In addition, the integrative approach allows for efficiencies in coverage of key concepts that are more challenging to achieve in a siloed approach. Consider the issue of different models for explaining public health problems. We use the epidemiologic triad to explain infectious disease transmission, the exposure pathways model to describe human exposure to environmental pollutants, and the social-ecological model to describe the complex causation of chronic disease risk (and other public health outcomes). Understanding each of the models involves being able to think about multiple constructs as causes or influences, characterizing and understanding the interrelations between the multiple constructs, and understanding the different ways in which constructs can intersect and interact to determine outcomes.

In a siloed approach, where the epidemiologic triad would be covered in an epidemiology course, the social-ecological model in a health behavior course, and the exposure pathways



model in an environmental health course, the relevant tasks of understanding multiple causes would have to be taught each time. In an integrative curriculum model, though, one can organize an undergraduate course around ways of understanding and explaining public health problems, teach the basic logic of multiple causes early on in the semester, and then introduce all three models in turn and use teaching and applying them as a way to provide repetition and practice across novel contexts as each model is learned and the principles of multiple causation observed and used repeatedly.

### AN EXAMPLE INTEGRATIVE CURRICULUM APPROACH

At the University at Buffalo's School of Public Health and Health Professions, the UGPH program began in Fall 2017 as the first ever BSPH program in the School. Incoming freshmen began direct admission in Fall 2018. From its inception, the UGPH curriculum is designed with five key elements: (1) major building blocks, (2) introductory coursework, (3) upperlevel coursework, (4) electives, and (5) one capstone course. The major building block courses entail 11 credit hours in

#### TABLE 2 | Undergraduate public health courses at the University at Buffalo.


the College of Arts and Sciences and include coursework in chemistry, political science, and statistics as well as 4 credit hours in the School of Medicine for a human physiology course. At the introductory level, students take two required lower division courses, with 200–300 students typically in each course. These courses expose them to a broad overview of the discipline and include basic principles of population health, with an integration of both historical and contemporary public health problems as a method to improve public health via the explicit integration of both content knowledge and approaches from all five core areas. At the upper division level, students take 16 credits over five courses, each currently capped at 75 students. Each integrate core curriculum content in a reflective manner, where students are challenged to assimilate the subject matter using deliberate teaching and evaluation criteria. At the upper division level, students complete nine credits of upperlevel electives from a growing menu of options (e.g., Public Health Nutrition, Social Determinants of Health). Finally, a four-credit capstone experience offers students the opportunity to synthesize and apply the knowledge and skills developed in previous coursework and out-of-classroom experiences in a holistic way. The capstone is capped at 30 students with the first cohort of BSPH graduates in Spring 2019. We envision several flavors of the capstone including but not limited to an independent research project, a study abroad experience, and a public health internship at a partnering public health agency (see **Tables 1**, **2**). The program currently has approximately 280 students enrolled.

## BSPH Curriculum—Distinctive Design Features

There are several advantages of using an integrative curriculum approach to the design and implementation of a BSPH program. First, the deliberate, distinctive design features of our UGPH curriculum allow for the careful and thoughtful integration of the five core public health areas in each and every one of the core courses throughout the curriculum, allowing for the high impact delivery of integrative learning experiences throughout the students' time in the major (36). In particular, students are presented with various opportunities for both breadth of exposure to core public content and ways of thinking and depth in a focus area of interest. The inclusion of a flexible capstone course allow synthesis and integration based on student experiences (e.g., undergraduate research, study abroad, internship), incorporating Kuh's capstone experiences as a high impact practice (37). Second, the leveraging of general education provides disciplinary foundations for public health learning at lower levels of the required curriculum. In designing the core courses, we identified general education offerings that provided important background knowledge relevant to each course and made those general education offerings prerequisites to the public health core. For example, a human physiology general education course is a pre-requisite for the core course addressing biological, psychosocial, and environmental mechanisms of health and disease and a political science introductory course is a pre-requisite for the course on public health systems and policies. This leveraging of general education not only makes the curriculum stronger, but also illustrates for students the integration of core knowledge from multiple disciplines into the public health approach to addressing and understanding health. The University at Buffalo's core general education curriculum explicitly incorporates several high impact practices into its design, including first year experiences, use of portfolios, capstone experiences at the end of the general education sequence, common intellectual experiences, and diversity/global learning (37). Third, the curriculum design allows for two primary options for student learning experiences: many students will assume a "stand alone" baccalaureate experience and others may opt for a seamless transition into a BSPH + MPH without unnecessary duplication for master's level coursework (in either a 3+2 or 4+2 track). The curriculum development and implementation plans were explicitly informed by and responsive to Framing the Future. The content coverage takes a thoughtful approach to integrate the course content, allowing students to make often implicit connections between courses explicit. Importantly, the UGPH curriculum successfully incorporates Critical Component Elements (see **Table 1**) from ASPPH's Framing the Future project (38, 39).

### BSPH Course Examples

To highlight samples of what an integrative approach to curriculum development and implementation looks like, we present two course examples in our BSPH program: PUB 315: Asking and Answering Scientific Questions in Public Health and PUB 320: Models and Mechanisms for Understanding Public Health.

In PUB 315, students engage in an overview of scientific methodology and evidence-based practice in public health where they learn about the epidemiological research methods used to collect data and the biostatistical methods used to evaluate that data in public health research and practice. They begin the course by learning about the importance of evidence-based practice for public health with an overview of the research process and the development of research questions. Students examine the types of empirical questions addressed in the discipline and the links between the types of questions and appropriate methods to collect, manage, and analyze public health data. These learning synergies provide students the opportunity to deliberately engage in understanding the interrelation between not only key public health concepts but also draw connections between conceptual commonalities in all five core areas of the discipline.

In PUB 320, students engage in course material to learn about how we understand and explain the causes of public health problems. Using active learning techniques, students gain an understanding of the complex causal mechanisms of different types of public health problems, including infectious diseases, chronic diseases, and environmental health hazards. These active learning strategies include working in groups to develop and apply multilevel explanations of public health problems and to develop knowledge through activities designed to compare and contrast the ways in which different models in public health reflect systems thinking principles. By first learning how to apply basic principles of model building to analyze and explain public health problems, they then explore the importance of identifying and understanding the relationship between and among causes within complex systems using various levels of the social ecological model and the epidemiologic triad. Using experiential learning strategies, students gain skills in describing and explaining factors that influence health-related behaviors using public health theories and environmental models. For example, students work to create the textbook through for the course, curating, and writing about core content knowledge in a way that involves experiential engagement in knowledge creation (40). Finally, students explore fundamental causes and use model applications to reduce health disparities at a population level using principles in the five core areas of public health.

### CONCLUSION

Integrative learning is an educational practice intended to produce high-quality, long-lasting learning experiences. Consistent with CEPH requirements for the undergraduate public health major and learning outcome recommendations in the Framing the Future Task Force report, an integrative approach to curriculum design and implementation derives from the interdisciplinary roots of the public health profession and the key notion that public health work is fundamentally integrated in nature. From a curriculum design perspective, it is critical to develop an educational program at the bachelor's level that helps students establish knowledge, problem-solving, and critical thinking skills for understanding and analyzing the complexities involved in public health phenomena as well as fostering proficiencies to effectively respond to these complex problems in the "real world" work of public health practice. These competencies focus on the utilization of interdisciplinary, multilevel approaches. An integrative curriculum for undergraduate public health students prepares them to enter the public health workforce with thoughtful, deliberate synthesis of key

#### REFERENCES


principles in our field and provides them with disciplinary foundations for their future careers as public health practitioners and researchers.

#### AUTHOR CONTRIBUTIONS

MK contributed to the conceptualization and design of the manuscript. MK and SP co-wrote the first draft of the manuscript. Each author contributed to manuscript revision, read, and approved the submitted version.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Kiviniemi and Przybyla. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Integrative Learning in US Undergraduate Public Health Education: A Review of Student Perceptions of Effective High-Impact Educational Practices at Georgia State University

#### Elizabeth Armstrong-Mensah<sup>1</sup> \*, Kim Ramsey-White<sup>1</sup> and Ernest Alema-Mensah<sup>2</sup>

*<sup>1</sup> Department of Health Policy and Behavioral Science, Georgia State University, Atlanta, GA, United States, <sup>2</sup> Morehouse School of Medicine, Atlanta, GA, United States*

#### Edited by:

*Andrew Harver, University of North Carolina at Charlotte, United States*

#### Reviewed by:

*Darcell P. Scharff, Saint Louis University, United States Edward J. Trapido, Louisiana State University, United States*

#### \*Correspondence:

*Elizabeth Armstrong-Mensah earmstrongmensah@gsu.edu*

#### Specialty section:

*This article was submitted to Public Health Education and Promotion, a section of the journal Frontiers in Public Health*

Received: *20 November 2018* Accepted: *08 April 2019* Published: *30 April 2019*

#### Citation:

*Armstrong-Mensah E, Ramsey-White K and Alema-Mensah E (2019) Integrative Learning in US Undergraduate Public Health Education: A Review of Student Perceptions of Effective High-Impact Educational Practices at Georgia State University. Front. Public Health 7:101. doi: 10.3389/fpubh.2019.00101* In 2003, the United States (US) Institute of Medicine of the National Academies recommended that all undergraduate students have access to an education in public health to assist with diversifying the public health workforce and ensuring an educated citizenry on public health issues. In line with this recommendation, and that of the Consensus Conference on Undergraduate Public Health Education, Georgia State University established a Bachelor of Science in Public Health (BSPH) program in 2016, with the mission of advancing health through leadership, scholarship, research, and service, to better the human condition and to promote the common good, especially for urban communities in the US and for global populations. Using integrative approaches that encourage student empowerment, self-development, integrative thinking, and reflective learning, the Georgia State University BSPH program currently offers a range of generalist introductory public health courses to over 400 students. This review seeks to examine student perceptions of integrative practices utilized by Georgia State University faculty in the BSPH program and to investigate the extent to which student perceive these integrative educational practices as preparing them to use insights gained in the classroom and from the field, to question, modify, connect, and integrate material learned in the academic setting, to real-life public health challenges. It also seeks to identify which of the integrative educational practices have the highest impact of helping students integrate the knowledge and skills gained to public health issues.

Keywords: integrative learning, high impact educational practices, undergraduate course work and assignments, service learning/community-based learning, collaborative projects, study abroad, undergraduate research, signature experience

### INTRODUCTION

Georgia State University is an urban public research institution located in Atlanta, Georgia. Established in 1913, the university has seven campuses throughout metro Atlanta with over 51,000 students of diverse backgrounds, enrolled in over 250 degree programs in 100 fields of study, including public health (1). In June 2016, The Council on Education for Public Health (CEPH) Board of Councilors accredited the School of Public Health (SPH) at Georgia State University. Georgia State University's SPH began as a Master of Public Health (MPH) program, accredited by CEPH since 2007—making it the first public university in Atlanta to gain that distinction (1). In 2016, a Bachelor's of Science in Public Health (BSPH) program was created within the SPH, with an emphasis on urban and global public health. The new BSPH program leverages the existing interdisciplinary make-up of Georgia State University's School of Public Health. The BSPH program seeks to prepare students for course work across public health disciplines, and to equip graduates with cross-professional competencies for public health jobs with urban, and global public health organizations. The SPH does this by utilizing integrative learning approaches that encourage student empowerment and self-development, and by equipping students with the requisite knowledge and skills to be integrative thinkers, critical and analytical problem solvers, and reflective learners. The establishment of the BSPH program supports Goal 3 of the Georgia State University Strategic Plan, as well as the recommendation of the Institute of Medicine (IOM) that, "all undergraduates should have access to education in public health" (2). The establishment of the BSPH program is also consistent with the IOM's call for expansion of undergraduate public health education to address two priority needs; (1) a serious disparity between the number of graduates produced by schools of public health and the number of workers needed (i.e., a workforce shortage), and (2) the need for a large, well-educated public health workforce that is able to respond effectively to emerging trends that impact population health, such as globalization, urbanization, population aging, health disparities, and alterations to the US health care system (3).

This review seeks to examine student perceptions of the high-impact educational practices (HIPs) utilized by Georgia State University faculty in the BSPH program, and the extent to which students see these HIPs as preparing them to use insights gained in the classroom and from the field, to question, modify, connect, and integrate academic material to real-life public health challenges. It also seeks to examine HIPs that have the greatest impact.

### HIPS AND STUDENT LEARNING

Students pursue a college education for several reasons, prime among them being the need for financial well-being and the ability to engage in leisure activities (4). As colleges and universities prepare students for their future careers, and work toward attracting the next set of potential students, they are faced with identifying educational practices that will prepare students for a dynamic and competitive workforce (5). These institutions are also faced with the question of whether their undergraduate students are academically engaged and learning enough in college, and whether their students are being taught foundational skills effectively (6).

Several studies have established a significant association between good educational practices and student college outcomes. Chickering and Gamsons assert that contact between students and faculty, reciprocity and cooperation among students, active learning, and prompt feedback are practices that can aid this process (7). These practices have been vetted, and found to positively influence undergraduate student learning and growth (8, 9) put together, these practices comprise high impact learning practices.

HIPs have been found to contribute to student increased academic engagement, knowledge, resolve, and general academic success (10). These educational practices afford students the opportunity to participate in activities beyond the classroom over a period of time, resulting in learning and personal development (11). HIPs not only enable students to apply what they have learned, or make meaning of their learning, they also contribute to metacognitive gains by students (12). In his article on HIPs, Kuh lists writing intensive courses, collaborative assignments and projects, undergraduate research, service learning, communitybased learning, and<underline >diversity/global learning as practices that help to increase student retention and learning (13). According to Kuh, the implementation of HIPs among other things, has the tendency to increase student interaction with faculty and their peers on course work, and to induce reflection on course material and integrated learning (14). Engaging students in activities that focus on "learning by doing" has the tendency to make classroom learning real and relevant through the application of new knowledge to real life settings (15). Kinzie (16) noted that faculty engagement of students in projects in and outside the class increases student learning. Indeed project-based learning integrates knowing and doing. Through this effort, students not only learn elements of the core curriculum, but also apply what they know to solve authentic problems and produce results that matter (16). Students exposed to good educational practices tend to obtain better grades, and have increased cognitive, emotional, and personal growth. They also tend to be more satisfied with their college experience (17).

### MATERIALS AND METHODS

### Setting and Population

The BSPH program has over 400 racially and economically diverse students enrolled in the public health major, and caters to both traditional and non-traditional students. The program is a 4-year degree that places special emphasis on urban and global health issues, and has a curriculum that focuses on elements of life and biological sciences, social sciences, and the humanities. The goal of the program is to provide students with an interdisciplinary understanding of public health, using a broad spectrum of approaches and course work. Students enrolled in the program acquire knowledge and skills needed for graduate school, and for careers in a wide range of public health and interdisciplinary professions.

### Sampling and Data Collection

We used a cross-sectional convenience sample of current and graduated BSPH students to assess student perceptions of the extent to which six HIPs implemented by Georgia State University's BSPH program faculty, impact undergraduate student cumulative learning, academic success, and career outcomes. Undergraduate course work and assignments, service learning/community-based learning, collaborative projects, study abroad, undergraduate research, and signature experience were the HIPs examined.

Students in the BSPH program were informed about the study and its purpose, through faculty announcements, emails, the BSPH general learning management system (iCollege), and were invited to take the survey. Completed survey questionnaires were collected on a daily basis using Qualtrics. All students associated with the program were given an equal opportunity to participate in the study.

To assess the impact of BSPH HIPs, data was collected on the amount of time students purposely devoted to course work and assignments, the extent of student interaction with peers and faculty on course work, and student opportunities to receive guidance and on-going feedback from faculty on their course work and assignments. Data was also collected on student's confidence in their ability to apply public health knowledge situations both in and outside the classroom.

The study was exempt from the Georgia State University IRB review process because, it was conducted in an established or commonly accepted educational setting that specifically involved normal educational practices that did not adversely affect students' opportunity to learn required educational content. The study was further exempt from IRB approval because it sought to research the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods.

#### Variables and Measurement

The HIPs survey comprised a 23-item questionnaire containing closed-ended questions on a six-point Likert scale ranging from "strongly disagree" to "strongly agree." The questionnaire was pilot-tested with randomly selected BSPH students for accuracy and consistency of measures. The questions assessed HIPs on six domains; (1) undergraduate course work and assignments, (2) service learning/community-based learning, (3) collaborative projects, (4) study abroad, (5) undergraduate research, and (6) signature experience. The survey questionnaire also asked for demographic and academic information.

The demographic and academic variables of the survey included academic status ranging from recent graduate to freshman, and the sex of students. The undergraduate course work and assignments variable focused on the amount of out of class time students devoted to course work and assignments, the availability of faculty to provide continuous feedback on course work and assignments, and student confidence in their ability to apply course material to real-life public health situations both in and outside the classroom. The service learning and communitybased learning variable focused on career and occupational skills development, preparation for professional employment, and the application of classroom knowledge to service and community-based learning activities. The collaborative projects variable focused on working with peers, enhanced understanding of course material, and integrating and merging of knowledge from other courses with team members to produce a high quality academic product. The study abroad variable focused



on decisions for postgraduate education and the acquisition of additional skills for a future career. The undergraduate research variable focused on student ability to research public health issues and their ability to present at conferences, and the final variable, signature experience, focused on the ability of students to create faculty reviewed academic products that were pertinent to their experience.

#### Statistical Analysis

The data collected in Qualtrics were cleaned and exported to SPSS for analysis. Univariate analysis was conducted to summarize and describe data. Missing data were excluded from calculations. Univariate, bivariate, and multivariate data analyses were conducted using SAS 9.4 and Stata 11. Univariate analysis was conducted to obtain descriptive statistics for the six HIPs domains, sex, and educational status. Bivariate data analysis was conducted to determine the relationship between academic status, sex, and the six domains. At the multivariate level, we conducted an ordinal logistic regression to determine whether there was a relationship between academic status and the six HIPs domains and adjusted for sex. The 95% confidence interval was used to provide an estimated range of values for each of the HIPs variables measured.

### RESULTS

### Univariate Statistics

#### Demographic and Academic Status

All 436 students currently enrolled in the BSPH program and the 19 students who have graduated from the program were invited to participate in the survey. One hundred and five students currently enrolled in the program and nine students who have graduated from the program, completed the survey. The majority of students (93.8%) are still enrolled in the program, and 88% of the respondents self-identified as female. **Table 1** shows the academic level of the students who participated in the survey and **Table 2** shows the number of students who responded to each of the HIPs domains by academic level.

#### Undergraduate Course Work and Assignments

Regarding undergraduate course work and assignments, 58% of students said they devoted a minimum of 3 h per day to their course work and assignments, 52% said they met with BSPH faculty outside of class for guidance on course work and assignments, and 66% said that they received feedback from BSPH faculty on their course work and assignments. Over half


of the students surveyed (58%) said that they were confident that they could apply material from various courses offered by the program to public health issues both in and outside the classroom. Undergraduate BSPH courses include Health Equity and Disparities and Introduction to Public Health.

#### Service Learning/Community-Based Learning

Forty-eight percentage of students in the BSPH program said service learning and community-based learning was an important part of their BSPH experience, and 49% said the experience helped them to develop career and occupational skills. Fifty-two percentage of students indicated that they were able to apply their classroom knowledge to the service and community-based activities they engaged in. Fifty-six and 54% of students, respectively, said that their participation in service and community-based activities gave them a sense of personal achievement and prepared them for professional employment opportunities. Forty-seven percentage of the students surveyed disclosed that they were able to integrate their classroom work into their service and community-based activities.

#### Collaborative Projects

On the issue of collaborative projects, 45% of students stated that working with peers was an important part of their BSPH experience, while 8% did not think so. Forty percentage of students stated that collaborative projects helped them to better understand course material. Less than half of the students surveyed (40%) said group projects inspired them to do their best on assignments, and 42% of students indicated that group projects allowed them to integrate the ideas and knowledge they acquired from various program courses, with that of their peers to create good academic products (papers, projects, and presentations).

#### Study Abroad

A little more than a third (36%) of students reported that their study abroad experience motivated them to consider pursuing graduate studies, and allowed them to acquire additional skill sets for their future careers. Less than half (39%) of the students neither agreed nor disagreed that it influenced their decision to enter the public health workforce or pursue graduate education.

#### Undergraduate Research

Undergraduate research is one of the most requested opportunities from the student body. Forty-two percentage of the respondents took a course that required them to do some research. As a result, 43% of these students were able to use their research skills to develop oral and poster presentations delivered both on and off Georgia State University's campus.

#### Signature Experience

Given that the BSPH program is only in its third year of development, and the signature experience courses are taken when the predominance of the major coursework is completed, most of the respondents in the program have not taken the signature experience course yet. Of those who have, 42% stated that they were able to integrate and apply what they had learned in the classroom to their signature experience, and 42% said that enrolling in the course exposed them to experiences they would otherwise not have gained in the classroom. Forty-two percentage of students who took the signature experience course indicated that, with guidance from faculty members, they were able to produce good final academic products.

#### Bivariate and Multivariate Analysis

We conducted bivariate analysis for sex, academic status (including BSPH graduates, seniors and juniors) by all the six HIPs domains. We found no statistically significant relationship between sex and the six HIPs domains, but we found statistically significant relationships between BSPH graduates and juniors for collaborative projects (p = 0.0357) and signature experience (p = 0.0085) with regards to the question, I am/was able to apply what I learned in the classroom to my signature experience. When it came to BSPH graduates and seniors, we also found statistically significant relationships for collaborative projects (p = 0.0456) regarding the question, Working on group projects collaboratively helps/helped me to integrate my ideas with those of others in the group to produce a good product; for study abroad (p = 0.0331) with regards to the question, Participating in the BSPH study abroad program allowed me to acquire additional skill sets for my future career and signature experience; and for signature experience (p = 0.0379) in relation to the question, I was able to apply my undergraduate research skills by presenting at a conference. Concerning seniors and juniors, we found statistically significant relationships for undergraduate course work and assignments (p = 0.0065) regarding the question, Faculty in the BSPH program provide/provided feedback on my coursework and assignments; for service learning/communitybased learning (p = 0.0464) with respect to the questions

TABLE 3 | Ordinal logistic regression model of respondents' domain by educational levels—adjusting for sex.


The BSPH service-learning experience will help/helped with the development of career/occupational skills and I am/was able to apply classroom knowledge to my service learning activities; and for collaborative projects (p = 0.0324) concerning the questions, Collaborative group projects helps /have helped me to better understand course material and "Working collaboratively on group projects inspires /inspired me to do my best in an assignment., undergraduate research. Concerning seniors and juniors, we also found statistically significant relationships for undergraduate research (p = 0.0224) regarding the question, I participated/was able to participate in undergraduate research opportunities as a requirement of a class that I took in the BSPH program; and for signature experience (p = 0.0448) in relation to the question, The BSPH signature experience caused me to produce a final product that was evaluated by a faculty member.

To examine the relationship between student educational levels in the BSPH program and the six HIPs domains, we performed multivariate analysis and adjusted for sex (using Stata 11).

**Table 3** shows the overall outcomes from the ordinal logistic regression model for each of the six HIPs domains. Statistically significant relationships were found between all of the HIP domains and student academic level. Service learning/community-based learning had the strongest impact in our ordinal logistic model with an odds ratio of 7.117. This domain had more than seven times impact, compared with the other domains, which were all protective (having odds ratios <1). It is therefore likely that more students benefitted from the service learning/community-based learning than the other HIPs domains.

### DISCUSSION

Results from the study provide baseline data of the effectiveness of HIPs utilized by the BSPH faculty to educate and prepare students for real-life public health challenges. The results also indicate that students perceive the HIPs implemented in the program as effective in impacting their learning and preparing them for real-life public health challenges.

### Undergraduate Course Work and Assignments

Our study revealed that most students in the BSPH program (58%) devote more than 3 h per week per semester on undergraduate course work and assignments. This is consistent with the National Survey of Student Engagement's (NSSE) findings, that the average student spends about 17 h per week on course work including homework, reading, and assignments. For the majority of students responding to the survey, interaction with faculty outside classes provided additional guidance on how to do their coursework and assignments. Previous research indicates that student-faculty interactions can have several positive influences. Indeed, Pascarella and Terenzini found that student-faculty interaction generally have a positive influence on the cognitive growth and development of college students (9). They also found that student-faculty interaction is positively related to students' academic achievement and that the frequency of contact is related to students' positive learning outcomes (9).

### Service

### Learning/Community-Based Projects

Service learning affords students the opportunity to apply knowledge gained in the classroom to practical communitybased projects. It also allows students to make real-life connections between what they learn in the classroom and what actually happens in practice. Close to half (48%) of the students who responded to our survey found service learning/communitybased learning to be beneficial. This statistic is modest and may be because of the infancy of the BSPH program and the fact that not many students are at a point where they can engage in the practice. We found that participating in service learning activities was a catalyst to student development of skills for future employment, increased personal insight, cognitive and social development, and created a sense of personal and academic achievement. This is consistent with earlier research findings by Austin et al. (18), Yoiro et al. (19), and Conway et al. (20).

### Collaborative Undergraduate Projects

The goal of collaborative undergraduate projects is to help students learn to work and solve problems in the company of others, and to sharpen their understanding by listening seriously to the insights of others, especially those with different backgrounds and life experiences. Unfortunately, not as many BSPH students (40%) found this HIP to be helpful in increasing their understanding of course material. These results are consistent with the findings of a previous study conducted by Premo et al. which found that, collaborative projects by themselves are not enough to promote increased student achievement (21). Group projects sometimes end up becoming a source of friction between students. At their best, collaborative projects foster productive team and idea sharing among future professionals, while at their worst, team projects force high-achieving students to compensate for those less willing to put in the effort. It may be because of the latter reason that this HIP was not as popular with BSPH students who responded to our survey (22).

### Study Abroad

Brazil, The Dominican Republic, China, India, and Uganda are the five countries where students in the BSPH program have had the opportunity to visit and explore different cultures, worldviews, and life experiences. Led by full-time faculty from the SPH, as well as faculty from geosciences, criminal justice and communications, students who took the survey indicated that studying abroad enabled them to acquire additional skills sets such as critical thinking, problem solving, communication, leadership, professionalism, and intercultural fluency that they need for their future careers. These competencies are consistent with the National Association of College and Employers (NACE) key career competencies (23). All BSPH programs are designed to prepare students to be informed and exposed to public health issues from an urban and global perspective. Through mentoring, a challenging overseas academic program, and hands on experiential activities, students are positioned to gain invaluable knowledge and skills that augment their academic preparation. Consistent with Sanchez's (24) previous research finding, we found that the study abroad experience by itself is an example of several HIPs wrapped into one academic experience that lasts for a lifetime, transforms student intellectual perspectives and personal growth, and causes students to rethink their majors and incorporate further studies abroad into their academic schedules (24).

### Undergraduate Research

Even though the number of students who indicated that they were able to participate in undergraduate research was <50% of those who responded, 43% reported that their understanding of how to conduct research, increased their ability to write papers and to present at conferences, both on and off campus. This is consistent with Lopatto's (25) findings on undergraduate research and HIPs. He found that the undergraduate research experience affected student career plans and sometimes helped them to fine-tune their career plans (25). We found evidence of a connection between undergraduate research experiences, and personal academic growth and confidence in foundational research methods, and increased interest in graduate education and preparation for the labor force. Lin et al. in a previous study confirm that working with faculty is an important component of a successful undergraduate research experience (26).

There is no other extracurricular activity that undergraduate students at Georgia State University request more, than doing research with faculty. Students focused on continuing their education beyond the baccalaureate degree, know that learning how to conduct public health research with experienced faculty can enhance their graduate school applications considerably. Thus, in the 3 years that the BSPH program has been in operation, opportunities for students to participate in undergraduate research has continued to increase. In the spring of 2017, just one semester after the program was inaugurated, a faculty member agreed to work with two undergraduate students on a funded research project to examine the impact of mindfulness on tobacco cessation in low-income communities. Since that spring, in each semester, additional faculty have worked with more and more undergraduate students in the program. As of the fall 2018 semester, four faculty members are working with more than 10 students on research projects related to health policy, water quality, nano-particles, and health promotion and behavior. Additionally, the lead author of this paper has started a research club to work with students to develop manuscripts based on primary and secondary data analysis and the systematic reviews of the literature. Within a week of the faculty member announcing the launching of the club, over 20 students had signed up.

### Signature Experience

All students at Georgia State University are encouraged to complete at least one signature experience. Utilizing experiential strategies, signature experience courses give students the opportunity to apply course content to real-life situations. While signature experience courses are highly encouraged across the university, they are not a required course in any program of study except in the BSPH program. In this program, students entering their senior year of study register for PH 4991 (Signature Experience 1–Prospectus) and PH 4992 (Signature Experience– Capstone). Both courses are offered every semester in a 7-week mini-mester, so both courses can be taken in one semester. This increases the odds for students to progress to graduation in any given semester. The prospectus course (4991) is the first of a two-course sequence required to meet the area H requirements of the BSPH program of study. Students have the opportunity to integrate, synthesize and apply their public health knowledge through cumulative and experiential activities. In this course, students complete a variety of projects, and written assignments designed to assess student acquisition of the required public health competencies covered within the public health major. Mastery of these competencies is required to receive the BSPH degree.

Forty-two percentage of students indicated that participation in the signature experience course exposed them to things that typically do not occur in a traditional classroom, and that they were able to integrate what they learned in the classroom into the course, and were able to produce quality, culminating artifacts that were evaluated by their faculty. This finding is consistent with that of Fitzpatrick et al. (27) who assessed undergraduate public health capstone courses in 2016. In their study, they found that capstone projects caused students to integrate what they had learned from various courses, into their capstone experience and that, their success was most evident in their ability to transform draft literature reviews into a final product (27).

### LIMITATIONS

While the results from the assessment are promising, it would have been good to have more BSPH graduates complete the survey. To address this challenge, faculty will ensure to create awareness among BSPH students of the importance of participating in such surveys in the future.

### CONCLUSION

The study assessed six HIPs utilized by Georgia State University BSPH faculty and the extent to which integrative educational practices prepare students for the labor force and for further education. It also sought to identify integrative educational practices that have the highest impact of helping students integrate knowledge and skills gained in the classroom to public health issues. Results from the study provide invaluable baseline data for the assessment of future integrative educational HIPs utilized by Georgia State University faculty in the BSPH program.

### REFERENCES


### AUTHOR CONTRIBUTIONS

EAr-M and KR-W wrote the draft manuscript, finalized, and edited the final manuscript. EAl-M performed the statistical analysis. All authors reviewed and approved the final manuscript.

### ACKNOWLEDGMENTS

We thank Mrs. Souvann St. John-Brown for her assistance with data collection and survey follow-up activities. We also thank faculty in the BSPH program at Georgia State University who, continue to work with and inspire the students in our program and for their help with announcing the survey and encouraging students to participate.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Armstrong-Mensah, Ramsey-White and Alema-Mensah. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.