A systemic transformation of an arts and sciences curriculum to nurture inclusive excellence of all students through course-based research experiences

Abstract

Introduction:

We describe herein a large-scale, multidisciplinary course-based undergraduate research experience program (CRE) developed at Lawrence Technological University (LTU). In our program, all students enrolled in CRE classes participate in authentic research experiences within the framework of the curriculum, eliminating self-selection processes and other barriers to traditional extracurricular research experiences.

Methods:

Since 2014, we have designed and implemented more than 40 CRE courses in our College of Arts and Sciences involving more than 30 instructors from computer science, mathematics, physics, biology, chemistry, English composition, literature, philosophy, media communication, nursing, and psychology.

Results:

Assessment survey data indicates that students who participate in CRE courses have an enhanced attitude towards research and discovery, as well as increased self-efficacy. This intervention is particularly relevant for non-traditional students, such as students who commute and/or have significant work or childcare commitments, who often experience limited access to research activities.

Discussion:

Herein we highlight the importance of a systemic institutional change that has made this intervention sustainable and likely to outlast the external funding phase. Systemic change can emerge from a combination of conditions, including: (1) developing a critical mass of CRE courses by providing instructors with both incentives and training; (2) developing general principles on which instructors can base their CRE activities; (3) securing and maintaining institutional support to promote policy changes towards a more inclusive institution; and (4) diversifying the range of the intervention, both in terms of initiatives and disciplines involved.

Introduction

This article describes the incorporation of course-based research experiences (CRE) in a large number of courses across diverse disciplines within the College of Arts and Sciences at Lawrence Technological University, a primarily undergraduate private institution in the Detroit, MI metropolitan area. The scope of integrating CRE in the Arts and Sciences curriculum was to promote inclusivity and guarantee accessibility to research activities to all categories of students, especially those historically underrepresented in STEM education and the STEM workforce in the United States.

Undergraduate research experience in the United States

Undergraduate research experiences (UREs) are a well-known pedagogical strategy for attracting and retaining students in science, technology, engineering, and mathematics (STEM). Specifically, UREs improve students’ learning experience (Lopatto, 2004, 2007; Seymour et al., 2004; Kinkel and Henke, 2006; Hunter et al., 2007), increase student interest and success in postgraduate studies and careers in science (Hathaway et al., 2002; Lopatto, 2004; Hunter et al., 2007; Russell et al., 2007), enhance examination performance (Barlow and Villarejo, 2004; Freeman et al., 2014; Ward et al., 2014), and promote retention in STEM disciplines (Nagda et al., 1998; Barlow and Villarejo, 2004; Braxton et al., 2004; Kinkel and Henke, 2006; Summers and Hrabowski, 2006; Gilmer, 2007; Carter et al., 2009; Olson and Riordan, 2012). Importantly, UREs are also an effective strategy to improve the recruitment and retention of underrepresented minority students in STEM disciplines (Barlow and Villarejo, 2004; Summers and Hrabowski, 2006; Tsui, 2007; Villarejo et al., 2008; CUGESEWP, 2011). Traditionally, students participate in UREs by individually joining a research group and working closely with a faculty member on an assigned research topic. Such an approach, supported, for example, by the National Science Foundation Research Experiences for Undergraduates (REU) program and by the National Institutes of Health Research Training Initiative for Student Enhancement (RISE) program, has been broadly adopted in the United States. While there is evidence that these single-student examples of the UREs may be especially beneficial for underrepresented students (Rorrer et al., 2018), there are significant limitations, including: (1) Scalability: funding, faculty member availability, and the relatively small number of students that can benefit from this URE model (Beninson et al., 2011; Ramirez et al., 2015); (2) Time of intervention: first-year students rarely benefit from this URE model because students are typically selected for these summer experiences as rising juniors or seniors; (3) Inclusion: underrepresented minority students may have limited exposure to URE programs due to college selection, geographical limitations, and other barriers to engagement in URE (Walpole, 2003).

Course-based research experiences

In recent years, the concept of course-based research experiences (CRE, also known as course-based undergraduate research experiences, CURE) has grown in popularity in institutions of higher education in the United States. In this article, we use the acronym CRE instead of CURE because the intervention described herein includes several graduate courses, while CURE specifically refers to undergraduate curricular interventions. CRE consists of embedding authentic research experiences within regular class activities. This pedagogical approach has been applied to a variety of disciplines, including general education, STEM, and even music education courses (Boomer et al., 2002; Elwess and Latourelle, 2004; Brodl, 2005; Howard and Miskowski, 2005; Hanauer et al., 2006; Hatfull et al., 2006; Drew and Triplett, 2008; Lopatto et al., 2008; Caruso et al., 2009; Ronsheim et al., 2009; Shaffer et al., 2010; Harrison et al., 2011; Corwin et al., 2015; Dvorak and Hernandez-Ruiz, 2019; Hernandez-Ruiz and Dvorak, 2020). CRE courses overcome some limitations of the individualized approach to UREs described above, including: (1) Scalability: CRE facilitates the simultaneous exposure of a higher number of students to research because it is embedded in a course; (2) time of intervention: CRE interventions can be offered early and often throughout a student’s academic journey, often impacting student success starting in the freshman year; (3) inclusion: CRE provides more opportunities for underrepresented minority students to engage in research experiences because this intervention eliminates self-selection.

In CRE interventions, the research activity is part of a regular course, meaning that all enrolled students are included in the intervention. Moreover, research activities can be conducted during regular classroom hours, and can be designed to include research endeavors conducted at home in some cases. Hence, CRE does not require that students spend long hours engaged in research activities outside of the class, making it ideal for commuter students, working students, or students with childcare or other significant non-academic responsibilities.

A notable example of CRE is the SEA-PHAGES program,¹ a large-scale multi-institutional program supported by the Howard Hughes Medical Institute (HHMI) that provides authentic biology research experiences to undergraduate students at diverse institutions at an early stage in their degree programs, leading them to experience discoveries and participate in student-authored publications (Caruso et al., 2009; Asai, 2013; Jordan et al., 2014; Pope and Hatfull, 2015). The parallel involvement of a multitude of institutions in a streamlined research paradigm facilitates a very effective and well-organized research experience in which students are more likely to publish their results because the protocols are well established and the possibility of discovery is higher. In contrast to the discipline-specific approach of the SEA-PHAGES program, we have implemented an inclusive initiative to promote and expand course-based research experiences across all disciplines within our diverse College of Arts and Sciences (Shamir et al., 2019).

Active learning environments, such as CRE programs, achieve positive academic outcomes (Lopatto, 2007; Russell et al., 2007; Shaffer et al., 2014) through multiple elements that impact academic belonging, which has been defined as “the extent to which individuals feel like a valued, accepted and a legitimate member in their academic domain” (Lewis et al., 2016). Numerous studies have shown the beneficial impact of CRE on student diversity and equitable access, including increased positive attitudes towards research (Osborne et al., 2003; Harrison et al., 2011; Brabec et al., 2018), enhanced self-efficacy (Chemers et al., 2011; Auchincloss et al., 2014; Carpi et al., 2017; Martin et al., 2021), the development of teamwork skills (Kapp, 2009; Hanauer and Hatfull, 2015; Dewey et al., 2022), and increased cultural sensitivity by introducing students of varying backgrounds to research regardless of their race or gender (Micari et al., 2007; Bangera and Brownell, 2014; Collins et al., 2019). The student experience survey employed to help measure the impact of our intervention includes an assessment of the association between these factors and participation in CRE courses.

CRE at LTU: a description of the initiative

A multidisciplinary intervention

Beginning in 2014, we began pursuing the goal of incorporating research experiences into our curriculum by transforming several traditional courses into CRE courses. With the support of internal and external funding, we designed a sustainable framework that has involved the transformation of dozens of diverse courses in the College of Arts and Sciences at LTU in addition to adopting strategies to ensure the long-term sustainability of the intervention (Shamir et al., 2019; Weinstein et al., 2019; Delogu, 2020).

Our initiative succeeded in promoting an enhanced research culture in our college, creating excitement around including students in research endeavors. While we differ in our involvement of many non-traditional CRE disciplines in a single coordinated program, the focus of our CRE intervention is well aligned with other CRE initiatives in the United States, with our goals being to: (1) Improve students’ persistence and success in STEM degree programs, (2) Make research accessible to a larger and more diverse group of students, (3) Positively influence students’ educational and career trajectories, and (4) Encourage students to pursue graduate education and research-related career paths (Dolan, 2016). Many CRE activities are often limited to the personal initiative of informed instructors and administrators, and rarely take the shape of a broad-scale institutional intervention involving many courses, disciplines, and departments. In contrast, we achieved a broad-scale implementation of CRE with the ultimate goal of promoting a systemic multidisciplinary intervention capable of changing the pedagogical vision of an entire college.

The most distinctive aspects of our program are the large scale and the heterogeneity of the CRE implementation. To date, our initiative has developed more than 40 CRE courses involving over 30 instructors in all disciplines within the College of Arts and Sciences, including biology, chemistry, physics, mathematics, computer science, psychology, art, literature, English composition, philosophy, and nursing. A typical CRE course at LTU includes research practices embedded in the regular coursework, in which students investigate questions with unknown answers using leading-edge practices and methods specific to the discipline. CRE practices promote student agency and ownership of the discovery process. The time and resources allocated to CRE in a course vary according to the discipline and specific course. Our intervention includes courses exclusively devoted to CRE for the entirety of the semester, courses substantially devoted to CRE in which research activities are conducted during at least the 50% of the classes, and sometimes courses in which a CRE module takes only one class period.

Conceptual model: framework, principles, and related initiatives

A CRE-related paradigm shift at LTU is being institutionalized as more faculty participate, and more research experiences are created. The number of faculty participating in CRE has grown from 22 in spring 2017 to 32 in spring 2022. During this time frame, the number of CRE courses has also grown steadily. Recognizing that our program is large and heterogeneous, and respecting the specificity of intervention in each field of investigation, we developed a set of principles that we call “Pillars of CRE at LTU” to which every CRE course adheres to. The process used to create this set of standards and theoretical principles was iterative and dynamic during the initial years of intervention. These “pillars” helped to establish and clarify the structure of CRE courses at LTU to help ensure authenticity and sustainability.

Herein we present the three fundamental principles of CRE at LTU, which constitute the “pillars” on which we base our intervention in every course (see Figure 1).

Figure 1

In support of the CRE pedagogical transformation, we developed the following related initiatives to foster a culture of institutional collaboration and sustainability:

• Culturally-responsive teaching: CRE courses at LTU promote the inclusion of all students, especially those who have been historically underrepresented in STEM fields. Moreover, cultural sensitivity is explicitly addressed as a topic in many courses by emphasizing how cultural, social, and economic differences have shaped and continue to shape the process of scholarly discovery. Students are encouraged to critically examine current practices in their selected professions from inclusivity perspectives. Whenever possible, student interests are integrated in the research activities.

• A CRE teaching and learning community: A core feature of the intervention is the creation of a cohesive community of faculty and administrators who are champions and active participants in the project. All community members are encouraged to participate in training events, journal clubs, regular meetings, and joint CRE-related dissemination initiatives, such as conference presentations and writing manuscripts for peer-reviewed publications.

• High school and community college dual enrollment programs: There are very few reports of CRE interventions in high schools and two-year colleges (Dolan, 2016). In order to broaden accessibility to CREs, we developed a network of collaborations and agreements with high schools and community colleges in metro Detroit. Within this network, LTU professors teach CRE courses and students from high schools and community colleges can attend CRE courses at LTU. CRE faculty also organize activities such as workshops, seminars, summer camps, and guest lectures to expose high school students to CRE-focused activities. Such initiatives take place both at LTU and at host institutions.

• Multi-institution collaborations: The grants that have supported LTU’s CRE initiatives over the years strongly encouraged multi-institution collaborations. For example, peer implementation clusters (PICs) are communities of HHMI-funded institutions linked by regional proximity that are encouraged to collaborate and share ideas on topics related to inclusive excellence. Sabbatical exchanges have also been used to start collaborations with other institutions interested in using research to broaden participation and success in STEM degrees.

• Seminar series on topics of diversity, equity, and inclusion: The LTU CRE community uses an “Idea Factory” seminar series² as an opportunity to discuss topics related to student research, diversity, inclusivity, and equity in academia and society. Seminars are open to the general public.

• Substantial involvement of non-STEM disciplines: Our intervention includes redesigning courses in non-STEM disciplines such as literature, English composition, art, philosophy, communication, and psychology. CRE projects in the humanities and social sciences often include cross-disciplinary bridges between methods, approaches, and bodies of knowledge on a specific subject of investigation, encouraging interdisciplinary collaborations. This inclusive multidisciplinary/interdisciplinary aspect of our intervention is distinctive, as CRE programs are often designed for, and target, STEM fields exclusively.

• Extracurricular CRE student researcher awards: CRE projects may be continued outside the classroom environment through our student researcher award program. This initiative allows students to help faculty members develop, test, and/or refine a protocol that will be implemented in a CRE course or may help a faculty member to finish or analyze the results of a CRE module that has already been implemented in a CRE course. Student researcher awards have increased the participation of students in research by providing funding for student stipends as well as a classroom platform to test ideas. This program also includes funding for undergraduate student travel to facilitate the dissemination of CRE projects to a broader audience beyond our campus community. More than three dozen student researcher awards have been awarded across a diverse array of disciplines over the five-year grant period.

CRE disciplines, course structure, and content at LTU

All CRE courses promote an inclusive learning environment in which the instructor facilitates a culturally sensitive classroom environment. CRE also promotes collaboration on several levels: between faculty and students, among and within teams of students, and, whenever possible, between the course and other communities (e.g., the campus, the city, similar courses at other universities, etc.). Whenever possible, student interests and personal initiatives are encouraged in the development of a question, problem, and/or experimental design. Scholarly discoveries are often reported to an audience beyond the course, such as an external community (a scientific article or conference presentation) or the campus community (presentation or written report at a university-wide, college, or department research event).

Figure 2 presents an example of a general overview of CRE activities within a CRE course at LTU. CRE courses are usually organized into four modules.

Figure 2

During module 1 (weeks 1–2), students explore questions and problems with unknown answers. In this first module, instructors also provide training about the importance of diversity, equity, and inclusion (DEI) in education and research in general as well as how research in the particular discipline and topic of the course relates to DEI. During module 2 (weeks 3–4), students acquire methodological awareness of current techniques and methods in the field. During module 3 (weeks 5–12), students engage in empirical research and generate results. Finally, during module 4 (weeks 13–16), students write a final report and disseminate results as term papers, posters, and oral presentations in class and also sometimes at internal or external conferences. This general structure and the time dedicated to each block of activities can substantially vary among instructors, disciplines, and research topics. Some instructors, for example, dedicate the entire semester to CRE while others blend CRE modules with traditional lectures.

In Table 1, we list examples of how CRE was implemented at LTU in many disciplines from 2014 to 2023. Since the program is currently active and new CRE projects are developed and implemented every semester, this list is necessarily incomplete. Thus, its purpose is merely to provide examples to instructors in specific fields interested in CRE implementation. Details on course implementation, research topics, and timelines of CRE activities are provided in Appendix 1.

The following sections include an assessment of the impact of the CRE Inclusive Excellence program on student success, student experience, and the faculty perception of the sustainability of the intervention at LTU. Our analysis includes institutional data about student academic achievement, survey data addressing students’ experience, and survey data from faculty about the sustainability of the CRE intervention. We used a mixed-methods approach involving both qualitative and quantitative methods of analysis.

Methods: assessment of CRE impact

We have been assessing the outcomes of our intervention since we began implementing CRE in 2016. Our evolving assessment efforts include measures of academic achievement, as well as self-reported data. In this study, we report the initial assessment of our first large-scale intervention (2016–2017), supported by the American Association of Colleges and Universities (AAC&U) Teaching to Increase Diversity and Equity in STEM (TIDES) program and an HHMI Inclusive Excellence grant.

The primary question we aimed to assess was the impact of CRE pedagogy at LTU as it relates to student success, the student experience, and the faculty perception of the sustainability of the intervention. We collected institutional data related to student academic achievement, survey data related to students’ experience, and survey data from faculty related to the sustainability of the CRE intervention.

Participants

All students enrolled in a CRE course in the fall 2016 or spring 2017 semesters were invited to complete a pre-post survey. A total of 484 CRE students completed at least one of the two surveys. After matching pre- and post-course survey responses, a total of 372 surveys (two per student, one before and one after the CRE course) were included in the final analysis. The final sample of CRE students was 186 students (110 male, 59%; 73 female, 39%; 3 undeclared, 2%). Thirty-seven students (20%) were freshman, 40 (21%) were sophomores, 37 (20%) were juniors, and 72 (39%) were seniors.

Institutional data regarding academic achievement

We gathered institutional data on participants to compare the academic achievement of CRE students (N = 186) and the general population of all College of Arts and Sciences students during the fall 2016 and spring 2017 semesters (N = 623). Academic achievement was assessed by comparing the yearly grade point average (GPA) and final grades of students enrolled in CRE courses with the general population of all students enrolled in any course within the College of Arts and Sciences. Independent sample T-tests were conducted to compare the mean GPA and final grades of CRE students compared to all College of Arts and Sciences students.

Survey regarding students’ experience

We developed an original survey to assess several aspects of the CRE experience of students in different disciplines. We used a new questionnaire instead of an existing one because previous instruments used to assess CURE (Lopatto, 2004) are specifically designed to assess STEM courses, while our CRE intervention includes both STEM and non-STEM courses. The survey included 15 items scored along a five-point Likert scale (see Appendix 2), as well as six demographic questions. The 15 questions focused on six main topics: attitude towards research, self-efficacy, teamwork, cultural sensitivity, gender issues, and race issues. Responses related to the same topic were collapsed and averaged together. A series of 6 ANOVA analyses, one for each topic, were conducted with the academic discipline (biology and chemistry, literature, mathematics and physics, philosophy, psychology) and gender (female vs. male) as between-subject factors and CRE experience (pre- vs. post-course) as within-subject factors. We could not include race as a factor in the main analysis because the low number of non-white students in the sample (12 African Americans, 17 Asians, 9 Hispanics, 2 Biracial versus 146 Whites) prevented the inclusion of the race factor in the main design. A separate series of ANOVA analyses were conducted to evaluate the influence of race on the six dependent measures. In this supplementary analysis, the between-subject factor race was paired with the within-subject factor CRE-experience in order to verify the influence of race on the CRE intervention.

Survey regarding faculty perspectives on the sustainability of CRE at LTU

Thirty faculty currently involved in our CRE pedagogical initiative were asked to complete a brief survey during a faculty CRE retreat in August 2021. The purpose of the survey was to obtain self-reported measures of how likely the CRE faculty team was willing to use course-based research experience pedagogy in the future, in absence of monetary rewards. To maintain anonymity, we avoided asking questions that could be used to trace the identity of the respondents. From the demographic data of attendance at the retreat, we can report that the sample was composed of a total of 30 participants, including 13 female faculty (43%), 2 adjunct faculty (0.06%), 7 non-tenure-track full-time faculty (23%), 11 tenure-track assistant professors (36%), and 10 tenured professors (33%) with 24 faculty from STEM fields (80%) and 6 from non-STEM disciplines (20%).

The survey included the following main question: “How likely are you to use course-based research experience pedagogy in the future, even if you will not be paid for any future CRE activities?” followed by three additional questions to assess faculty opinions on the sustainability of the three pillars of CRE:

How likely are you to use course-based research experience in the future in order to:

• Promote original discovery in the classroom

• Require inclusive collaboration

• Facilitate communication of course discoveries

To verify possible associations between the amount of CRE experience and the willingness of faculty to continue using CRE pedagogy, we asked participants to report how many distinct CRE courses and sections of courses they had taught at LTU. Finally, we asked participants to share any comments, ideas, and challenges about the use of CRE in their future pedagogical plans.

Results

Academic achievement

Independent sample T-tests were conducted to compare the grade point average (GPA) of CRE students to the GPA of students in the general student population in the College of Arts and Sciences during the targeted semesters. We also compared the final grades of students enrolled in CRE courses to the final grades of all students enrolled in courses in the College of Arts and Sciences. Prior to the analysis, the normality of GPA and final grades in CRE and all students was estimated using skewness and kurtosis. The criteria for the normality of the academic achievement data were met as skewness and kurtosis <2 (Asai, 2013). Accordingly, GPA and final grades were assumed to be normally distributed.

Independent sample T-test results on student GPAs (T = 2.68, df = 804, p = 0.007, Cohen’s d = 0.225), indicated that the GPA of students enrolled in CRE courses (M = 3.30, SEM = 0.04) was significantly higher than the GPA of the general student population in the College of Arts and Sciences (M = 3.16, SEM = 0.03). The T-test results (T = 4.14, df = 795, p < 0.001, Cohen’s d = 0.349) also indicated that CRE students (M = 86.33, SEM = 1.23) obtained higher final course grades than the general population of College of Arts and Sciences students as a whole (M = 79.93, SEM = 0.76).

While these differences between CRE students and the general student population were significant, we could not control for demographic covariates since CRE and non-CRE student populations partially overlapped. Specifically, CRE students are also part of the general college population. Also, a large portion of CRE students have taken part in more than one CRE class. In spite of these limitations, two main considerations lead us to postulate that the two populations are comparable: (1) The heterogeneity of CRE courses well represent the variety of disciplines in the College of Arts and Sciences. In fact, CRE courses are distributed among the three departments of natural sciences, mathematics and computer science, and humanities, social sciences and communication, which include STEM, non-STEM, and social science courses in similar proportions. (2) When students decided which courses to take at the beginning of a semester, they did not know if they were going to participate in a CRE experience or not. In this way, we avoided the self-selection of historically high-achieving students that could lead to potential sampling biases and, likely, to systematic higher achievements in CRE students unrelated to the actual CRE experience.

The student experience

Normality of the survey data were assessed prior to analysis via skewness and kurtosis; criteria for normality of the data were met as skewness and kurtosis <2 (Asai, 2013). Below we report the results of the ANOVA analyses measuring the impact of CRE and the influence of academic discipline and gender on students’ attitude towards research, self-efficacy, teamwork, cultural sensitivity, gender issues, and race issues. Descriptive statistics of this analysis is provided in Table 2.

Attitude towards research

The main effect of CRE was significant, F(1, 173) = 25.19, p < 0.001, η²_p = 0.13. Participants rated their attitude towards research more positively after CRE (M = 2.77, SE = 0.04) than before CRE (M = 2.56, SE = 0.04). The main effect of discipline was also significant, F(4, 173) = 3.33, p = 0.012, η²_p = 0.07. Tukey post-hoc analysis found biology & chemistry students reported significantly higher research attitude than students in other fields (see Table 2). The main effect of gender was not significant, F(1, 173) = 0.87, p = 0.34 suggesting the positive effect of CRE on students’ attitude towards research is inclusive across students regardless of gender. All the other interactions between main factors were not significant.

Academic self-efficacy

The main effect of CRE was significant, F(1, 173) = 5.14, p = 0.025, η²_p = 0.029. Participants provided higher ratings of academic self-efficacy after CRE (M = 3.29, SE = 0.06) than before CRE (M = 3.17, SE = 0.05). The main effects of discipline, F(4, 173) = 0.47, p = 0.76, gender, F(1, 173) = 0.05, p = 0.831 and all the interactions between factors were not significant.

Teamwork

The main effect of CRE, F(1, 173) = 0.78, p = 0.378, gender, F(1, 173) = 0.64, p = 0.424, and all the interactions between factors were not significant. The main effect of discipline was significant, F(1, 173) = 2.62, p = 0.037, η²_p = 0.057, with Tukey post-hoc analysis finding biology & chemistry students reported significantly higher teamwork attitude than students in other fields (see Table 2).

Cultural sensitivity

The main effect of CRE, F(1, 167) = 0.34, p = 0.56, discipline, F(4, 167) = 0.5, p = 0.736, gender, F(1, 167) = 1.94, p = 0.166, and all the interactions between factors were not significant.

Gender issues

The main effect of CRE, F(1, 170) = 0.01, p = 0.925, gender, F(1, 170) = 2.17, p = 0.143, and all the interactions between factors were not significant. The main effect of discipline was significant, F(4, 170) = 2.74, p = 0.031, η²_p = 0.06.

Race issues

The main effects of CRE, F(1, 168) = 0.67, p = 0.413, discipline, F(4, 168) = 2.25, p = 0.066, gender, F(1, 168) = 2.32, p = 0.197, and all the interactions between factors were not significant. However, results do suggest the race factor may be relevant in CRE courses with regards to attitude towards research, F(1, 181) = 8.33, p = 0.004, η²_p = 0.04. Tukey post-hoc analysis indicated African-American students declared a more positive attitude towards original research (M = 2.96) than White students (M = 2.63, p = 0.15). The race factor also had a significant influence on the race issue measurement, F(4, 176) = 4.04, p = 0.004, η²_p = 0.08. Tukey post-hoc analysis indicated that Asian students perceived more racial discrimination in their classes (M = 1.66) than White students (M = 0.88, p = 0.006) and Hispanic students (M = 0.39, p < 0.004). The influence of race on all the other dependent measures, namely, academic self-efficacy, F(4, 181) = 0.41, p = 0.803, teamwork, F(4, 181) = 1.76, p = 0.14, cultural sensitivity, F(4, 175) = 1.99, p = 0.098, and gender issues, F(4, 178) = 1.69, p = 0.154, was not significant.

Faculty perspectives regarding sustainability

As detailed in the methods, 29 CRE faculty completed a brief survey during a faculty CRE retreat. The survey was aimed at exploring how likely the CRE faculty team was willing to use course-based research experience pedagogy in future semesters following the end of the grant period.

The CRE faculty who took part in the survey (29 out of 30) expressed a very strong intention to continue using CRE pedagogy in spite of the end of financial incentives. The average answer to the question “How likely are you to use course-based research experience pedagogy in the future, even if you will not be paid for any future CRE activities?” was significantly greater than a neutral value of 4 (see Table 2 for further details). Also, participants expressed a strong intention to continue using all the three pillars of CRE in their pedagogy. Specifically, participants’ self-reported likelihood to promote original discovery in the classroom, require inclusive collaboration, and facilitate communication of course discoveries were all very high and significantly different from a neutral response (see Table 2).

We also calculated the correlation between CRE expertise with respect to the number of courses taught at LTU and the responses to the four questions about sustainability. Results show that none of the sample’s answers to the sustainability question correlated with CRE expertise (see Table 3). This finding indicates that a high level of faculty commitment to CRE pedagogy, even in absence of financial support, is shared by all faculty involved in the project, regardless of the amount of previous experience within the CRE program.

Lastly, included in the survey was an open-ended question to gain direct feedback from faculty participants. While the response to the open-ended question was not mandatory, the majority of the sample population (65%) decided to provide additional feedback pertaining to their future use of CRE in their pedagogical practices, which included the pillars of CRE as the framework. Participants offered general comments (40%), challenges (30%) and ideas (30%) while also describing their interests and support for sustaining CRE pedagogy as an ongoing teaching practice. In addition, a thematic analysis was conducted to identify key themes from faculty experiences and their perspectives related to pedagogical commitments beyond the grant funding period (Yin, 2015). Patterned codes were analyzed to help determine if faculty found value in CRE as a sustainable practice.

The overall summative responses support the future use of CRE pedagogy and sustainability beyond the grant funding period by noting that the benefits of incorporating CRE pedagogy outweigh the costs. Faculty described the strengths and challenges related to the idea of continuing to develop and teach course-based research courses. Emergent themes were identified and reported: 1. CRE is an inclusive teaching practice that strengthens the credibility of learning and its academic content; 2. CRE offers an enhanced academic learning environment to both students and instructors; and 3. CRE teaching and learning goes beyond a stipend despite its challenges.

Faculty shared their view that CRE pedagogy is an inclusive teaching practice that strengthens faculty pedagogy, offers significant benefits to the classroom environment, and positively engages everyone involved. One faculty member described how “using CRE helps students learn much better and I really love it.” These descriptions align with faculty being satisfied with the learning process of their students. The academic benefits of CRE pedagogy are widely viewed as invaluable. One faculty member was encouraged by the program and shared that “course-based research allowed my students to explore the content as it provided them an avenue to learn and retain it.” As noted, CREs cultivate a richer learning environment that incentivizes faculty and students to engage in active research and deepens student learning. Therefore, faculty see the value in both the instructor and learner having positive experiences in an engaged and active classroom.

Survey data also confirmed faculty are motivated to engage in this pedagogical practice beyond the grant stipend period, thus confirming it can be sustained. Faculty noted that while the stipend helped to offset the time required for course development, it is not a necessary precursor to implement CREs. An instructor shared, “I did not need to teach over the summer because of the CRE stipend, however, CRE has allowed me to implement and promote more original discovery and in different ways – it has and will continue to do so even after funding.” Another faculty reflected, “financial and institutional support might not be necessary for me to teach CRE, but it would encourage me to teach more CRE courses, and incorporate the CRE pedagogy more thoroughly.” While faculty acknowledged the importance of the financial benefits, most of them expressed the opinion that the stipend is not a necessary requirement for CRE sustainability. In a theory of change theoretical framework (Reinholz and Andrews, 2020), we can argue that it is likely that financial support was important to overcome faculty’s initial resistance to change. Specifically, as the adoption of CRE pedagogy requires changes of course content, methods, and classroom dynamics that are necessarily costly in terms of time and effort, a financial reward is a great motivation to change from traditional lecturing methods to CRE. However, at the end of the financial incentives, the estimation of costs and benefits may have been changed permanently, with costs not as high as when starting and benefits that grow with CRE experience and expertise. This hypothesis is in line with the expectancy theory of motivation (Vroom and Yetton, 1973) and can be empirically tested in future studies.

Other experiences and challenges were also noted to understand a broader view of the workload and faculty expectations related to CRE. Some faculty expressed that the incorporation of original research in their discipline, which is one of the three pillars of CRE, was challenging. For example, two faculty shared the difficulty of promoting original discovery as a challenge. One said: “The largest challenge I have found in implementing course-based research or projects in mathematics courses is that often the most applicable course material occurs very late in the semester, after systematically building the course up over several months. It is challenging to have the student effectively define and analyze problems based on course material with which they are not yet familiar.”

Other faculty expressed concerns related to producing original research in their respective fields, they said, “Ensuring original discovery at a level acceptable to the scientific community in my field is next to impossible.” Furthermore, these challenges imply the need for faculty to continue to develop creative teaching strategies to further promote and facilitate inclusive course-based research practices in their academic content area in order to support and strengthen the credibility and authenticity of the program.

Discussion

The CRE program at LTU engages undergraduate students in authentic research experiences. Consistent with previous research that URE enhances academic performance (Barlow and Villarejo, 2004; Freeman et al., 2014; Ward et al., 2014), we determined that the course grades and GPA of students enrolled in CRE courses were significantly higher than the grades of the general population of students in the College of Arts and Sciences. We believe that the increased academic performance of CRE students is likely to be related to both the pedagogical characteristics of CRE and the subjective experience of students in CRE classrooms. While grades, examination scores, and failure rates in CRE and non-CRE courses are frequently compared to assess the efficacy of CRE pedagogy (Freeman et al., 2014; Ing et al., 2021 for a large meta-analysis), there are reasons to believe that such a measure is incomplete. In fact, CRE and non-CRE courses most likely have different instructors, are taught in different semesters to different students, utilize different pedagogical methods that emphasize different topics, and use different assessment tools and metrics. Therefore, we believe that using grades as the main method to measure the impact of CRE pedagogy highlights the challenges of controls and must be necessarily supported by other indicators, such as, for example, the assessment of the experience of CRE students and instructors.

Regarding student experiences in CRE courses, in line with previous findings (Harrison et al., 2011; Brabec et al., 2018), students’ attitude towards research was more positive after CRE than before it. This effect was not dependent on gender, highlighting the inclusive nature of CRE in improving students’ attitude towards research. We did find that biology and chemistry students rated their attitude towards research and teamwork more positively than students in other disciplines. We believe that such differences can be explained by student expectations of the educational research activities and teamwork in biology and chemistry laboratory courses. The main positive effect of CRE on research attitude underscores the strength of CRE in fostering a positive attitude towards research in all students, including those in disciplines that do not traditionally involve research in the classroom.

There is evidence that CRE can boost self-efficacy in students (Brownell et al., 2012). The results of our survey confirm this finding. In fact, students indicate that their academic self-efficacy was positively influenced by CRE, with students expressing a higher confidence in their academic abilities and potential following CRE interventions.

One of the most compelling aspects of our program is the heterogeneity of the intervention. Students in the LTU College of Arts and Sciences take multiple CRE courses during their undergraduate degree program, including both STEM and non-STEM disciplines. All students graduating from the College of Arts and Sciences in the past 5 years experienced CREs in more than one discipline. Such a vast and diverse range of fields creates a fertile environment in which creativity, problem-solving, and research methodologies in one field can be translated to another field. It is important to note that, since many of our CRE courses are part of LTU’s core curriculum, students from other academic colleges (Engineering, Architecture and Design, and Business and Information Technology) also benefit from this initiative. Our CRE model already counts several instances of direct cross-disciplinary integration incomputer science and art, psychology and philosophy, and English composition and design, for example. We are interested in understanding if the heterogeneity of the CRE intervention can provide additional benefits to CREs (Latham, 2018). However, more data is necessary to support the hypothesis that students who experience CRE in multiple courses develop an enhanced sense of methodological self-reflection and that the comprehensive nature of the CRE intervention at LTU develops a positive transfer of ideas. We are currently using focus groups and interviews to assess this hypothesis. Our preliminary results are in contrast with Brabec and colleagues, who found that first year students in a CRE biology laboratory course did not show an increased interest in research (Brabec et al., 2018).

According to the results of our surveys, students did not feel racial and gender discrimination in their classrooms. We are aware that such results could be biased by the fact that the vast majority of the students in the sample were White. Interestingly, the discipline in which CRE took place influenced the awareness of gender discrimination in the academic environment, with a greater awareness in literature and philosophy than in biology and chemistry. This is an intriguing result because it suggests that non-STEM disciplines can perhaps encourage cultural sensitivity more than STEM disciplines, resulting in a greater awareness of possible gender and racial issues. Therefore, interventions in non-STEM disciplines can be a crucial factor in the promotion of systemic institutional change that aims to develop an inclusive research environment for all students. Concerning the influence of race on students’ opinions and attitudes, results indicate that African American participants shared a more positive attitude towards original research than White and Asian participants. This finding confirms previous results showing that CRE interventions are particularly effective for underrepresented minority students, resulting in improved learning gains (Lopatto, 2007). Finally, the race of students had a significant influence on the perception of racial discrimination. Specifically, Asian participants perceived more racial discrimination in their classes than White, African American, and Hispanic participants. While racial discrimination towards Asian American students has previously been documented (Sue et al., 2009), our results are in contrast with previous findings that African American and Hispanic students report higher rates of microaggression incidents than Asian students (Torres-Harding and Turner, 2014; Forrest-Bank and Jenson, 2015). The difference is likely related to our small sample size or the different structure of surveys, with ours including a few general questions about ethnic, racial, and gender discrimination in the classroom while Forrest-Bank and Jenson’s and Torres-Harding and Turner’s findings were acquired using the racial and ethnic microaggressions scale developed by Nadal (2011).

While peer-reviewed publication is not an indispensable condition for a successful CRE (Dolan, 2016), it is definitely an aspect that can add value to the experience, improving a student’s sense of ownership, academic self-efficacy, and sense of belonging to the scientific or academic community (Asai, 2013). CRE can also be beneficial for faculty productivity in certain contexts (Gibson et al., 1996; Morales et al., 2017).

As a result of our CRE intervention, several peer-reviewed journal articles have resulted from CRE courses. Examples of CRE studies resulting in peer-reviewed articles with at least one undergraduate student in the list of authors include publications in computer science (Kuminski and Shamir, 2016; Chung and Kocherovsky, 2018; Paul et al., 2018; Shamir et al., 2019; Pleune et al., 2020), psychology (Delogu et al., 2016, 2020a,b; Delogu and Lilla, 2017; Delogu, 2020), and chemistry (Willbur et al., 2016; Zhou and Zhou, 2020; Large et al., 2023). In our experience, not all the students involved in a given CRE course are included in the list of authors for several reasons. In many cases, the CRE course is a pilot phase of a research project that requires more time than one semester to be completed. In other cases, CREs can be reiterated several times in different semesters. While only a fraction of CRE experiences can likely culminate in a peer-reviewed publication, most CRE students have the opportunity to present their work to audiences external to their classrooms.

In addition to peer-reviewed publications, students at LTU routinely present their CRE projects at national and international conferences. Some examples include physics (Houck and Bhattacharya, 2021), computer science (Shamir et al., 2019), and psychology (Delogu and Lilla, 2017). Finally, in the past 5 years, hundreds of CRE students also had the opportunity to present their work at regional conferences such as the Michigan Academy of Science, Arts, and Letters (MASAL) annual conference and at LTU Research Day, a yearly symposium dedicated to showcasing scholarly projects by students and faculty. The completion of the research cycle has several important beneficial consequences, such as the improvement of communication skills, the development of a sense of autonomy and research ownership, and a sense of self-efficacy (Spronken-Smith et al., 2013).

Faculty commitment is a crucial aspect of the sustainability of any systematic curricular transformation. In this regard, our CRE faculty expressed a very strong intention to continue using CRE pedagogy in spite of the end of financial incentives. Such self-reported commitment is shown by all the participants in CRE, regardless of the amount of previous experience within the program. While faculty acknowledged the importance of the financial benefits, most instructors shared that the stipend is not a necessary requirement for CRE sustainability. In a theory of change theoretical framework, we can argue that it is likely that financial support was important to overcoming faculty’s initial resistance to change. Specifically, as the adoption of CRE pedagogy requires changes of course content, methods, and classroom dynamics that are necessarily costly in terms of time and effort, a financial reward is a great motivation to change from more traditional teaching methods to CRE. However, at the end of the financial incentives, the estimation of costs and benefits may have been changed permanently, with costs not as high as when starting and benefits that grow with CRE experience and expertise. Such a hypothesis is in line with the expectancy theory of motivation at work (Jones and Vroom, 1964) and can be empirically tested in future studies.

Considerations and future research

As a result of several years of practice, we believe that our CRE intervention produced a second order institutional change that promotes the inclusive access of all students to authentic research experiences that nurture students’ self-efficacy and academic potential. Several conditions facilitated the emergence of a sustainable systemic change that is likely to outlast the external funding phase of the project: (1) we created a critical mass of CRE courses and instructors by providing both incentives and training to overcome resistance to change; (2) we developed a cohesive community of instructors who share common general principles and practices; (3) our program has the support of the university administration; (4) we diversified the range of intervention by facilitating an understanding of the concepts of scholarship, research, and discovery in diverse disciplines and by developing conceptual intersections between multiple CRE experiences in different fields; and (5) we integrated the intervention with partners that share similar goals or practices; for example, with internal institutions such as the Center for Teaching and Learning and the Office of Diversity, Equity, and Inclusion, student organizations, and external collaborators and networks. (6) we strengthened our inclusivity mission with the practice of culturally-responsive teaching and the development of a teaching and learning community, multi-institution collaborations, the development of a dedicated seminar series, the substantial involvement of non-STEM disciplines, and student researcher awards.

We believe that our program is an excellent option for the instructors interested in implementing a problem-based learning (PBL) approach in their teaching practices. In fact, CRE includes all PBL main features, such as self-directed learning, the independent use of resources, peer collaboration, data collection, flexibility in learning outcomes, the development of problem-solving skills, and the promotion of intrinsic motivation (Wood, 2003; Hmelo-Silver, 2004). In addition to PBL advantages, CRE also includes a clear orientation to original research, in which students and instructors collaborate to solve problems with unknown solutions. This aspect of originality invigorates students’ sense of ownership, promoting a sense of academic self-efficacy and intrinsic motivation.

While undoubtedly positive and formative, our experience also presented challenges, which were expressed by faculty and students in their open-ended reports and discussions. We summarize the main points of concern as follows, together, when possible, with strategies to mitigate them: (1) Embedding CRE in a regular course takes time away from the traditional curriculum that must be compressed and/or reduced to provide time for research activities. Some courses, especially the ones in which the curriculum must necessarily cover basic concepts in STEM, did not have the required time flexibility to allow any CRE insertion or limited CRE to a small number of class periods. The solution we adopted is to allow the time dedicated to CRE as much flexible as possible; for example, some CREs required just a few class sessions, while others involved the entire semester; (2) as students did not decide whether to participate in CRE (i.e., no self-selection), a small number of them manifested frustration with the additional work, creativity, and problem-solving processes often required to perform original research. A strategy to reduce these frustrations is to explicitly engage these students in the ownership of the CRE project through collaboration, as well as present students with the potential advantages of CRE for their academic preparation and career paths; (3) teamwork in CRE is fundamental, but can be challenging. Survey data indicates that many students are frustrated by the unequal distribution of work and/or by sharing parts of their grade with other students. This is particularly true for successful high-achieving students who often claim to have worked more than their teammates. Possible strategies include fun and engaging team-building activities, dividing work into very small groups to mitigate “hiding,” encouraging team member engagement (e.g., teams of 2–3 students focus on a very precise task), asking for anonymous and evidence-based evaluations of the work of teammates, inviting team members to complete a team participation contract, and encouraging team members to emerge as leaders and take an active role in facilitating project completion.

As we continue investigating the effects of CRE implementation at LTU, we are interested in studying a variety of factors. For example, given more opportunities (and requirements) for students to participate in CRE remotely, we could investigate the differences in CRE delivery remotely versus in person. As we increase faculty participation in teaching CRE courses, we could investigate the gender effects of instructors within and between disciplines. We are also exploring additional constructs to measure through student surveys, such as self-awareness, critical thinking, and reflective skills.

Conclusion

In this study, we described a large-scale multidisciplinary CRE program at LTU, and assessed its impact on student success and the student experience as well as faculty perceptions pertaining to the sustainability of the program following the grant period. In our transformative program, a large portion of College of Arts and Sciences faculty actively promote and facilitate the three pillars of CRE, namely (a) discovery through scholarly practice, (b) inclusive collaboration, and (c) communication of relevance. Our assessment data indicates that students have positive experiences, and tend to do better academically when they are engaged in course-based research practices. Importantly, faculty involved in CRE pedagogy fully support the mission and values of the CRE community and intend to continue to implement CREs in absence of external financial support. Our experience and data supports the idea that CRE has become an integral part of the LTU core curriculum teaching practice and is now considered an important part of the fabric and culture of LTU as an institution.

References

• 1

  AsaiD. (2013). Persistence in Science by All Students: Lessons From the Little Red Hen and Other Fairy Tales. Proceedings of the 2013 AAC&U Conference on Transforming STEM Education: Inquiry, Inclusion and Evidence, San Diego.

  • Google Scholar

• 2

  AuchinclossL. C.LaursenS. L.BranchawJ.EaganK.GrahamM.HanauerD. I.et al. (2014). Assessment of course-based undergraduate research experiences: a meeting report. CBE—life sciences. Education13, 29–40. doi: 10.1187/cbe.14-01-0004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3

  BangeraG.BrownellS. E. (2014). Course-based undergraduate research experiences can make scientific research more inclusive. CBE—life sciences. Education13, 602–606. doi: 10.1187/cbe.14-06-0099

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4

  BarlowA. E. L.VillarejoM. (2004). Making a difference for minorities: evaluation of an educational enrichment program. J. Res. Sci. Teach.41, 861–881. doi: 10.1002/tea.20029

  • CrossRef
  • Google Scholar

• 5

  BeninsonL. A.KoskiJ.VillaE.FaramR.O’ConnorS. E. (2011). Evaluation of the research experiences for undergraduates (REU) sites program. Counc. Undergrad. Res. Q.32, 43–48.

  • Google Scholar

• 6

  BoomerS. M.LodgeD. P.DuttonB. E. (2002). Bacterial diversity studies using the 16S rRNA gene provide a powerful research-based curriculum for molecular biology laboratory. Microbiol. Educ.3, 18–25. doi: 10.1128/me.3.1.18-25.2002

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7

  BrabecJ. L.VosM. R.StaabT. A.ChanJ. P. (2018). Analysis of student attitudes of a neurobiology themed inquiry based research experience in first year biology labs. J. Undergrad. Neurosci. Educ.17, A1–A9. PMID:

  • Pubmed Abstract
  • Google Scholar

• 8

  BraxtonJ. M.HirschyA. S.McClendonS. A. (2004). Understanding and Reducing College Student Departure. ASHE-ERIC Higher Education Report, vol. 30. Indianapolis, IN: Jossey-Bass, An Imprint of Wiley.

  • Google Scholar

• 9

  BrodlM. R. (2005). Tapping recent alumni for the development of cutting-edge, investigative teaching laboratory experiments. Bioscene31, 13–20.

  • Google Scholar

• 10

  BrownellS. E.KloserM. J.FukamiT.ShavelsonR. (2012). Undergraduate biology lab courses: comparing the impact of traditionally based "cookbook" and authentic research-based courses on student lab experiences. J. Coll. Sci. Teach.41, 36–45.

  • Google Scholar

• 11

  CarpiA.RonanD. M.FalconerH. M.LentsN. H. (2017). Cultivating minority scientists: undergraduate research increases self-efficacy and career ambitions for underrepresented students in STEM. J. Res. Sci. Teach.54, 169–194. doi: 10.1002/tea.21341

  • CrossRef
  • Google Scholar

• 12

  CarterF. D.MandellM.MatonK. I. (2009). The influence of on-campus, academic year undergraduate research on STEM Ph.D. outcomes: evidence from the Meyerhoff scholarship program. Educ. Eval. Policy Anal.31, 441–462. doi: 10.3102/0162373709348584

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  CarusoS. M.SandozJ.KelseyJ. (2009). Non-STEM undergraduates become enthusiastic phage-hunters. CBE—life sciences. Education8, 278–282. doi: 10.1187/cbe.09-07-0052

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14

  ChemersM. M.ZurbriggenE. L.SyedM.GozaB. K.BearmanS. (2011). The role of efficacy and identity in science career commitment among underrepresented minority students. J. Soc. Issues67, 469–491. doi: 10.1111/j.1540-4560.2011.01710.x

  • CrossRef
  • Google Scholar

• 15

  ChungC.KocherovskyM. (2018). CS+ PA 2: Learning Computer Science with Physical Activities and Animation – A MathDance Experiment. 2018 IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, 262–267.

  • Google Scholar

• 16

  CollinsC. M.CarrascoG. A.LopezO. J. (2019). Participation in active learning correlates to higher female performance in a pipeline course for underrepresented students in medicine. Med. Sci. Educ.29, 1175–1178. doi: 10.1007/s40670-019-00794-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17

  CorwinL. A.GrahamM. J.DolanE. L. (2015). Modeling course-based undergraduate research experiences: an agenda for future research and evaluation. CBE—life sciences. Education14:es1. doi: 10.1187/cbe.14-10-0167

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 18

  CUGESEWP. (2011). Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads. Washington, DC: Committee on Underrepresented Groups and the Expansion of the Science and Engineering Workforce Pipeline, National Academy of Sciences.

  • Google Scholar

• 19

  DeloguF. (2020). Course-Based Research Experience as a Pedagogical Model to Make Higher Education Accessible to All Students. Proceedings of the 19th Best Practices Conference on Teaching and Learning (BPC): Practical Approaches in STEM Education and Research. Pontificia Universidad Católica de Puerto Rico (PUCPR)-Ponce, February 28 & 29, 2020.

  • Google Scholar

• 20

  DeloguF.BarnewoldM.MeloniC.ToffaliniE.ZiziA.FanariR. (2020a). The Morra game as a naturalistic test bed for investigating automatic and voluntary processes in random sequence generation. Front. Psychol.11:551126. doi: 10.3389/fpsyg.2020.551126

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21

  DeloguF.GravinaM.DongX.FrolkaM.KuhnD.YuN. (2020b). Tactile beauty is in the hand, but also in the eye of the beholder: interaction between haptic and visual experiences in aesthetic judgement. Psychol. Aesthet. Creat. Arts15, 725–734. doi: 10.1037/aca0000327

  • CrossRef
  • Google Scholar

• 22

  DeloguF.HuddasC.StevenK.HachemS.LodhiaL.FernandezR.et al. (2016). A dissociation between recognition and hedonic value in caloric and non-caloric carbonated soft drinks. Front. Psychol.7:36. doi: 10.3389/fpsyg.2016.00036

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23

  DeloguF.LillaC. C. (2017). Do you remember where sounds, pictures and words came from? The role of the stimulus format in object location memory. Memory25, 1340–1346. doi: 10.1080/09658211.2017.1300668

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  DeweyJ.EversA.SchuchardtA. (2022). Students’ experiences and perceptions of the scientific research culture after participating in different course-based undergraduate research experience models. CBE—life sciences. Education21:ar36. doi: 10.1187/cbe.21-10-0304

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25

  DolanE. L. (2016). Course-based undergraduate research experiences: current knowledge and future directions. Natl Res Counc Comm Pap1, 1–34.

  • Google Scholar

• 26

  DrewJ. C.TriplettE. W. (2008). Whole genome sequencing in the undergraduate classroom: outcomes and lessons from a pilot course. J. Microbiol. Biol. Educ.9, 3–11. doi: 10.1128/jmbe.v9.89

  • CrossRef
  • Google Scholar

• 27

  DvorakA. L.Hernandez-RuizE. (2019). Outcomes of a course-based undergraduate research experience (CURE) for music therapy and music education students. J. Music. Ther.56, 30–60. doi: 10.1093/jmt/thy020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28

  ElwessN. L.LatourelleS. L. (2004). Inducing mutations in paramecium: an inquiry-based approach. Bioscene30, 25–35.

  • Google Scholar

• 29

  Forrest-BankS.JensonJ. M. (2015). Differences in experiences of racial and ethnic microaggression among Asian, Latino/Hispanic, black, and white young adults. J. Sociol. Soc. Welf.42, 141–161. doi: 10.15453/0191-5096.3891

  • CrossRef
  • Google Scholar

• 30

  FreemanS.EddyS. L.McDonoughM.SmithM. K.OkoroaforN.JordtH.et al. (2014). Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci.111, 8410–8415. doi: 10.1073/pnas.1319030111

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31

  GibsonP. R.KahnA. S.MathieV. A. (1996). Undergraduate research groups: two models. Teach. Psychol.23, 36–38. doi: 10.1207/s15328023top2301_7

  • CrossRef
  • Google Scholar

• 32

  GilmerT. (2007). An understanding of the improved grades, retention and graduation rates of STEM majors at the academic Investment in Math and Science (AIMS) program of Bowling Green State University (BGSU). J STEM Educ8, 11–21.

  • Google Scholar

• 33

  HanauerD. I.HatfullG. (2015). Measuring networking as an outcome variable in undergraduate research experiences. CBE—life sciences. Education14:ar38. doi: 10.1187/cbe.15-03-0061

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34

  HanauerD. I.Jacobs-SeraD.PedullaM. L.CresawnS. G.HendrixR. W.HatfullG. F. (2006). Teaching scientific inquiry. Science314, 1880–1881. doi: 10.1126/science.1136796

  • CrossRef
  • Google Scholar

• 35

  HarrisonM.DunbarD.RatmanskyL.BoydK.LopattoD. (2011). Classroom-based science research at the introductory level: changes in career choices and attitude. CBE—life sciences. Education10, 279–286. doi: 10.1187/cbe.10-12-0151

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36

  HatfullG. F.PedullaM. L.Jacobs-SeraD.CichonP. M.FoleyA.FordM. E.et al. (2006). Exploring the mycobacteriophage metaproteome: phage genomics as an educational platform. PLoS Genet.2:e92. doi: 10.1371/journal.pgen.0020092

  • CrossRef
  • Google Scholar

• 37

  HathawayR. S.NagdaB. A.GregermanS. R. (2002). The relationship of undergraduate research participation to graduate and professional education pursuit: an empirical study. J. Coll. Stud. Dev.43, 614–631.

  • Google Scholar

• 38

  Hernandez-RuizE.DvorakA. L. (2020). Replication of a course-based undergraduate research experience for music students. Nord. J. Music. Ther.29, 317–333. doi: 10.1080/08098131.2020.1737186

  • CrossRef
  • Google Scholar

• 39

  Hmelo-SilverC. E. (2004). Problem-based learning: what and how do students learn?Educ. Psychol. Rev.16, 235–266. doi: 10.1023/B:EDPR.0000034022.16470.f3

  • CrossRef
  • Google Scholar

• 40

  HouckA.BhattacharyaB. (2021). Understanding Introductory Physics Concepts with Computational Essays. Virtual Conference for Undergraduate Women in Physics, 2021. Available at: https://meetings.aps.org/Meeting/CUWIP21/Session/U20.3 (Accessed September 25, 2023).

  • Google Scholar

• 41

  HowardD. R.MiskowskiJ. A. (2005). Using a module-based laboratory to incorporate inquiry into a large cell biology course. Cell Biol. Educ.4, 249–260. doi: 10.1187/cbe.04-09-0052

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42

  HunterA.-B.LaursenS. L.SeymourE. (2007). Becoming a scientist: the role of undergraduate research in students' cognitive, personal, and professional development. Sci. Educ.91, 36–74. doi: 10.1002/sce.20173

  • CrossRef
  • Google Scholar

• 43

  IngM.BurnetteJ. M.IIIAzzamT.WesslerS. R. (2021). Participation in a course-based undergraduate research experience results in higher grades in the companion lecture course. Educ. Res.50, 205–214. doi: 10.3102/0013189X20968097

  • CrossRef
  • Google Scholar

• 44

  JonesS. C.VroomV. H. (1964). Division of labor and performance under cooperative and competitive conditions. J. Abnorm. Soc. Psychol.68, 313–320. doi: 10.1037/h0042378

  • CrossRef
  • Google Scholar

• 45

  JordanT. C.BurnettS. H.CarsonS.CarusoS. M.ClaseK.DeJongR. J.et al. (2014). A broadly implementable research course in phage discovery and genomics for first-year undergraduate students. MBio5, e01051–e01013. doi: 10.1128/mBio.01051-13

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  KappE. (2009). Improving student teamwork in a collaborative project-based course. Coll. Teach.57, 139–143. doi: 10.3200/CTCH.57.3.139-143

  • CrossRef
  • Google Scholar

• 47

  KinkelD. H.HenkeS. E. (2006). Impact of undergraduate research on academic performance, educational planning, and career development. J. Nat. Resour. Life Sci. Educ.35, 194–201. doi: 10.2134/jnrlse2006.0194

  • CrossRef
  • Google Scholar

• 48

  KuminskiE.ShamirL. (2016). A computer-generated visual morphology catalog of ~3,000,000 SDSS galaxies. Astrophys. J. Suppl. Ser.223:10. doi: 10.3847/0067-0049/223/2/20

  • CrossRef
  • Google Scholar

• 49

  LargeD. N.Van DoornN. A.TimmonsS. C. (2023). Cancer and chemicals: A research‐inspired laboratory exercise based on the Ames test for mutagenicity. Biochem Mol Biol Educ.51, 103–113.

  • Google Scholar

• 50

  LathamX. (2018). A Student’s Perspectives on the Benefits of Experiential Learning Idea Factory Lecture Series, Lawrence Technological University, Southfield, MI, November 18, 2018.

  • Google Scholar

• 51

  LewisK. L.StoutJ. G.PollockS. J.FinkelsteinN. D.ItoT. A. (2016). Fitting in or opting out: a review of key social-psychological factors influencing a sense of belonging for women in physics. Phys. Rev. Phys. Educ. Res.12:020110. doi: 10.1103/PhysRevPhysEducRes.12.020110

  • CrossRef
  • Google Scholar

• 52

  LopattoD. (2004). Survey of undergraduate research experiences (SURE): first findings. Cell Biol. Educ.3, 270–277. doi: 10.1187/cbe.04-07-0045

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 53

  LopattoD. (2007). Undergraduate research experiences support science career decisions and active learning. CBE—life sciences. Education6, 297–306. doi: 10.1187/cbe.07-06-0039

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 54

  LopattoD.AlvarezC.BarnardD.ChandrasekaranC.ChungH. M.duC. (2008). Genomics education partnership. Science322, 684–685. doi: 10.1126/science.1165351

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55

  MartinA.RechsA.LanderholmT.McDonaldK. (2021). Course-based undergraduate research experiences spanning two semesters of biology impact student self-efficacy but not future goals. J. Coll. Sci. Teach.50, 33–47.

  • Google Scholar

• 56

  MicariM.PazosP.HartmannM. J. (2007). A matter of confidence: gender differences in attitudes toward engaging in lab and course work in undergraduate engineering. J. Women Minorities Sci. Eng.13, 279–293. doi: 10.1615/JWomenMinorScienEng.v13.i3.50

  • CrossRef
  • Google Scholar

• 57

  MoralesD. X.GrineskiS. E.CollinsT. W. (2017). Increasing research productivity in undergraduate research experiences: exploring predictors of collaborative faculty–student publications. CBE Life Sci. Educ.16:ar42. doi: 10.1187/cbe.16-11-0326

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58

  NadalK. L. (2011). The racial and ethnic microaggressions scale (REMS): construction, reliability, and validity. J. Couns. Psychol.58, 470–480. doi: 10.1037/a0025193

  • CrossRef
  • Google Scholar

• 59

  NagdaB. A.GregermanS. R.JonidesJ.von HippelW.LernerJ. S. (1998). Undergraduate student-faculty research partnerships affect student retention. Rev. High. Educ.22, 55–72. doi: 10.1353/rhe.1998.0016

  • CrossRef
  • Google Scholar

• 60

  OlsonS.RiordanD. (2012). Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Report to the President. PCAST (President's Council of Advisors on Science and Technology), February 7, 2012.

  • Google Scholar

• 61

  OsborneJ.SimonS.CollinsS. (2003). Attitudes towards science: a review of the literature and its implications. Int. J. Sci. Educ.25, 1049–1079. doi: 10.1080/0950069032000032199

  • CrossRef
  • Google Scholar

• 62

  PaulN.PleuneM.ChungC.WarrickB.BleicherS.FaulknerC. (2018). ACTor: A Practical, Modular, and Adaptable Autonomous Vehicle Research Platform. 2018 IEEE International Conference on Electro/Information Technology (EIT), Rochester, MI, 2018, 0411-0414.

  • Google Scholar

• 63

  PleuneM.PaulN.FaulknerC.ChungC. J. (2020). Specifying Route Behaviors of Self-Driving Vehicles in ROS Using LUA Scripting Language with Web Interface. 2020 IEEE International Conference on Electro Information Technology (EIT), Chicago, IL, USA, 535–539.

  • Google Scholar

• 64

  PopeW. H.HatfullG. F. (2015). Adding pieces to the puzzle: new insights into bacteriophage diversity from integrated research-education programs. Bacteriophage5:e1084073. doi: 10.1080/21597081.2015.1084073

  • CrossRef
  • Google Scholar

• 65

  RamirezM.McNicholasJ.GilbertB.SaezJ.SiniawskiM. (2015). Creative funding strategies for undergraduate research at a primarily undergraduate liberal arts institution. Counc. Undergr. Res. Q.36, 5–9. doi: 10.18833/curq/36/2/1

  • CrossRef
  • Google Scholar

• 66

  ReinholzD. L.AndrewsT. C. (2020). Change theory and theory of change: what’s the difference anyway?Int. J. STEM Educ.7, 1–12.

  • Google Scholar

• 67

  RonsheimM. L.PregnallA. M.SchwarzJ.SchlessmanM. A.Raley-SusmanK. M. (2009). Teaching outside the can: a new approach to introductory biology. Bioscene35, 12–22.

  • Google Scholar

• 68

  RorrerA. S.AllenJ.ZuoH. (2018). A National Study of Undergraduate Research Experiences in Computing: Implications for Culturally Relevant Pedagogy. Proceedings of the 49th ACM Technical Symposium on Computer Science Education, 604–609.

  • Google Scholar

• 69

  RussellS. H.HancockM. P.McCulloughJ. (2007). Benefits of undergraduate research experiences. Science316, 548–549. doi: 10.1126/science.1140384

  • CrossRef
  • Google Scholar

• 70

  SeymourE.HunterA.-B.LaursenS. L.DeAntoniT. (2004). Establishing the benefits of research experiences for undergraduates in the sciences: first findings from a three-year study. Sci. Educ.88, 493–534. doi: 10.1002/sce.10131

  • CrossRef
  • Google Scholar

• 71

  ShafferC. D.AlvarezC.BaileyC.BarnardD.BhallaS.ChandrasekaranC.et al. (2010). The genomics education partnership: successful integration of research into laboratory classes at a diverse group of undergraduate institutions. CBE—life sciences. Education9, 55–69. doi: 10.1187/09-11-0087

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 72

  ShafferC. D.AlvarezC. J.BednarskiA. E.DunbarD.GoodmanA. L.ReinkeC.et al. (2014). A course-based research experience: how benefits change with increased investment in instructional time. CBE—life sciences. Education13, 111–130. doi: 10.1187/cbe-13-08-0152

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 73

  ShamirM.KocherovskyM.ChungC. (2019). A Paradigm for Teaching Math and Computer Science Concepts in K-12 Learning Environment by Integrating Coding, Animation, Dance, Music and Art. 2019 IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, USA, 62–68.

  • Google Scholar

• 74

  Spronken-SmithR. A.BrodeurJ. J.KajaksT.LuckM.MyattP.VerburghA.et al. (2013). Completing the research cycle: a framework for promoting dissemination of undergraduate research and inquiry. Teach. Lear. Inq. ISSOTL J1, 105–118. doi: 10.2979/teachlearninqu.1.2.105

  • CrossRef
  • Google Scholar

• 75

  SueD. W.BucceriJ.LinA. I.NadalK. L.TorinoG. C. (2009). Racial microaggressions and the Asian American experience. Asian. Am. J. Psychol.S, 88–101. doi: 10.1037/1948-1985.S.1.88

  • CrossRef
  • Google Scholar

• 76

  SummersM. F.HrabowskiF. A. (2006). Preparing minority scientists and engineers. Science311, 1870–1871. doi: 10.1126/science.1125257

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 77

  Torres-HardingS.TurnerT. (2014). Assessing racial microaggression distress in a diverse sample. Eval. Health Prof.38, 464–490. doi: 10.1177/0163278714550860

  • CrossRef
  • Google Scholar

• 78

  TsuiL. (2007). Effective strategies to increase diversity in STEM fields: a review of the research literature. J. Negro Educ.76, 555–581.

  • Google Scholar

• 79

  VillarejoM.BarlowA. E. L.KoganD.VeazeyB. D.SweeneyJ. K. (2008). Encouraging minority undergraduates to choose science careers: career paths survey results. CBE—life sciences. Education7, 394–409. doi: 10.1187/cbe.08-04-0018

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 80

  VroomV. H.YettonP. W. (1973). Leadership and Decision-Making (Vol. 110). Pittsburgh, PA: University of Pittsburgh Press.

  • Google Scholar

• 81

  WalpoleM. (2003). Socioeconomic status and college: how SES affects college experiences and outcomes. Rev. High. Educ.27, 45–73. doi: 10.1353/rhe.2003.0044

  • CrossRef
  • Google Scholar

• 82

  WardJ. R.ClarkeH. D.HortonJ. L. (2014). Effects of a research-infused botanical curriculum on undergraduates’ content knowledge, STEM competencies, and attitudes toward plant sciences. CBE Life Sci. Educ.13, 387–396. doi: 10.1187/cbe.13-12-0231

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 83

  WeinsteinM.MooreH.-P.DeloguF.ShamirL. (2019). “Culturally responsive computational science through research experience in core-curriculum courses” in Culturally Responsive Strategies for Reforming STEM Higher Education. Eds. SotoM.MackK. M.WinterK. (Bingley, United Kingdom: Emerald Publishing Limited), 135–151.

  • Google Scholar

• 84

  WillburJ. F.VailJ. D.MitchellL. N.JakemanD. L.TimmonsS. C. (2016). Expression, purification, and characterization of a carbohydrate-active enzyme: a research-inspired methods optimization experiment for the biochemistry laboratory. Biochem. Mol. Biol. Educ.44, 75–85. doi: 10.1002/bmb.20928

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 85

  WoodD. F. (2003). Problem based learning. BMJ326, 328–330. doi: 10.1136/bmj.326.7384.328

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 86

  YinR. K. (2015). Qualitative Research From Start to Finish. New York, NY: Guilford Publications.

  • Google Scholar

• 87

  ZhouX.ZhouM. (2020). Polyvinylpyrrolidone-stabilized iridium nanoparticles catalyzed the transfer hydrogenation of nitrobenzene using formic acid as the source of hydrogen. Chemistry2, 960–968. doi: 10.3390/chemistry2040061

  • CrossRef
  • Google Scholar