The Effects of Undergraduate Research Experiences as Reported by Texas A&M University System Louis Stokes Alliance for Minority Participation Students

Introduction

The overall goal of the National Science Foundation (NSF) LSAMP program “is to assist universities and colleges in diversifying the nation’s science, technology, engineering and mathematics (STEM) workforce by increasing the number of STEM baccalaureate and graduate degrees awarded to populations historically underrepresented in these disciplines: African Americans, Hispanic Americans, American Indians, Alaska Natives, Native Hawaiians, and Native Pacific Islanders” (National Science Foundation, n.d.). “The Texas A&M University System (TAMUS) Louis Stokes Alliance for Minority Participation (LSAMP) program…(focuses on) encouraging and supporting…underrepresented minority (URM) STEM majors at… Alliance member” (Merriweather et al., 2017, p. 1) institutions. “Formally called TX LSAMP, the Alliance was one of the first six LSAMPs funded by NSF” (Merriweather et al., 2017, p. 1). Since 1991, TAMUS LSAMP has supported over 11,500 “undergraduates for one or more semesters of their undergraduate studies” (Merriweather et al., 2017, p. 1) and Alliance institutions have awarded over 22,000 STEM degrees to URMs.

“Using a carefully conceived suite of opportunities specially designed for URM undergraduate (UG) students…the Alliance has” (Merriweather et al., 2017, p. 1) sought improvement of academic success for underrepresented students. Programming and the number of member institutions have varied in the Alliance’s 30 years of operation, but the emphasis on academic success and advancement has remained constant. The current member institutions of TAMUS LSAMP are Texas A&M University at College Station (TAMU), a Very High Research Activity institution in the Carnegie Classification System (Indiana University Center for Postsecondary Research, n.d.), Prairie View A&M University (PVAMU), an Historically Black College and University (HBCU) (U.S. Department of Education, 2020), Texas A&M University–Corpus Christi (TAMUCC), an Hispanic-Serving Institution (HSI) (National Center for Education Statistics, 2018), and Texas A&M International University (TAMIU), an HSI (National Center for Education Statistics, 2018). TAMUCC’s Carnegie classification is Doctoral Universities: High Research Activity. Both PVAMU and TAMIU are in the Master’s Colleges and Universities Larger Program category (Indiana University Center for Postsecondary Research, n.d.). TAMUCC and TAMIU were among the first institutions designated as HSIs as they appear on the Excelencia in Education map of 1994–1995 HSIs, with 35.4 and 93.1% Hispanic student enrollment, respectively (Excelencia in Education, n.d.). Both campuses have continued to have more than 25% of their student population identifying as Hispanic/Latino in each subsequent year [TAMUCC 52% in fall 2020, TAMIU 95% in fall 2020 (National Center for Education Statistics [NCES], 2020a,b)].

Undergraduate research (UR) has been a component through the entire history of TAMUS LSAMP and became a primary emphasis in 2007. UR has a broad base of support in the literature including compendia of practice, process, and outcomes specific to the sciences (Laursen et al., 2010). It is also recognized as a high impact practice in higher education (American Association of Colleges and Universities, n.d.; Kuh and O’Donnell, 2013). Evidence indicates that having conducted research strengthens students’ confidence and their understanding of research (Laursen et al., 2010), ability to visualize themselves as an engineer and researcher (Bowman and Stage, 2002; Watson and Froyd, 2007), and that undergraduate research experiences provide the impetus for continuation to graduate school (Schmidt, 2003; Preuss et al., 2020). However, many URM students participating in research at predominantly White institutions like TAMU may find themselves the only URM in their lab, contributing to feelings of isolation (Perez et al., 2018). These circumstances motivated the TAMUS LSAMP to focus on undergraduate research experience as a modality of student support while gathering data to assess its efficacy for students from underrepresented groups on the Alliance campuses.

Contributions made to the literature by this article include confirmation and extension of ideas as well as new information. Overall, the material extends the knowledge base regarding the impacts of UR on URM students by confirming that findings from other studies apply to URM audiences and providing consideration of new material in respect to UR for URM students. The unique contributions of this consideration are establishing that general support of a variety of UR experiences was efficacious for URM students, that the same practices were effective at four institutions in different Carnegie classifications two of which are HSIs and one of which is an HBCU, and inclusion of student reports about the patterns of instruction and training included in their UR experiences.

Pedagogical Framework

As a grant-funded project focused on improving outcomes for URM STEM students, the instructional framework of TAMUS LSAMP has shifted across the 30 years of continuous operation. Practices emphasized in the earlier cycles became established and institutionalized and new endeavors were added. Examples of early emphases that have been institutionalized are learning communities, community college transfer programming, and summer bridge programming (Merriweather et al., 2017). Undergraduate research has been a particular emphasis with site-specific and Alliance-wide workshops about UR offered and student engagement in UR encouraged and financially supported.

While UR has been emphasized, the possible forms of involvement in it have not been restricted. The spectrum of UR opportunities for STEM students is broad and, due to the established efficacy of undergraduate research, TAMUS LSAMP leaders chose not to limit the possible forms of involvement. In an Alliance comprised of four state universities, one in the Very High Research Activity category, one in the High Research Activity category, and two classified as Master’s Colleges and Universities Larger Program (Indiana University Center for Postsecondary Research, n.d.), most forms of UR have been available to TAMUS LSAMP participants including international research experiences (Garcia et al., 2017; Preuss et al., 2020, 2021). The project personnel and their partners at each member institution recruited participants independently in this and all other areas. Thus, a specific framework and pedagogical context in which the UR experiences of the students occurred cannot be detailed. They extend from traditional approaches like inclusion as a student worker in a lab-based investigation to definition of individual projects by students, with the assistance of faculty advisors, that were then executed in international settings. All, though, involved participation in active STEM investigations. Taking this approach, a generalized pattern of facilitation as promotional and informational workshops plus fiscal support, made verification that the impacts of UR noted in the literature would occur for students necessary as activity was not limited to a closely defined context. The information regarding outcomes presented below was gathered to ascertain whether the facilitation pattern was in fact efficacious and whether variation in outcomes existed between institutions or subsets in the pool of participants.

Learning Environment

The learning environment was not restricted. Students were allowed to complete research as part of study abroad programming, were supported as participants in institutionally-based international research efforts, worked on project teams at Alliance institutions (grant-funded and otherwise), and pursued individual projects guided by faculty.

These experiences took place in a wide variety of contexts. They included universities in Europe, South America, and Mexico, American universities within and outside the Alliance, a research center in Belize, national labs, and community-based undertakings. Students participated in a great many of the traditional aspects of a research project (see Figure 2 and Table 2). All these processes were completed under the supervision of faculty from the four institutions or at a university to which the student traveled. The unifying characteristic was participation in an active investigation in a STEM field under the supervision of a university faculty member. In addition to mentoring by faculty, each institution provided use of equipment and supplies as well as, when applicable, support for students to present their work at a research conference.

FIGURE 1

Figure 1. Number of female and male students and percentage of underrepresented minorities (URMs) by campus for 2019 survey.

FIGURE 2

Figure 2. Response counts for elements of research experienced in undergraduate research.

TABLE 1

Table 1. Survey responses 2007–2016.

TABLE 2

Table 2. Grouping of research experiences by response rate.

Methodology

Much of the material discussed in this paper was gathered for project evaluation as a means of supporting the development of distinct patterns of programming and assessing their impact. The primary emphasis was on obtaining information about and understanding the impact of each intervention. These data when considered across a period of years facilitated broad analysis of impact.

Method: Data Sources

Many of the applicable data sources were identified by reviewing the 150 evaluation reports generated for the TAMUS LSAMP project between the 2007–2008 and 2019–2020 school years. This included quantitative data like participant counts and responses to survey questions, and qualitative data in the form of short answers to open-ended survey questions and interview and focus group transcripts. Journal articles published regarding the TAMUS LSAMP project were also consulted as sources of information (Graham et al., 2001; Garcia et al., 2017; Merriweather et al., 2017; Preuss et al., 2020) as were programmatic and institutional data.

To arrive at aggregated sets summarizing findings over a period of years, elements of related data sets were combined. This was a simple process when the same questions were used for several years. Interruptions to patterns occurred when the project felt a construct under investigation had received sufficient consideration, when the program shifted emphases or began support of new patterns of programming, or when questions were determined to be ineffective for producing the intended result. Most shifts in emphasis occurred in conjunction with the 5-year project funding cycle. Brief descriptions of how data were combined and when changes in data patterns occurred will precede discussion of findings from each of the data sets.

Much of these data came in the form of student self-reports and addressed experience, self-assessment, and personal opinions. Given the intention of evaluating LSAMP programming using student self-assessment and feedback regarding personal experience, control groups were not included. Thus, the majority of information considered herein is descriptive.

Two streams of data regarding undergraduate research will be presented and discussed. The first is survey responses from LSAMP-supported students at three of the four Alliance institutions who participated in UR from the 2007–2008 school year through 2015–2016 (TAMIU became an Alliance member in 2019). The second is findings from surveys of Alliance students who completed presentations about their research at the annual TAMUS LSAMP research symposium. The information from students who presented their research, while gathered in 2019, provides insight regarding the responses gathered from 2007 through 2016. They have been employed in this way as the students, while not the same parties, participated in the same processes by which UR was facilitated at and through the same institutions.

Method: Ethics Statement

Project implementation and assessment was completed in alignment with applicable federal and state regulations and guidelines for grant funded endeavors. All evaluation and project data gathering were completed in accordance with Institutional Review Board (IRB) approved protocols. A project IRB protocol was maintained at Texas A&M University in College Station, where the TAMUS LSAMP offices are located, and a separate evaluation IRB protocol was maintained at West Texas A&M University where the evaluation unit was housed.

Method: Data Analysis

Data analysis had both historic and current patterns. The original data or detailed summaries of participant responses were processed for project evaluation and later accessed to determine whether and how the material overlapped and could be combined for this article. Since this process varied slightly for each source, a brief account of what transpired will precede discussion of each form of data. Once identified, data sources were combined as applicable. Quantitative analysis completed was descriptive and tabular. Applicable qualitative data were aggregated from original sources and underwent an open coding process (Kolb, 2012). Four volunteers from the TAMUS LSAMP implementation team were provided de-identified sets of aggregated comments. A brief written set of instructions was provided to each coder and a Zoom call was held to allow them to ask questions of the evaluator who set up and participated in this process. The four independently developed codebooks were reconciled by the evaluator and the reconciled codebook was sent out for comment and approval by the coding team. Suggestions made regarding the reconciled codebook were addressed and resolved as a group.

Results

Survey Responses 2007–2016

From the 2007–2008 through the 2015–2016 school years, TAMUS LSAMP participants were asked a series of questions about their undergraduate research experience. The topics were based on the project goals and impacts of UR documented in the literature, some with broad support and others that have received less consideration.

The influence participating in UR has on interest in graduate school is a frequently investigated topic (Craney et al’s., 2011; Eagan et al., 2013; Conrad et al., 2015; Chang et al., 2016; Frederick et al., 2021). Impact on academic outcomes (Hunter et al., 2006; Lopatto, 2007; Linn et al., 2015), understanding of course content (Jonides et al., 1992; Kardash, 2000; Flaherty et al., 2017), performance in courses (Jonides et al., 1992; Lopatto, 2007; Sell et al., 2018), and career choice (Russell et al., 2007; Chang et al., 2016; Kilgo and Pascarella, 2016; Powers et al., 2018) have also received significant attention. Other topics are UR impact on the student’s interest in continued involvement with research (Seymour et al., 2004), interest in courses (Seymour et al., 2004), and confidence in choice of major (Seymour et al., 2004). The TAMUS LSAMP data include responses from 358 students, the vast majority of whom were URM students (90.4% of participants identified as URMs from the 2013–2014 through 2019–2020 school years). It considers all the topics listed above providing the potential to confirm efficacy of general support of UR for URM students at institutions of different types. The relative lack of diversity in STEM fields makes this a critical point as Carpi et al’s. (2017) suggest undergraduate research “serves as a powerful equalizer…to address the longstanding under-representation of minorities in the sciences” (p. 169).

The data summarized in Table 1 were gathered with surveys and for the first 3 years, the queries remained unchanged. All the queries addressed impact participation in UR had on the informant. In the 2010–2011 school year, two questions were eliminated and another was added and the response pattern was shifted from a four-point Likert scale (strongly disagree, disagree, agree, strongly agree) to a five-point Likert scale (strongly disagree, disagree, neither agree or disagree, agree, strongly agree). While these changes presented challenges to consideration of the nine years of data as one unit, the differences facilitate comparison of responses from two similar groups in respect to the same questions and elucidation of the earlier group of responses as the addition of a neutral response increased precision. Both groups were large, 175 students and 183 students, respectively, and the response rates were high, ranging from 71.4 to 100%. Even at the lowest levels, 71.4 and 84.2% response rates, the results meet a 95% confidence level with a 4.7 and 3.15 confidence interval, respectively (calculated at 50%), and at the highest level they include responses from every party asked to participate. Thus, even at its weakest points, these data are significant.

The responses can be rank ordered by level of agreement (combining responses of Agree and Strongly Agree).

– Interest in continuing with research (89.4%).

– Effect on academic life (83.2%).

– Increased interest in classes (82.2%).

– Increased understanding of course content (81.3%).

– Helped with career choice (75.4%).

– Improved performance in classes (61.8%).

– Increased interest in graduate school (61.2%).

– Increased confidence in choice of major (44.2%).

These outcomes align with findings from prior investigations (see details below), although most of the results other researchers published were for general student populations rather than URM students.

Russell et al. (2007) reported that UR “helped clarify students’ interest in research” (p. 548) and 89.4% of the TAMUS LSAMP informants noted interest in continuing with research. Jonides et al. (1992) reported UR participants completed more credit hours than same aged peers while Bowman and Holmes (2018) and Sell et al. (2018) reported association with higher GPAs, findings that align with the LSAMP survey’s general category, effect on academic life, which 83.2% of informants affirmed. Seymour et al. (2004) found “shifts in attitudes to learning” (p. 493) associated with UR participation and Flaherty et al. (2017) reported “increased…confidence in…perceived knowledge of science” (p. 701), both of which align with the impact on academic life just noted, the 82.2% of LSAMP informants who reported increased interest in classes, the 81.3% who reported increased understanding of course content, and the 61.8% stating their performance increased in classes. Increased confidence in choice of major had the lowest affirmation level among TAMUS LSAMP informants, 44.2%, but is related to career choice and interest in graduate school which have strong support in the literature and moderate to high affirmation rates for TAMUS LSAMP, 75.4 and 61.2% respectively.

Craney et al’s. (2011) state that UR had a “key specific outcome…clarification and reinforcement of a graduate school career path” (p. 107) while also being associated with “more favorable attitudes toward research as a career option” (p. 107). Flaherty et al. (2017) found contribution toward “clarification of career goals” (p. 701) which parallels earlier results from Seymour et al. (2004) and Thiry et al. (2011). Important for this consideration, Carpi et al’s. (2017) noted that UR “increases career ambitions for underrepresented students in STEM” (p. 169) and “is seen to have a transformative effect for many students at MSIs” (p. 169). The TAMUS LSAMP informant responses align with the impacts noted by Seymour et al. (2004), Craney et al’s. (2011), Thiry et al. (2011), Carpi et al’s. (2017), and Flaherty et al. (2017).

A related construct, UR impact on interest in graduate school, also has strong support in the literature. Lopatto (2007), Eagan et al. (2013), and Chang et al. (2016) all note impact with Eagan et al. stating “participation…significantly improved students’ probability of indicating plans to enroll in a STEM graduate program” (p. 683). Borrego et al. (2018) associated this with increased self-efficacy; “for every one-unit increase in students’ scores on the Self-efficacy scale, they were over eight times more likely to plan to enroll in a master’s program relative to not attending graduate school, and they were 13 times more likely to enroll in a Ph.D. program relative to not attending graduate school” (p. 154). The query listed at the bottom of Table 1 provides insight into the extent to which UR contributed to interest in graduate school for TAMUS LSAMP respondents. A total of 63.6% of informants indicated it contributed “A lot” or “A great deal.” Only 4.3% of respondents indicated it did not contribute.

That all of the above occurred for TAMUS LSAMP students who participated in UR is notable. That all the impacts applied to a population that was over 90% URM students, is highly encouraging and supports Carpi et al’s. (2017) assertion that UR can be “a powerful equalizer…to address the longstanding under-representation of minorities in the sciences” (p. 169).

Survey Responses 2019

The TAMUS LSAMP project sponsors an educational, networking, and research presentation symposium on an annual basis. The 2018–2019 symposium was held in the spring of 2019 and the 2019–2020 symposium was held in conjunction with the TAMUS Annual Pathways Student Research Symposium in the late fall of 2019. All students who had been supported by LSAMP to complete UR and who presented about their project at one of the two symposia were surveyed. There were 67 students representing all four Alliance institutions. A total of 49 submitted survey responses. This meets a 95% confidence level with a 7.31 confidence interval when calculated at 50%. Figure 1 provides a breakdown of gender and percentage of URM participants by campus for 2019 survey respondents.

Demographics gathered about these students confirmed that the sample had nearly the same distribution of females and males as the total population. All Alliance institutions were represented although students from TAMU, the largest group, were the least likely to respond (33% participation), with TAMIU at 65%, PVAMU 80%, and TAMUCC 100% participation. The counts of respondents per institution ranged from seven (TAMU) to 16 (PVAMU). There were 23 male and 26 female informants (males were undersampled; 46.9% of sample and 58.8% of population). There were 28 students who identified as Hispanic/Latino and 21 who did not. Five of the Hispanic students identified with more than one racial group and the remaining students’ racial identities were African American (n = 13), Asian (n = 4), Hispanic/Latino (n = 21), Native Hawaiian/Pacific Islander (n = 1), and White (n = 4) with one informant selecting “I do not care to answer.” The only ethnic or racial category that was not balanced across the institutions was African American as 12 of 13 students describing themselves this way attended PVAMU. Most of the informants were upper-level undergraduates. Four were sophomores, seven were juniors, and 38 were seniors. Thus, 81.6% of informants identified as URM students with the sample distributed across all four institutions.

The questions asked on the survey address topics about which there is limited evidence in the literature, especially in respect to URM populations. These are length and continuity of experience in research, number of investigations the students contributed to, the level of independence experienced in research settings, whether the UR experience had a perceivable training pattern, and the tasks in which the student was engaged. There is also evidence regarding impact on future plans with the potential to shed additional light on the topics in the 2007–2016 data.

Length and Continuity of Experience in Research

Respondents were asked to select all that apply from a list of options about when their experience with research began. Their options started with a statement that they had research experience prior to coming to college and included each year of undergraduate study.

– Three (3) noted experience prior to attending college.

– Four (4) stated they had research experience as freshmen.

– Seventeen (17) indicated research experience as sophomores.

– Seventeen (17) noted research experience in their junior year.

– Twenty-two (22) had research experience during their senior year.

– Approximately 20% (n = 10) of the students reported involvement with research in two or more years of undergraduate study.

The three students who reported having research experience prior to entering college may have misunderstood the question as they did not report continuing experience following that by selecting additional responses to the question. It is possible that they understood the question to be asking when their experience in research began.

Number of Investigations to Which Students Contributed as Undergraduates

The LSAMP Symposium presenters were asked about the number of research investigations to which they had contributed during their undergraduate career. This was a multiple-choice question for which one answer could be selected. The answers ranged from one study to four or more. Responses occurred in each category with 17 indicating experience with one research project, 16 two projects, 10 three projects, and six reporting work on four or more projects.

The three students who reported research involvement before college but not during college also indicated they contributed to two, three, and four or more studies. While the number of studies may be a function of the lab or faculty mentor rather than the student, this pattern appears to support the idea that the students misunderstood the earlier question about when they were involved with research thinking they were being asked for a “start date” rather than periods of involvement. The response pattern also suggests that URM students who engage in UR become motivated to continue in UR as 65.3% of respondents contributed to two or more studies. Thiry et al. (2012) note the significance of the LSAMP student reports regarding length, continuity, and number of investigations students worked on during UR when they stated their “findings suggest that students benefit from multi-year UR experiences” (p. 260).

Relative Level of Independence in Research

Respondents were asked to respond to a multiple-choice question about the level of independence they had experienced in research. To the best of the authors’ knowledge, this query is unique to the TAMUS LSAMP data set. Students were permitted to select all that applied to their circumstances from a list of descriptive statements. The choices presented are listed below with the number of responses in parentheses. Two students did not respond to this question.

– As a student worker completing basic tasks (21 students).

– As a student who was provided guidance to autonomously complete tasks (19 students).

– As a student member of a research team in which I could contribute ideas (20 students).

– As an independent researcher operating with assistance from a faculty member (18 students).

– As a completely autonomous researcher defining and completing my own projects (four students).

The four students noting autonomous research were from three different institutions. Many of the students who reported earlier involvement with research also reported multiple levels of responsibility/independence (n = 22). Students reporting multiple forms of responsibility were upper-level undergraduates, with one exception (one sophomore, two juniors, 19 seniors). These patterns suggest that faculty supervisors facilitate increased levels of independence in processes as students gain experience in research.

Relationship of Research Experience to a Training Pattern

Informants were asked about the relationship of their research experience to a set of education goals or a training pattern, specifically whether they perceived that a deliberate training pattern had been enacted. There were three possible responses and students could select more than one to allow for expanded responsibility or involvement in several projects. The prompts, with associated counts of responses, follow. One student chose not to respond to this question.

– Been primarily at one level of responsibility (11 students).

– Involved learning different tasks and having several areas of responsibility for a project but these were assigned based on project needs rather than my educational goals (30 students).

– Involved a sequence of steps and variety of activity that was deliberately planned as training pattern (13 students).

Five students selected more than one response. Two indicated primarily one level of responsibility and involvement in different tasks as assigned. Two others noted different tasks being assigned without reference to personal goals and a deliberately planned sequence. One selected all three options. These persons were all upper-level students with involvement in two or more studies. While material presented above suggests increases in independence in research as student experience increases, this set of responses indicates approximately 75% of the students did not perceive a deliberate training pattern as the basis of their experience. Craney et al’s. (2011) reported “research advisor[s] “provided needed instruction/direction” for 79% of the participants” (p. 103). Craney et al’s.’s (2011) prompt is superior to that used in the TAMUS LSAMP survey as it measures provision of needed assistance rather than perception of a structured training program. The TAMUS survey results, while informative regarding student perceptions and suggesting that faculty could provide more or more explicit explanation of the purposes and process in training the students undertaking certain responsibilities, does not address the more important issue, provision of appropriate guidance/assistance.

Practical Experience Achieved in Research Settings

Respondents were also asked to select all responses that applied to their experience from a list of 21 types of engagement in research projects. The 20th was “None of the above” and the 21st was “Other” followed by the opportunity to provide an alternate response in a text entry box. Figure 2 lists all the fixed-answer responses for which submissions were received and the number of responses in each category. No student selected “none of the above” and only one selected “Other” and submitted the explanation “prototyping.”

Table 2 lists the responses of the students in groups formed based on the natural breaks in the response counts. There were four places at which there were differences of three or more points creating a five-tier pattern. Statements for which there was the same number of responses are listed in the order in which they occurred on the survey.

Several generalizations are possible based on the material presented in Figure 2 and Table 2. First, it appears the students were involved, as undergraduates, in many important aspects of research projects. Second and as an extension of the first, they were receiving a broad introduction to research. Third, the students were more likely to report involvement in commonly understood major elements of research like designing the question, completing statistical analysis, and summarizing research findings than in the detailed and skill dependent activities like collecting physical samples and conducting interviews or focus groups (although the latter may have been impacted by the students being STEM majors). The synopsis of qualitative findings confirms these patterns. They chronicle and parallel learning in 12 of 13 areas reported in response to a quantitative query (Figure 3).

FIGURE 3

Figure 3. Learning reported regarding research processes.

Comparing the areas in which students reported the most learning (Figure 3), those with 25 or more responses of “Learned a lot,” to the most frequently reported research elements included in the student experiences (Figure 2), those with 26 or more responses, results in a thumbnail sketch of a UR experience in TAMUS LSAMP. It also suggests that the pattern is effective as most students reported they “Learned a lot” or “Gained some experience” in the processes noted. The thumbnail sketch of a TAMUS LSAMP UR experience includes the following elements: (1) identifying a topic, (2) refining the research question, (3) designing an investigative method, (4) completing literature review, (5) digital modeling, (6) performing quantitative analysis, (7) summarizing findings from analysis, (8) drawing conclusions, (9) preparing presentation materials, and (10) traveling to present findings.

Research Experience Impact on Student Plans

A prompt “My involvement with research has impacted my thinking about” was followed by four possible responses. Informants could select all that applied. The categories with counts of responses submitted are listed below.

– College course selection (selected by 19 students).

– Identifying a mentor or person from whom I can solicit advice (selected by 29 students).

– My career goals (selected by 39 students).

– My intentions regarding graduate school (selected by 32 students).

The relative academic level of the students in the sample, 45 of the 49 were juniors or seniors, may have impacted the course selection responses. Students at higher academic levels are more likely to have reached a more degree-specific and less flexible set of course options.

The ability to identify a mentor who can provide advice is beneficial to students (Craney et al’s., 2011). Frederick et al. (2021) assert “the benefits conferred through mentoring relationships with faculty are among the most important advantages undergraduate students gain through co-curricular research” (p. 2). At the opposite end of the spectrum, Powers et al. (2018) point out that “negative experiences (such as…poor mentors) caused some students to change career or education plans” (p. 3). That 59.2% of informants felt they had found, through involvement with UR, a person who could and would provide them advice is, therefore, a positive outcome. Response patterns for the other items, career goals and graduate school intentions, parallel findings from the 2007 to 2016 data described above and align with impacts noted there from the work of other researchers (career goals: Seymour et al., 2004; Craney et al’s., 2011; Thiry et al., 2011; Carpi et al’s., 2017; Flaherty et al., 2017; graduate school: Lopatto, 2007; Eagan et al., 2013; Chang et al., 2016; Borrego et al., 2018).

Learning Experienced Through Research Participation

The survey respondents were also asked to rate 13 statements as: (1) an area in which you learned a lot, (2) an area in which you gained some knowledge or experience, or (3) an area in which you did not learn much or gained little to no experience. Figure 3 shows the response pattern for this question.

The counts in Figure 3 do not total 49 as some students did not rate all the prompts and several completed the question incorrectly.

The responses from TAMUS LSAMP participants to this question reflect patterns in the literature. Craney et al’s. (2011) noted development in three skill areas, communication, problem-solving, and forming a research question. Lopatto (2007) reported advancements in understanding the literature, data analysis, communication skills. Especially relevant, Frederick et al. (2021) found UR “strengthened…research skills” (p. 5) in a study focused on Hispanic/Latinx students. TAMUS LSAMP findings proved similar to these and Kardash’s (2000) findings in which “the extent to which14 research skills were enhanced” (p. 191) was addressed with some advancing to a greater extent than others.

Most Valuable About Research Participation

One of the final questions asked of the Symposium presenters was what had been most valuable about their experience in undergraduate research. The responses were coded by a group of four TAMUS LSAMP personnel. Summaries of the respondents’ statements based on codings are listed below. These parallel and support the findings described above and confirm that the impacts of UR described in the literature, also noted above, were replicable with a predominantly URM student population at four distinct universities.

– UR increased interest in research and graduate school.

– UR increased confidence and self-efficacy particularly in research settings and in respect to being a worthy graduate school candidate.

– UR provided opportunities to apply classroom learning through active involvement in real world settings, to be mentored, to have role models, to network with faculty and student peers, to participate in related workshops, and to present research findings.

– UR facilitated development and honing of personal and professional skills: (1) being responsible, (2) being organized, (3) collaborating, (4) planning, discussing, and executing research activity, (5) technical writing, and (6) planning and completing research presentations.

To summarize, participants saw the value of participating in UR as expanded perspectives, improved motivation, receiving a preview of “what grad school is like,” receiving insight into ways to fund graduate school, and learning about a variety of STEM career paths. Of the 67 Symposium undergraduate attendees, at least 61 (91.0%) have graduated with bachelor degrees. At least 19 of these 61 graduates (31.1%) have enrolled in graduate school.

Summary Discussion

Findings suggest that UR sponsored through TAMUS LSAMP achieved its “potential as a powerful programmatic and pedagogical tool” (Kilgo and Pascarella, 2016, p. 575). This is especially the case as only four of the studies cited herein specifically targeted understanding UR impacts for URM students, yet the relevant findings from all studies were replicable in an LSAMP setting and at four different universities indicating potential to generalize UR outcomes in the literature to URM audiences.

Some researchers of UR have reported different outcomes by gender (Kardash, 2000) while others have not (Lopatto, 2007; Bowman and Holmes, 2018), the 2019 data set which included demographic information, showed no significant differences in effect by gender, ethnicity, or race. All the benefits of UR were uniformly realizable for all students in the 2019 sample highlighting UR’s potent to serve “as a powerful equalizer…to address the longstanding under-representation of minorities in the sciences” (Carpi et al’s., 2017, p. 169) as well as that of females.

Seven of the eight perceived benefits of UR assessed in the 2007 to 2016 data were reported by over 60% of the respondents. Six of these benefits, (1) increased interest in continuing engagement with research, (2) increased interest in classes, (3) increased understanding of classes, (4) increased performance in classes, (5) increased interest in graduate school, and (6) better informed career choices, were also present in the quantitative and qualitative data sets from 2019. These findings affirm that the general pattern of facilitation of UR was effective in producing the benefits noted in the literature and for students at a variety of institution types who were predominantly URMs. These perceived benefits were also reported by sophomores, juniors, and seniors, so the age of the student does not appear to limit potential for impact (Preuss et al., 2021). This is valuable information as the simple pattern enacted can be replicated at any institution of higher education.

Most of the informants began involvement in undergraduate research as juniors and seniors, but this could be a function of the LSAMP recruiting patterns. For example, TAMUCC focuses their efforts on students in their junior and senior years. Even with engagement beginning in the last 2 years of undergraduate study for most informants, UR involvement extended across more than 1 year for many. Their experiences were distributed across four different types of engagement, from performing basic research tasks to being fully autonomous, although autonomous activity was limited to 8% of respondents. Most frequently, the students learned tasks as needed for a project but approximately one quarter reported perceiving a “deliberately planned sequence of steps and variety of activity” as a training pattern. That, however, is not the same as there not having been a deliberate training pattern and use of Craney et al’s. (2011) prompt, or something similar, about receiving needed assistance and guidance would have been a preferable query.

The elements of student engagement with research during their UR experience are reported in Figure 2 and Table 2. Linn et al. (2015) state that “the ideas that students learn (in UR) are often isolated or fragmented rather than integrated and coherent…(and) Rigorous research is needed to identify the ways to design research experiences so they promote integrated understanding” (p. 628). They suggest that “powerful and generalizable assessments that can document student progress, help distinguish effective and ineffective aspects of the experiences, and illustrate how students interpret the research experiences they encounter” (p. 628) are needed. While TAMUS LSAMP did not attempt to create generalizable assessments, the outcomes reported by students do provide evidence regarding student progress, where the greatest learning took place indicating in which areas general facilitation of UR was effective, and the self-reported data provide insight into how the students interpreted their experience. Beyond having between 61 and 95% of respondents reporting learning in 12 of 13 areas (Figure 3), students: (1) were involved, as undergraduates, in many important aspects of research projects, (2) received a broad introduction to research, and (3) were more likely to report involvement in commonly understood major elements of research. They saw the value of participating in UR as expanded perspectives, improved motivation, receiving a preview of “what grad school is like,” receiving insight into ways to fund graduate school, and learning about a variety of STEM career paths.

Comparison of the areas in which students reported the most learning (Figure 3), those with 25 or more responses of “Learned a lot,” to the most frequently reported research elements included in the student experiences (Figure 2), those with 26 or more responses, results in a thumbnail sketch of a UR experience in TAMUS LSAMP. It also suggests that the pattern is effective as most students reported they “Learned a lot” or “Gained some experience” in the processes noted. These could be the basis of further investigation at an increased level of rigor as suggested by Linn et al. (2015). The thumbnail sketch of a TAMUS LSAMP UR experience includes the following elements: (1) identifying a topic, (2) refining the research question, (3) designing an investigative method, (4) completing literature review, (5) digital modeling, (6) performing quantitative analysis, (7) summarizing findings from analysis, (8) drawing conclusions, (9) preparing presentation materials, and (10) traveling to present findings. This pattern, and the reported gains in learning/skill, align with results reported in the literature (Kardash, 2000; Lopatto, 2007; Craney et al’s., 2011; Frederick et al., 2021), but demonstrate potential to generalize them to UR completed by URM students and conducted at Minority-Serving Institutions.

The ability to identify a mentor is another commonly cited outcome from involvement with UR (Craney et al’s., 2011; Linn et al., 2015; Powers et al., 2018). Frederick et al. (2021) asserted “the benefits conferred through mentoring relationships with faculty are among the most important advantages undergraduate students gain through co-curricular research” (p. 2). This benefit was realized for many of the LSAMP participants as 59.2% of informants felt they had found, through involvement with UR, a person who could and would provide them advice.

Practical Implications

TAMUS LSAMP seeks to serve underrepresented students at four Alliance institutions. These fit in three different Carnegie classifications. The Very High Research category institution, TAMU, is a predominantly White institution. The other partnering institutions are an HBCU and two HSIs. The outcomes described above existed for students of each institution type. It is notable that there were no differences in outcome by institution type, gender, ethnicity, or race. This confirms assertions made in Laursen et al.’s (2010) and other sources (American Association of Colleges and Universities, n.d.; Kuh and O’Donnell, 2013) about the efficacy of undergraduate research in general and for students from underrepresented groups. It also suggests that the general facilitation pattern enacted by TAMUS LSAMP would be effective at many other institutions given uniform effects over a 13-year span at several universities with differing Carnegie classifications and student populations. Given the need to expand the STEM workforce in the United States and the limited diversity in the existing STEM workforce (Bayer Corporation, 2012; Linley and George-Jackson, 2013; Collins, 2018; National Science Foundation, 2018), the ability, demonstrated by TAMUS LSAMP data, of UR to act “as a powerful equalizer…to address the longstanding under-representation of minorities in the sciences” (Carpi et al’s., 2017, p. 169) as well as that of females is of critical importance.

Limitations

The data discussed were student self-reports and control group data were not gathered thus the degree to which the outcomes vary from those for students not participating in the LSAMP programming is unknown. Demographic information included in the 2007–2016 data set could not be reintegrated with the survey responses eliminating the potential to disaggregate by gender, ethnicity, race, home institution, etc. Thus, while the informant group in 2007–2016 was representative of the pool of participants based on high response rates, comparison of reported impact between demographic subsets for these data was not possible.

Data Availability Statement

The datasets presented in this article are not readily available because the data include responses from a small number of participants. This could make information individually identifiable. Requests to access the datasets should be directed to MP, exquiri.michael@gmail.com.

Ethics Statement

The studies involving human participants were reviewed and approved by Texas A&M University Institutional Review Board. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

MP compiled applicable data from evaluation materials, completed the quantitative data analysis and secondary research, planned and led the group analysis of qualitative data, and drafted the manuscript. SM and JA coordinated the data gathering from partnering institutions, completed analysis of those data, helped to plan the presentation, and provided comments on drafts. KB-P, KW, SW, PO, FP, JM, and MR coordinated the site-specific activity, commented on plans for the manuscript, and gathered the data from PVAMU and TAMUCC. JK and HL planned and conducted the evaluation processes from which the data described were drawn. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the National Science Foundation under Grants Nos. HRD-0703290, 1304975, and 1911375.

Author Disclaimer

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Conflict of Interest

MP was employed by Exquiri Consulting, LLC, and provided external evaluation services for TAMUS LSAMP.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to acknowledge the assistance of Yahaira Franco and Jessica Hernandez from Texas A&M International University with coding evaluation themes presented in the manuscript. We would also like to acknowledge Mahmoud Khasawneh, John Kilburn, and Jared Dmello for their leadership and involvement with the LSAMP efforts on the TAMIU campus and thank Kilburn for comments provided regarding this manuscript.

References

American Association of Colleges and Universities. (n.d.). High impact educational practices. Available online at: https://www.aacu.org/node/4084 (Accessed on Jan 26, 2021)

Google Scholar

Bayer Corporation (2012). Bayer facts of science education XV: a view from the gatekeepers – STEM department chairs at Americas top 200 research universities on female and underrepresented minority undergraduate STEM students. J. Sci. Educ. Technol. 21, 317–324. doi: 10.1007/s10956-012-9364-1

CrossRef Full Text | Google Scholar

Borrego, M., Knight, D. B., Gibbs, K., and Crede, E. (2018). Pursuing graduate study: Factors underlying undergraduate engineering students’ decisions. J. Eng. Educ. 107, 140–163. doi: 10.1002/jee.20185

CrossRef Full Text | Google Scholar

Bowman, M. H., and Stage, F. K. (2002). Personalizing the goals of undergraduate research. J. Coll. Sci. Teach. 32, 120–125.

Google Scholar

Bowman, N. A., and Holmes, J. M. (2018). Getting off to a good start? First-year undergraduate research experiences and student outcomes. Higher Educ. 76, 17–33. doi: 10.1007/s10734-017-0191-4

CrossRef Full Text | Google Scholar

Carpi, A., Ronan, D. M., Falconer, H. M., and Lents, N. H. (2017). Cultivating minority scientists: Undergraduate research increases self-efficacy and career ambitions for underrepresented students in STEM. J. Res. Sci. Teach. 54, 169–194. doi: 10.1002/tea.21341

CrossRef Full Text | Google Scholar

Chang, T., Armstrong, W., Danube, C., McKinney, K., and Reed, M. (2016). “Undergraduate research participation: Predictors and relationship with research careers,” in 6^th World Conference on Psychology, Counseling, and Guidance, (San Diego: University of California).

Google Scholar

Collins, K. H. (2018). Confronting color-blind STEM talent development: Toward a Contextual model for Black student STEM identity. J. Adv. Acad. 29, 143–168. doi: 10.1177/1932202X18757958

CrossRef Full Text | Google Scholar

Conrad, L. F., Auerbach, J. L., and Howard, A. (2015). “The impact of a robotics summer undergraduate research experience on increasing the pipeline to graduate school,” in Paper presented at 2015 American Society for Engineering Education (ASEE) Annual Conference & Exposition, Seattle, WA. doi: 10.18260/p.24875

CrossRef Full Text | Google Scholar

Craney, C., McKay, T., Mazzeo, A., Morris, J., Prigodich, C., and de Groot, R. (2011). Cross-discipline perceptions of the undergraduate research experience. J. Higher Educ. 82, 92–113. doi: 10.1353/jhe.2011.0000

PubMed Abstract | CrossRef Full Text | Google Scholar

Eagan, M. K., Hurtado, S., Chang, M. J., and Garcia, G. A. (2013). Making a difference in science education: The impact of undergraduate research programs. Am. Educ. Res. J. 50, 683–713. doi: 10.3102/0002831213482038

PubMed Abstract | CrossRef Full Text | Google Scholar

Excelencia in Education. (n.d.). 25 Years of HSIs. Available online at: http://bit.ly/25YrsHSIMaps (Accessed on Dec 6, 2021)

Google Scholar

Flaherty, E. A., Walker, S. M., Forrester, J. H., and Ben-David, M. (2017). Effects of course-based undergraduate research experiences (CURE) on wildlife students. Wildlife Soc. Bull. 41, 701–711. doi: 10.1002/wsb.810

CrossRef Full Text | Google Scholar

Frederick, A., Grineski, S. E., Collins, T. W., Daniels, H. A., and Morales, D. X. (2021). The emerging STEM paths and science identities of Hispanic/Latinx college students: Examining the impact of multiple undergraduate research experiences. CBE Life Sci. Educ. 20, 1–10. doi: 10.1187/cbe.20-08-0191

PubMed Abstract | CrossRef Full Text | Google Scholar

Garcia, S. J., Alves, M. C., Pariyothorn, M., Myint, A., and Hardman, A. K. (2017). “ELCIR - engineering learning community introduction to research: a research and global experience program supporting first generation low incoming underrepresented minority students,” in Proceedings of the American Society for Engineering Education (ASEE) Annual Conference & Exposition, Columbus, OH. doi: 10.18260/1-2-28209

CrossRef Full Text | Google Scholar

Graham, J. M., Caso, R., and Rierson, J. (2001). “The effect of the Texas A&M University system amp on the success of minority undergraduates in engineering: a multiple outcome analysis,” in Paper presented at 2001 American Society for Engineering Education (ASEE) Annual Conference & Exposition, Albuquerque, NM. doi: 10.18260/1-2-9161

CrossRef Full Text | Google Scholar

Hunter, A. B., Laursen, S. L., and Seymour, E. (2006). Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Sci. Educ. 91, 36–74. doi: 10.1002/sce.20173

CrossRef Full Text | Google Scholar

Indiana University Center for Postsecondary Research. (n.d.). The Carnegie Classification of Institutions of Higher Education. Available online at: https://carnegieclassifications.iu.edu/index.php (Accessed on Jan 26, 2021).

Google Scholar

Jonides, J., von Hippel, W., Lerner, J. S., and Nagda, B. A. (1992). “Evaluation of minority retention programs: The undergraduate research opportunities program at the University of Michigan,” in 100^th Annual Meeting of the American Psychology Association. (ERIC Document Service Reproduction No. ED 355 309), (Massachusetts: American Psychology Association).

Google Scholar

Kardash, C. M. (2000). Evaluation of undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. J. Educ. Psychol. 92, 191–201. doi: 10.1037/0022-0663.92.1.191

CrossRef Full Text | Google Scholar

Kilgo, C. A., and Pascarella, E. T. (2016). Does independent research with a faculty member enhance four-year graduation and graduate professional degree plans? Convergent results with different analytical methods. Higher Educ. 71, 575–592. doi: 10.1007/s10734-015-9925-3

CrossRef Full Text | Google Scholar

Kolb, S. M. (2012). Grounded theory and the constant comparative method: valid research strategies for educators. J. Emerg. Trends Educ. Res. Pol. Stud. 3, 83–86. doi: 10.1093/acprof:oso/9780199922604.003.0004

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuh, G. D., and O’Donnell, K. (2013). Ensuring Quality and Taking High Impact Practices to Scale. Washington, DC: American Association of Colleges and Universities.

Google Scholar

Laursen, S., Hunter, A.-B., Seymour, E., Thiry, H., and Melton, G. (2010). Undergraduate Research in the Sciences: Engaging Students in Real Science. San Francisco: John Wiley & Sons, Inc.

Google Scholar

Linley, J. L., and George-Jackson, C. E. (2013). Addressing underrepresentation in STEM fields through undergraduate interventions. New Dir. Student Serv. 144, 97–102. doi: 10.1080/09553002.2021.1988182

PubMed Abstract | CrossRef Full Text | Google Scholar

Linn, M. C., Palmer, E., Baranger, A., Gerard, E., and Stone, E. (2015). Undergraduate research experiences: Impacts and opportunities. Science 347, 627–634. doi: 10.1126/science.1261757

PubMed Abstract | CrossRef Full Text | Google Scholar

Lopatto, D. (2007). Undergraduate research experiences support science career decisions and active learning. CBE Life Sci. Educ. 6, 297–306. doi: 10.1187/cbe.07-06-0039

PubMed Abstract | CrossRef Full Text | Google Scholar

Merriweather, S. P., Lamm, H. A., Walton, S. D., Butler-Purry, K. L., Kelley, J., Thomasson, K. E., et al. (2017). “TAMUS LSAMP project: 25 years of success - finding and implementing best practices for URM STEM Students,” in Paper Presented at 2017 American Society for Engineering Education (ASEE) Annual Conference & Exposition, Columbus, OH. doi: 10.18260/1-2-28904

CrossRef Full Text | Google Scholar

National Center for Education Statistics [NCES]. (2020a). College navigator: Texas A & M International University [Data set]. Available online at: https://nces.ed.gov/collegenavigator/?q=texas+a+%26+m+international+university&s=all&id=226152#enrolmt

Google Scholar

National Center for Education Statistics [NCES]. (2020b). College navigator: Texas A & M University – Corpus Christi [Data set]. Available online at: https://nces.ed.gov/collegenavigator/?q=texas+a+%26+m+university+-+corpus+christi&s=all&id=224147#enrolmt

Google Scholar

National Center for Education Statistics (2018). Enrollment and Degrees Conferred in Hispanic-Serving Institutions, by Institution Level and Control, Percentage Hispanic, Degree Level, and Other Specific Characteristics: Fall 2018 and 2017-2018. Available online at: https://nces.ed.gov/programs/digest/d19/tables/dt19_312.40.asp?current=yes (accessed August 12, 2021).

Google Scholar

National Science Foundation (2018). Louis Stokes Alliances for Minority Participation (LSAMP) (nsf20590). Available online at: https://www.nsf.gov/pubs/2020/nsf20590/nsf20590.htm (Accessed on Dec 2, 2021)

Google Scholar

Perez, D. II, Garcia-Louis, C., Ballysingh, T. A., and Martinez, E. (2018). “Advancing an anti-deficit achievement framework for Latinx/a/o college students,” in Latinx/a/os in Higher Education: Exploring Identity, Pathways, and Success, eds A. E. Batista, S. M. Collado, and D. Perez II (Washington, DC: National Association of Student Personnel Administrators).

Google Scholar

Powers, K., Chen, H. L., Prashad, K. V., Gilmartin, S. K., and Shephard, S. (2018). “Exploring how engineering internships and undergraduate research experiences inform and influence college students’ career decisions and future plans,” in 2018 ASEE Annual Conference and Expo (Paper No. 23905), (Minneapolis: ASEE).

Google Scholar

Preuss, M., Merriweather, S., Avila, J., Butler-Purry, K., Watson, K., Walton, S., et al. (2021). The Impacts of Global Research and International Educational Experiences on Texas A&M University System LSAMP Participants. Front. Educ. 6:674772. doi: 10.3389/feduc.2021.674772

CrossRef Full Text | Google Scholar

Preuss, M. D., Merriweather, S. P., Walton, S. D., and Butler-Purry, K. L. (2020). “International Research Exposure: Impact on Early-Career, Undergraduate Engineering Students,” in Proceedings of IConSES 2020–International Conference on Social and Education Sciences, eds V. Akerson and I. Sahin (Chicago: ISTES Organization), 1–15. doi: 10.36941/jesr-2021-0046

CrossRef Full Text | Google Scholar

Russell, S. H., Hancock, M. P., and McCullough, J. (2007). Benefits of undergraduate research experiences. Science 316, 548–549. doi: 10.1126/science.1140384

PubMed Abstract | CrossRef Full Text | Google Scholar

Schmidt, P. (2003). The Stakes for Minority Groups in the Supreme Court’s Review of Affirmative Action. Chron. Higher Educ. 49:39.

Google Scholar

Sell, A. J., Naginey, A., and Stanton, C. A. (2018). The impact of undergraduate research on academic success. Scholar. Pract. Undergr. Res. 1, 19–29. doi: 10.18833/spur/1/3/8

CrossRef Full Text | Google Scholar

Seymour, E., Hunter, A. B., Laursen, S. L., and Deantoni, D. (2004). Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Sci. Educ. 88, 493–534. doi: 10.1002/sce.10131

CrossRef Full Text | Google Scholar

Thiry, H., Laursen, S. L., and Hunter, L. B. (2011). What experiences help students become scientists? A comparative study of research and other sources of personal and professional gains for STEM undergraduates. J. Higher Educ. 82, 357–388. doi: 10.1353/jhe.2011.0023

PubMed Abstract | CrossRef Full Text | Google Scholar

Thiry, H., Weston, T. J., Laursen, S. L., and Hunter, L. B. (2012). The benefits of multi-year research experiences: Differences in novice and experienced students’ reported gains from undergraduate research. CBE Life Sci. Educ. 11, 260–272. doi: 10.1187/cbe.11-11-0098

PubMed Abstract | CrossRef Full Text | Google Scholar

U.S. Department of Education (2020). White House Initiative on Historically Black Colleges and Universities. Available online at: https://sites.ed.gov/whhbcu/one-hundred-and-five-historically-black-colleges-and-universities/ (Accessed on Jan 20, 2021).

Google Scholar

Watson, K., and Froyd, J. (2007). Diversifying the US engineering workforce: A new model. J. Eng. Educ. 96, 19–32. doi: 10.1002/j.2168-9830.2007.tb00912.x

CrossRef Full Text | Google Scholar