A utility value intervention to support student engagement in online upper-level undergraduate courses

Shillington, Katie J.; Lo, Stanley M.; Hobbie, Lawrence

doi:10.3389/feduc.2025.1636446

ORIGINAL RESEARCH article

Front. Educ., 04 September 2025

Sec. STEM Education

Volume 10 - 2025 | https://doi.org/10.3389/feduc.2025.1636446

A utility value intervention to support student engagement in online upper-level undergraduate courses

Katie J. Shillington^1,2,3^*^†

Stanley M. Lo^4,5,6^†

Lawrence Hobbie⁷^†

¹Department of Neurobiology, University of California, San Diego, La Jolla, CA, United States
²Center for Empathy and Social Justice in Human Health, T. Denny Sanford Institute for Empathy and Compassion, University of California, San Diego, La Jolla, CA, United States
³Department of Psychology, Faculty of Science, Wilfrid Laurier University, Waterloo, ON, Canada
⁴Department of Cell and Developmental Biology, University of California, San Diego, La Jolla, CA, United States
⁵Joint Doctoral Program in Mathematics and Science Education, University of California, San Diego, La Jolla, CA, United States
⁶Research Ethics Program, University of California, San Diego, La Jolla, CA, United States
⁷Department of Biology, Adelphi University, Garden City, NY, United States

Introduction: Utility value interventions are assignments designed to increase student motivation by helping them find personal value in what they are learning. Previous studies have found utility value interventions to lead to improved student outcomes. The purpose of this study was to investigate the impact of a utility value (UV) intervention on the academic outcomes of undergraduate students in online courses, compared to students in a control condition.

Methods: A total of 1,243 students in upper-level online biology courses participated and were randomized either to UV (n = 589) or control (n = 654) conditions.

Results: The intervention had a positive effect, as UV participants engaged with and participated in their courses significantly more than students in the control condition. Utility value participants also wrote significantly longer essays, and used more pronouns, social process words, and cognitive process words than students in the control condition. However, UV participants did not have higher course grades than students in the control condition. Additionally, underrepresented minority (URM) status was a significant predictor of course grade; non-URM students had significantly higher course grades than URM students.

Discussion: The utility value intervention did have a positive effect on student course engagement but, contrary to most previous studies, did not lead to higher student course grades, perhaps because these classes were online, because they were upper-level instead of introductory, or because the study was conducted during the COVID-affected spring 2020 quarter. Findings from this study can help institutions decide whether to adopt UV interventions in online courses.

Introduction

How can educators best help their students learn? One approach has been to try to understand at a fundamental level how people learn (i.e., change their concepts), with the hope that this understanding will give insights into how to most effectively facilitate learning. Posner et al. (1982) suggested that conceptual change can happen when an individual becomes dissatisfied with the current concept and learns about a potential replacement concept that is plausible, intelligible, and fruitful. Conceptual change in this model is viewed entirely as a cognitive process, and this model has thus been called “cold” conceptual change. Subsequently, researchers recognized that psychological and social factors can play important roles in facilitating or preventing conceptual change, leading to “warm” models of conceptual change (Dole and Sinatra, 1998; Gregoire, 2003; Sinatra, 2005). For example, in the recently proposed Dynamic Model of Conceptual Change (Nadelson et al., 2018), social and psychological factors are postulated to affect a learner's motivation to engage with a new concept; a learner's deep engagement with a concept in turn is essential for real conceptual change to take place. Situated Expectancy-Value Theory proposes that a learner's motivation to engage in an academic task is determined by a combination of the learner's expectancy for success (self-efficacy) and the task value that the learner places on the subject (Eccles and Wigfield, 2020; Wigfield and Eccles, 2000). Task value is differentiated into four components: utility value (usefulness for near- or long-term goals), attainment value (importance of a task because it is aligned with one's identity), intrinsic value (interest and enjoyment), and cost (especially the effort involved).

Researchers have empirically tested warm models of conceptual change by investigating whether targeting social and psychological processes with brief interventions can facilitate learning. These social-psychological interventions have in general shown good success in improving student outcomes in both K-12 and higher education contexts (Harackiewicz and Priniski, 2018; Lazoswki and Hulleman, 2016; Tibbetts et al., 2016; Walton and Wilson, 2018). For example, a meta-analysis of 92 intervention studies found an average effect size, including both academic and psychological effects, of 0.49 or about half a standard deviation (Lazoswki and Hulleman, 2016). This compares with an average effect size on student academic performance of the adoption of active learning practices of 0.47 (based on a meta-analysis of 158 studies; Freeman et al., 2014). The effects of social-psychological interventions are thus relatively large by the standards of educational interventions, especially given that they generally require modest investments of time and effort compared to many other types of educational reforms (Lazoswki and Hulleman, 2016). The strongest effects of the interventions are often among disadvantaged students such as underrepresented minorities and first-generation college (Lazoswki and Hulleman, 2016; Harackiewicz and Priniski, 2018), a differential effect also seen for active learning (Theobald et al., 2020).

Social-psychological interventions can be classified according to two parameters (Harackiewicz and Priniski, 2018): (1) The psychological processes targeted in students: how students value academic tasks (task value interventions), how students frame academic challenges (framing interventions), or students' personal values (personal values interventions); and (2) The levels at which these interventions work: a specific course, a field of study, or school general. Utility value interventions are a type of task value course-specific intervention designed to increase the task value that students place on a course or subject. Several forms of the intervention have been tested, including telling students why a subject may be valuable (Canning and Harackiewicz, 2015), assigning students to write essays about how the subject may be valuable personally or to others (Harackiewicz et al., 2016), or assigning them to evaluate and respond to quotations that describe reasons a subject may be valuable (Gaspard et al., 2015; Kosovich et al., 2019). Tests of the essay-based utility value intervention (the most frequently-used format) have also attempted to define the optimum delivery parameters by varying the frequency and timing of the assignment (Canning et al., 2018), varying the focus of the essay (value to self vs. value to others; Priniski et al., 2019), testing the effects of student choice and variety in the focus of the assignments (Rosenzweig et al., 2018; Priniski et al., 2019), and testing the effects of increasing the frequency of connections students make between the material and their lives (Hulleman et al., 2017). The somewhat complicated results of these studies, especially when the effects on different groups of students are considered, lead to the overall recommendation that three administrations of the utility value intervention in a semester, beginning with a self-focused essay followed by an other-focused essay and with some choice, is likely to give the best outcome for the greatest number of students. In most tests of the utility value intervention, students with low prior academic achievement (in the course, or in high school) do better academically in the course after receiving the intervention. The observation that a single utility value intervention can often improve a student's grade point average (GPA) and retention in a STEM major up to three years later (Asher et al., 2023; Hecht et al., 2019) suggests that success in the course can in turn increase a student's sense of self-efficacy and motivation, and thereby increase their success in subsequent classes, creating a virtuous cycle. Indeed, many studies that have directly measured various psychological factors have found increases in interest in a subject, career motivation, and competence-related beliefs among students who received a utility value intervention, especially lower-performing students or those with lower performance expectations (Hulleman and Harackiewicz, 2009; Hulleman et al., 2010; Acee and Weinstein, 2010; Brown et al., 2015; Harackiewicz et al., 2016; Hulleman et al., 2017; Harackiewicz and Priniski, 2018; Rosenzweig et al., 2020).

Despite the many studies that report encouraging and even impressive results for social-psychological interventions, there is some evidence in the literature that outcomes can be variable and sensitive to context. For example, one study of > 7,000 students at a flagship state university who were given either a growth mindset or social belonging intervention found that both interventions led to increased full-time enrollment through the entire first year for students from socially and economically disadvantaged backgrounds (Yeager et al., 2016). By contrast, a study of over 6,000 incoming first year students at a different major state university given either a growth mindset or social belonging intervention found that the social belonging intervention had no effect on first year GPA for any group and that the growth mindset intervention resulted in an improvement in GPA only for Latino/a students and not for African American students (Broda et al., 2018). Additionally, Walton et al. (2023) administered a social belonging intervention to 26,911 students across 22 post-secondary institutions in the US and found that the intervention improved students' progress in university, especially among those who belonged to groups that historically progressed at lower rates and at institutions perceived as providing a supportive environment. As another example, a study of 798 students in introductory biology given two “doses” of a values affirmation intervention found that the gap between first generation and continuing generation students in course grades and semester GPA was narrowed by 50% (Harackiewicz et al., 2014). However, a study of over 2,000 students given a values affirmation and/or a utility value intervention by the same research group in the same class found that the values affirmation intervention had no effect on student outcomes (Harackiewicz et al., 2016). The utility value intervention was found to have a small positive effect on GPA for all students, but the strongest effect was for students who were both first generation and underrepresented minority (URM; Harackiewicz et al., 2016). The authors suggest that the greater achievement gap observed for students in the first study made it easier to detect the positive effects of the values affirmation intervention as compared to the second study. All of the studies cited above used a randomized controlled design, which has been found in a meta-analysis to give an average effect size for social-psychological interventions of 0.43 (based on 64 studies) compared to an effect size for quasi-experimental studies of 0.64 (based on 28 studies; Lazoswki and Hulleman, 2016).

Almost all of the studies of social-psychological interventions have been done in traditional courses in which instruction is provided face-to-face. However, online education has become a major part of the overall higher education marketplace, especially for non-traditional students who value the flexibility that asynchronous courses afford those with work and family obligations (Vanslambrouck et al., 2018). Statistics from the US in Fall 2021 show that about 61% of all post-secondary students took at least one course online, and 28% took exclusively online courses (National Center for Education Statistics, 2023). A “mega-analysis” summarizing 16 meta-analyses on the effectiveness of distance education (i.e., looking at student learning outcomes) concluded that online learning overall seems to be as or more effective than traditional educational approaches (Simard et al., 2019), but this conclusion is not universally accepted and may not be true for all types of students in all contexts (Bettinger et al., 2017; Xu and Jaggars, 2013). Some evidence suggests, for example, that students with various types of disadvantages at community colleges may do worse in online classes than in traditional classes (Xu and Jaggars, 2013), and students at a large for-profit university who had the option of taking classes either in-person or online did worse when they took the classes online (Bettinger et al., 2017). Many studies have been carried out of student behavior in online classes. Among the findings are that student self-regulated learning strategies of time management, metacognition, effort regulation, and critical thinking are effective in online classes (Broadbent and Poon, 2015). Students with greater intrinsic motivation tend to use more productive learning strategies, whereas students with high extrinsic motivation (i.e., performance focused) tend to use more superficial and less productive study strategies (e.g., memorization, re-reading; Stark, 2019). Furthermore, students' utility value for a course and their self-efficacy have been found to be associated in online courses with productive study strategies (Marchand and Gutierrez, 2012) and academic success (Cho and Heron, 2015). The connections between students' motivations, use of productive study strategies, and academic outcomes online suggest that social-psychological interventions might be useful in promoting student success in online courses.

The decisions by many universities to move their classes online in response to the COVID-19 outbreak dramatically increased the number of students in online courses. This presented an opportunity to test if social-psychological interventions, and more specifically utility value interventions, might help improve student success in such classes. Therefore, the overall aim of this study was to investigate the impact of a utility value intervention on the academic outcomes of undergraduate students in online courses during the COVID-19 pandemic, compared to students in a control condition. Specifically, the purpose of this study was fourfold: (1) to assess differences in course performance in general and among URM students between groups (utility value vs. control; primary); (2) to assess differences in student engagement with course material via page views and participation between groups (secondary); (3) to determine whether socio-demographic variables (i.e., gender, URM status) predicted course grades among utility value and control participants (tertiary); and (4) to explore differences in essay content between conditions (exploratory).

Methods

Study design

This was a cross-sectional randomized controlled trial. The intervention occurred during the 10-week Spring quarter of 2020 at a 4-year public university in the United States. Ethics approval was obtained from the Institutional Review Board (#170886).

Participants

A total of 1,243 students participated in the study (n = 589 utility values; n = 654 control). Not all participants were included in the analyses due to the way by which data were collected and anonymized, as well as missing data. Most participants identified as women (n = 836; 68.40%) and did not identify as a URM (n = 948, 77.30%). It is worth noting that students who identified as Asian or White were considered non-URM in this study. For context, in Fall 2019, 41.3% of undergraduate students enrolled in Life Sciences–including Biological Sciences–at the host institution identified as Asian, 22.9% identified as Hispanic/Latino(a), 21.9% White, 8.3% International, 2.9% African American, 2.2% Domestic Unknown, 0.4% American Indian, and 0.1% Native Hawaiian/Pacific Islander (UC InfoCenter, 2025). Additionally, 38.5% of undergraduate students enrolled in Life Sciences identified as first generation (UC InfoCenter, 2025). Full demographic details by group can be found in Table 1.

Table 1

Table 1. Demographic information.

Study procedures

A total of five instructors, representing four upper-year courses in biological sciences, agreed to participate in the study. The courses were in the areas of genetics (two lecture sections), genomics, evolution, and human reproduction, and primarily enrolled third and fourth year students. All of the courses had required prerequisites of one semester of introductory cell biology and one semester of introductory organismic and evolutionary biology. The human reproduction course also required a semester of introductory human physiology as a prerequisite. The courses generally used a mix of lecture and active learning instructional strategies. Assessments and grading schemes varied by course and can be found in Appendix. Students in these courses were given the opportunity to opt-out of participating in the study. Instructors informed their students through the learning management system (LMS) that: (1) their data could be used in educational studies; (2) their data would be anonymized before analysis; and (3) there were no risks to them participating. Students were provided with a link to an opt-out form for those who did not wish to participate. Students who did not complete the opt-out form were assumed to have given consent for their data to be used in the study. Each of the courses had multiple recitation sections (from 2 to 11 per lecture section). The recitation sections were randomly assigned to receive either the utility value intervention or a control “course material summary” intervention (details of both are given below), so all students in a recitation section received the same intervention. An independent-samples t-test was run to determine if there were differences in prior preparation using cumulative GPA between control and intervention conditions. There was no significant difference in cumulative GPA between conditions, t_{(1, 224)} = 0.85, p = 0.39. Two assignments (for both control and interventions groups) were administered in each course and delivered online through the LMS. The dates of the two assignments were selected in consultation with the instructors to minimize conflicts with scheduled exams and therefore differed for each course. Students had 5 days to complete each assignment and submit it online. Because the institution has a quarter system with 10-week quarters, two writing assignments were given rather than the three recommended for a 15–16 week semester (Canning et al., 2018). Instructors were blind to whether students received the intervention or the control treatment.

The intervention

The utility value intervention writing assignments were both self- and other-focused (i.e., adapted from Harackiewicz et al., 2023). Students were asked to select a concept covered in the course, formulate a question, answer the question, and explain how the topic applied to their life and to helping others. The prosocial utility value intervention was selected as the students participating in the intervention were in STEM disciplines, which offer a powerful pathway for advancing prosocial goals, with meaningful impact on public health, environmental sustainability, and overall human wellbeing (Harackiewicz et al., 2023). Demonstrating the connection between STEM coursework and students' prosocial aspirations can enhance engagement, boost motivation, and improve academic performance (Harackiewicz et al., 2023). Students in the control condition were similarly asked to select a concept covered in the course and formulate a question; however, instead of explaining how the topic applied to their life and others, control participants were asked to answer the question using information from class notes and readings. The same writing assignments were given for writing assignment 1 and 2 across all courses. The assignments were graded for scientific content, writing quality, and following directions. Instructors were asked to award a small amount of credit for correct completion of each assignment to increase compliance with the intervention. Full intervention and control writing assignments can be found in the Supplementary material.

Data collection

The following data were collected on participants from the LMS at the end of the quarter: overall course grade, page views, and participation. Page views were counted by the number of times pages in the LMS were loaded. Student participation was measured as the number of certain actions students took within a course, specifically posting in the discussion forum and submitting an assignment or quiz. Page views and participation are two of many standard measures of general student engagement in studies using LMS log data (Ahmadi et al., 2023). Additionally, students' grades and socio-demographic characteristics (i.e., cumulative GPA, gender, and URM status) were obtained from the university's Registrar's Office. Lastly, students submitted their writing assignments via the LMS, which were analyzed using Linguistic Inquiry and Word Count (LIWC) software. LIWC software “connects important psychosocial constructs and theories with words, phrases, and other linguistic constructions” (Boyd et al., 2022, p. 2). LIWC consists of target words and dictionary words (Boyd et al., 2022). Target words refer to the words in essays that are read and analyzed by LIWC, while dictionary words are those contained in the LIWC dictionary file (Boyd et al., 2022). The LIWC software assesses text within an essay and compares it against the LIWC dictionary “by counting all of the words in a target text, then calculating the percentage of total words that are represented in each of the LIWC sub-dictionaries” (Boyd et al., 2022, p. 3). For the purpose of this study, five dictionaries/sub-dictionaries were selected for analysis: (1) word count (i.e., total word count of the essay); (2) personal pronouns–1st person singular words (e.g., I, me, my, myself); (3) personal pronouns–2nd person words (e.g., you, your, u, yourself); (4) cognitive processes (e.g., but, not, if, or, know); and (5) social processes (e.g., you, we, he, she). The dictionaries selected were based on LIWC dictionaries frequently used in utility value studies (Harackiewicz et al., 2016; Priniski et al., 2019).

Data analysis

To assess differences in course performance between groups (control vs. intervention) an independent samples t-tests were conducted and to examine the effects of URM status and condition on course grades and GPA, a series of two-way ANOVAs were conducted (primary purpose). To assess differences in student engagement with course material via course website page views and course participation between groups, independent samples t-tests were conducted (secondary purpose). Where there were unequal variances between groups, Welch's t-tests were used. To determine whether socio-demographic variables (i.e., gender, URM status) predicted course grades among utility value and control participants, multiple linear regression was run (tertiary purpose). Lastly, to explore differences in essay content between conditions, Linguistic Inquiry and Word Count (LIWC) text analysis was used (exploratory purpose). The LIWC scores were averaged across the two writing assignments, yielding a total score for each dictionary/sub-dictionary and t-tests were conducted to compare average scores between conditions (utility value vs. control).

It is worth noting that in order to obtain students' grades and socio-demographic characteristics for analysis (i.e., cumulative GPA, gender, and URM status) data were sent to a central unit at the university and were returned anonymized per procedures detailed in the Institutional Review Board protocol. In doing this, the Registrar's Office removed data pertaining to page views and course participation. As such, two datasets were used for analysis: one consisting of the data collected prior to it being sent to the Registrar's Office (inclusive of condition, numerical course grade, page views, participation, and LIWC variables); and a second, anonymized, dataset that was returned from the Registrar's Office (inclusive of condition, gender, URM status, letter course grade, and cumulative GPA).

Results

To determine if there were significant differences in writing assignment completion between utility value and control conditions, chi square tests of association were conducted. A total of 1,003 participants completed writing assignment 1 (n = 473 utility values; n = 530 control) and 992 completed writing assignment 2 (n = 478 utility values; n = 514 control). Overall, there were no significant differences between conditions in writing assignment completion for assignment 1 [χ²(1) = 0.11, p = 0.74] or assignment 2 [χ²(1) = 1.15, p = 0.28].

Primary purpose

An independent samples t-test was run to determine if there were differences in course grade between utility value and control group participants in general. There were 557 utility value participants, and 620 control group participants included in the analysis. Utility value participants had slightly higher course grades (88.98 ± 7.41)¹ than control group participants (88.60 ± 7.72); however, there was not a statistically significant difference between groups, M_difference = 0.38 (95% CI, −0.48 to 1.25), t_{(1, 175)} = 0.87, p = 0.39, d = 0.05.

A two-way ANOVA was conducted to examine the effects of URM status and condition on course grades. As explained in Methods, only course letter grades, not numerical grades, were available for students with demographic information attached. Therefore, in order to assess differences, the letter grades were converted to numerical grade points using the university's 4-point scale ranging from 0 (fail) to 4 (A, A+). The grading system did not have a numerical equivalent for “no pass” (NP) and “pass” (P) and a relatively small number of students (n = 118) received grades of NP or P. As such, students with these letter grades were not included in the analysis. There was no statistically significant interaction between URM status and condition for course grades, F_{(1, 1, 104)} = 0.057, p = 0.81, partial η²= 0.00. Therefore, an analysis of the main effects for URM status and condition were performed. There was a statistically significant main effect of URM status on course grades, F_{(1, 1, 104)} = 45.00, p = < 0.001, partial η²= 0.039. Specifically, non-URM students had significantly higher course grades than URM students (M_difference = 0.23; 95% CI, 0.16–0.30). There was no statistically significant main effect of condition on course grades, F_{(1, 1, 104)} = 1.45, p = 0.23, partial η²= 0.001.

Similar to the above, a two-way ANOVA was conducted to examine the effects of URM status and condition on cumulative GPA. There was no statistically significant interaction between URM status and condition for cumulative GPA, F_{(1, 1, 222)} = 0.36, p = 0.55, partial η²= 0.00. There was a statistically significant main effect of URM status on cumulative GPA, F_{(1, 1, 222)} = 157.77, p = < 0.001, partial η²= 0.11. Similar to course grades, non-URM students had significantly higher GPAs than URM students in both conditions (M_difference = 0.35; 95% CI, 0.29–0.40). There was no statistically significant main effect of condition on cumulative GPA, F_{(1, 1, 222)} = 0.45, p = 0.50, partial η²= 0.00.

Secondary purpose

Page views

A Welch t-test was run to determine if there were differences in student engagement with course material via course website page views between utility value and control groups. There were 563 utility value participants, and 630 control group participants included in the analysis. The utility value group engaged with the course website course more (858.17 ± 405.29) than the control group (797.88 ± 347.69). A statistically significant difference of 60.29 (95% CI, 17.14–103.44), t_{(1, 113.86)} = 2.741, p = 0.006, d = 0.16 was found between groups.

Participation

An independent samples t-test was run to determine if there were differences in student engagement with course material via course participation (posting to LMS discussions and submitting assignments and assessments on the LMS) between utility value and control groups. There were 566 utility value group participants, and 629 control group participants included in the analysis. The utility value group participated in the course more (40.49 ± 9.09) than the control group (38.86 ± 8.81). A statistically significant difference of 1.63 (95% CI, 0.61–2.65), t_{(1, 193)} = 3.15, p = 0.002, d = 0.18 was found between groups.

Tertiary purpose

Multiple linear regression was run to understand the effect of gender and URM status on course grades. The overall regression, including both predictors, was statistically significant (R = 0.20, R² = 0.039, adjusted R² = 0.037, F = 22.50, p = < 0.001; see Table 2 for full regression results). Gender and URM status accounted for 3.9% of the variation in course grade. One significant predictor was identified, URM status (b = 0.23, p = < 0.001).

Table 2

Table 2. Multiple linear regression of gender and underrepresented minority status predicting course grade.

Exploratory purpose

Participants who completed both writing assignments were included in the exploratory analyses, as data from essays 1 and 2 were averaged for each dictionary/sub-dictionary.

An independent samples t-test was run to determine if there were differences in essay length (i.e., total word count) between utility value and control group participants. There were 434 utility value participants, and 482 control participants included in the analysis. The utility value group wrote longer essays (503.01 ± 117.44) than the control group (447.04 ± 109.67). A statistically significant difference of 55.96 (95% CI, 41.23–70.69), t₍₉₁₄₎ = 7.46, p < 0.001, d = 0.49 was found between groups.

A Welch t-test was run to determine if there were differences in the use of personal pronouns–1st person singular words between utility value and control group participants. There were 434 utility value participants, and 482 control participants included in the analysis. The utility value group used 1st person words (1.13 ± 0.71) more than the control group (0.25 ± 0.52). A statistically significant difference of 0.88 (95% CI, 0.80–0.97), t_(787.69) = 21.25, p < 0.001, d = 1.52 was observed between groups.

A Welch t-test was run to determine if there were differences in the use of personal pronouns–2^nd person words between utility value and control group participants. There were 434 utility value participants, and 482 control participants included in the analysis. The utility value group used 2nd person words (0.76 ± 0.66) more than the control group (0.15 ± 0.42). A statistically significant difference of 0.61 (95% CI, 0.54–0.68), t_(721.21) = 16.60, p < 0.001, d = 1.24 was found between groups.

An independent samples t-test was run to determine if there were differences in use of social process words between utility value and control group participants. There were 434 utility value participants, and 482 control participants included in the analysis. The utility value group used words such as “you, we, he, and she” (6.90 ± 2.11) more than the control group (4.53 ± 2.02). A statistically significant difference of 2.38 (95% CI, 2.11–2.64), t₍₉₁₄₎ = 0.18, p < 0.001, d = 0.01 was observed between groups.

A Welch t-test was run to determine if there were differences in use of cognitive process words between utility value and control group participants. There were 434 utility value participants, and 482 control participants included in the analysis. The utility value group used words such as “but, not, if, or, and know” (13.36 ± 2.13) more than the control group (11.78 ± 2.53). A statistically significant difference of 1.59 (95% CI, 1.28 to 1.89), t_(910.01) = 10.29, p < 0.001, d = 0.68 was found between groups.

Discussion

The aim of this study was to investigate the impact of a utility value intervention on the academic outcomes of undergraduate students in online courses during the COVID-19 pandemic, compared to students in a control condition. Results revealed that there were no significant differences in course performance between conditions; however, URM status was a significant predictor of course grade. Specifically, non-URM students had significantly higher course grades and cumulative GPAs than URM students. The utility value intervention did have significant effects in this study, as students in the utility value condition engaged with and participated in their courses significantly more than students in the control condition according to LMS data. Further, LIWC analysis revealed that utility value participants wrote significantly longer essays, and used more personal pronouns, second person pronouns, social processes words, and cognitive processes words than students in the control condition. To reduce the burden on students in this first quarter of COVID-19-related online instruction, we did not administer questionnaires to directly assess changes in psychological factors such as interest, motivation or self-efficacy.

It is not surprising that students in the utility value group engaged with and participated in their courses significantly more than students in the control group, as utility value interventions have been found to improve both students' interest in a subject and their academic outcomes (Asher et al., 2023; Hulleman et al., 2010; Totonchi et al., 2025). While educators have little influence on initial student interest in course content, they play a key role in course design and implementation, which can influence whether students perceive the course to be valuable (Hulleman et al., 2010). Notably, Hulleman et al. (2010) encouraged students to consider the relevance of their course material via a utility value intervention. Students were asked to reflect on how the course material could be applied to their lives through a writing exercise (Hulleman et al., 2010). The authors concluded that the intervention increased perceptions of utility value, which predicted interest in course material and performance (Hulleman et al., 2010). Similarly, Totonchi et al. (2025) tested the efficacy of a utility value intervention on improving community college students' perceived math relevance and achievement in introductory math courses. The authors concluded that students in the utility value condition perceived course material to be more relevant, and thus benefitted from this perception, compared to participants in the control condition (Totonchi et al., 2025). Additionally, the intervention's positive effect on subject relevance was more prominent among first generation students than other demographic groups (Totonchi et al., 2025). The evidence to support utility value interventions to improve the academic pipeline for students from diverse backgrounds is growing. Specifically, Asher et al. (2023) implemented a utility value intervention in an introductory chemistry course with the goal of promoting persistence and diversity in STEM. Not only did participation in the intervention improve STEM persistence in general, but the effects were largest for students from marginalized and underrepresented racial/ethnic backgrounds, who were 14% more likely to persist in STEM fields following the intervention (Asher et al., 2023). It is evident that participating in utility value interventions can positively impact the academic trajectory of students, and it is therefore not surprising that course engagement and participation was significantly higher among participants in the utility value group compared to the control condition in the current study. It is plausible that by reflecting on the relevance of course material, utility value students were more invested in their learning and success compared to students in the control group, thus explaining students' increased engagement and participation.

It is interesting, however, that there were no significant differences in course grades between groups in our study, unlike in most previous studies. This might be explained, in part, by the courses from which students were recruited for the study. Notably, in previous utility value studies students were enrolled in introductory courses (Asher et al., 2023; Harackiewicz et al., 2016; Priniski et al., 2019); however, students in the current study were enrolled in upper-division courses. The only comparable study of which we are aware tested the utility value intervention in an online upper-level inorganic chemistry course taken primarily by third- and fourth-year students (Wang and Lewis, 2022). In that study, students who completed three utility value assignments over the semester showed no statistically significant difference in grades on exams given after these assignments (including the final exam) compared to students who completed “content summary” control assignments. These results and ours suggest the possibility that utility value interventions have less of an effect on upper-year students, perhaps because students enrolled in upper-division courses are already committed to their majors and motivated to perform well (Wang and Lewis, 2022). Additionally, the topics of these courses might have inherent utility value for students based on their trajectory (e.g., pre-healthcare profession students). Moreover, it is worth noting that the current study took place at the beginning of the COVID-19 pandemic, when grading policies were more lenient (Chan, 2023). Therefore, it is plausible that students' grades in both conditions were higher than usual, thus explaining the lack of significance between conditions.

There were no significant differences in course grade between conditions and there was no statistically significant interaction between URM status and condition for course grade. Interestingly, these findings do not align with the literature, as utility value interventions have been found to narrow the achievement gap for URM students (Asher et al., 2023). Given that the interaction between URM status and condition was not significant, results suggest that the utility value intervention did not positively impact URM students' academic performance. However, it is worth noting that URM status was found to be a significant predictor of course grade. Specifically, non-URM students had significantly higher course grades and cumulative GPAs than URM students. This is not surprising as it is well-established that URM students perform poorly academically compared to non-URM students (Alon et al., 2023; Simmons and Heckler, 2020; Whitcomb and Singh, 2021). The current study's findings highlight this disparity. In the context of online learning and the COVID-19 pandemic, Alon et al. (2023) found that the transition to online learning significantly widened the achievement gap between URM students and students in the majority group. The widened achievement gap coupled with lenient grading policies as a result of the COVID-19 pandemic (Alon et al., 2023; Chan, 2023) might explain why the intervention did not have a significant impact on URM students' academic performance.

The LIWC analysis revealed that the essay content of those in the utility values condition was significantly more meaningful than the essay content of students in the control condition. These findings align with the literature, as Priniski et al. (2019) conducted a utility value intervention and concluded that the utility values writing assignments had a more personal focus (i.e., used more personal pronouns) and used more social and cognitive process words than control writing assignments. Similarly, Harackiewicz et al. (2016) found that students in the utility values condition used more personal pronouns and that their essays contained more social and cognitive mechanism words than students in the control condition. The results from these studies align with the current study, as utility value participants used more personal pronouns, second person pronouns, social process words, and cognitive process words than students in the control condition. Pronouns and verb tense, such as use of first and second person pronouns, can help identify focus and, in turn, highlight individual priorities, intentions, and processing (Tausczik and Pennebaker, 2010). In contrast, pronouns within social processes can demonstrate the quality of a close relationship (Tausczik and Pennebaker, 2010). For example, the social process word “we” has been found to be related to higher relationship quality compared to second person words such as “you” (Tausczik and Pennebaker, 2010). Additionally, cognitive mechanisms such as “cause, know, and ought” can indicate more complex language (Tausczik and Pennebaker, 2010). Thus, findings from the current study underscore that utility value participants were more cognitively and socially engaged in course content compared to control participants.

Recommendations for future research

The finding from the current study that a utility value intervention in online upper-level biology classes did not improve student outcomes (grades) over a control treatment, which is similar to the result obtained in an online upper-level chemistry class (Wang and Lewis, 2022) and differs from the results of many previous studies carried out in introductory classes taught in person (Harackiewicz and Priniski, 2018), suggests several avenues for future research. First, are utility value interventions generally ineffective in improving student outcomes in upper-level classes, and if so, why? Second, are utility value interventions ineffective in improving student outcomes in online classes, and if so, why? Studies using utility value interventions in upper-level in-person classes and in online introductory classes that incorporate measures of various psychological factors could help to answer these two questions.

Limitations

Limitations of this study include the following: first, no psychosocial measures were administered in the current study. Though utility value interventions have demonstrated positive effects on the psychological outcomes of students in the past, in an effort to reduce participant burden, this data were not collected in the current study. Given the abrupt transition to online learning and the evolving nature of the COVID-19 pandemic, the research team purposefully limited data collection to demographics and data that could be collected from the university, to avoid students having to self-report. Future studies might consider assessing psychological and social outcomes of students participating in an online utility value intervention. Second, the sample size of URM students was relatively small. This is likely due to the demographics of the study institution's student body. Institutional data was received post-intervention on student demographics, including URM status. The research team was limited to the data that could be obtained and thus, few demographic details are reported in the current study. Given that in-person utility value interventions have been found to promote retention and persistence in STEM (Asher et al., 2023), it is recommended that future research focus on implementing in-person utility value interventions for students of disadvantaged backgrounds in an effort to dismantle systemic inequities and narrow the achievement gap between URM students and their peers that was further exacerbated by the COVID-19 pandemic. Third, the current study was limited to quantitative data. Future research is encouraged to include qualitative methods to gain deeper insights into students' experiences with the intervention in online environments. Lastly, because the intervention was carried out at a single institution during one quarter only, the study findings might not be generalizable to other types of institutions and student populations. It is also worth highlighting that the current study was conducted during the COVID-19 pandemic, which may impact the generalizability of study findings. Interestingly, findings from the current study mirrored those of Wang and Lewis (2022), who implemented a utility value intervention with upper-division students during the pandemic and similarly observed no significant effects. While this may suggest that such interventions are less impactful for upper-year students, it is also plausible that byproducts of the pandemic–such as heightened stress, disrupted learning environments, and diminished motivation–interfered with the intervention's effectiveness. It is recommended that researchers replicate the study to assess whether the findings could be generalizable to upper-division and/or online courses today.

Conclusion

The aim of this study was to investigate the impact of a utility value intervention on the academic outcomes of undergraduate students in online courses during the COVID-19 pandemic, compared to students in a control condition. Overall, the intervention had a positive effect as utility value participants engaged with and participated in their courses significantly more than students in the control group. There were no differences observed in course performance between conditions; however, non-URM students had significantly higher course grades and cumulative GPAs than URM students. Despite the fact that the intervention did not significantly impact URM students, overall, the intervention was successful as findings from the LIWC analysis underscored that utility value participants were more actively engaged with course content than control participants. Study findings demonstrate that a utility value intervention administered to upper-division classes in an online learning environment can yield positive effects, though the effects might not be as strong as in-class utility value interventions. Future research should aim to narrow the achievement gap between URM students and their non-URM peers by offering a utility value intervention specifically for students of diverse backgrounds. University personnel might consider incorporating utility values tasks into large first-year introductory online courses in an effort to intervene at a time that is crucial and positively alter the academic trajectory of URM students.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by University of California, San Diego Institutional Review Board (#170886). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

KS: Writing – review & editing, Writing – original draft, Formal analysis. SL: Conceptualization, Writing – review & editing, Supervision. LH: Conceptualization, Writing – review & editing, Investigation, Data curation, Supervision.

Funding

The author(s) declare that no financial support was received for the research and/or publication of this article.

Acknowledgments

This project was developed and initiated during LH's sabbatical at the School of Biological Sciences, University of California, San Diego. LH thanks Adelphi University for the sabbatical support, Drs. Stanley Lo and Mark Estelle for sabbatical hosting, and Dr. Thomas Bussey and graduate students in the Joint Doctoral Program in Mathematics and Science Education for discussions and guidance. The authors would also like to thank the instructors who assisted with recruitment, Drs. Christopher Day, Katie Petrie, Jim Cooke, Keefe Reuther, and Sarah Stockwell, as well as the participants of the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2025.1636446/full#supplementary-material

Footnotes

1. ^Data are mean ± standard deviation, unless otherwise stated.

References

Acee, T. W., and Weinstein, C. E. (2010). Effects of a value-reappraisal intervention on statistics students' motivation and performance. J. Exp. Educ. 78, 487–512. doi: 10.1080/00220970903352753