Exploring approaches to measuring student engagement across computer-supported collaborative learning contexts

Abstract

Introduction:

Supporting meaningful student engagement is an important goal of educators and educational researchers. However, there are many variations in definitions and measures of engagement, making it challenging to recognize and design for deep engagement. To address this challenge, the purpose of this work is to interrogate the ways that different theoretical and methodological choices impact how we describe student engagement.

Methodology:

We applied two student engagement coding schemes to classroom video from two computer- supported collaborative learning (CSCL) contexts. We compare how each coding scheme made different types of engagement more or less visible and explore potential decision points and synergies.

Findings:

The coding schemes differed in many ways, including their theoretical grounding, level of analysis, and inclusion of various dimensions of engagement. We argue that each of these methodological choices orient researchers to video data differently, which has implications for how researchers make sense of group work, next analytical steps, and claims. There are also substantial challenges that researchers may encounter when using engagement coding schemes outside of the contexts in which they were developed, including achieving reliability and the relevance of indicators of engagement to the novel context.

Discussion:

This study contributes to our understanding of how to make sense of engagement in classroom video and the synergies between different coding choices and theoretical perspectives.

1 Introduction

In science education, educators have increasingly challenged learners to become “doers” of science, leading to classroom participation structures that involve active collaboration in small groups (Forman, 2018; National Research Council, 2012; Zangori and Pinnow, 2020). Understanding how students engage in these small groups is essential for designing for equitable participation– that is, participation where all students have access to high-quality, meaningful opportunities to learn science (Bang et al., 2017). The ways in which educators and/or researchers both define and measure “student engagement” shape which students are seen as participating meaningfully and which are not (Culver et al., 2024), and students whose engagement does not match the assumptions of a given measure risk being read as less capable rather than differently engaged (Lee, 2008). This risk also falls unevenly across students. For example, schemes built around visible, verbal markers of participation can undercount quieter forms of engagement, such as active listening or distributed contribution, that may be more common among multilingual learners or students newer to a classroom's discourse norms (Jennings et al., 2025; Lee, 2008), with consequences for the support and expectations those students are then given. This pilot study contributes to closing that gap by comparing how two established observational measures of engagement classify the same classroom video data, examining where their classifications converge and where they diverge. Our goal is not to move towards one optimal measure of engagement but rather to examine how different indicators are made salient in each coding scheme and how they might render different conclusions about student engagement.

2 Literature review

Although most educational researchers agree that student engagement is important to support, there is disagreement about how to define and measure it (Appleton et al., 2008). Engagement is a term that is often used colloquially and, thus, ambiguously in educational practice and literature (Azevedo, 2015). Even when it is defined, there is lack of agreement across the literature on those definitions or dimensions of interest (Fredricks et al., 2011). Methodologically, there are also many ways of “seeing” engagement, from classroom videos, to surveys, to student or teacher interviews (Steinberg et al., 2023). Each of these ways of measuring engagement has its own affordances and constraints and highlights different elements of classroom activity.

Definitionally, engagement is often defined in relationship to similar constructs, such as motivation (De Loof et al., 2021; Henrie et al., 2015) or students' connection with school or a particular subject (Fried and Konza, 2013; Skinner et al., 2009; Sun et al., 2022). Fredricks et al. (2004) assert that engagement is a multidimensional construct, consisting of behavioral, cognitive, and emotional layers. Other researchers define additional dimensions of engagement, including connective, productive disciplinary, and consequential engagement (e.g., Agarwal and Sengupta-Irving, 2019; Engle and Conant, 2002; Gresalfi et al., 2009). Some study only one dimension (e.g., cognitive; Chen and Terada, 2021), while others argue that the dimensions are so intertwined that it is not productive to separate them (e.g., Bergdahl et al., 2018; Renninger and Bachrach, 2015). This lack of consensus makes it difficult to compare findings across studies, since two papers reporting on “engagement” may be describing different constructs altogether (Azevedo, 2015). Rather than arguing for a single consistent coding scheme, we suggest that researchers be explicit about which dimensions of engagement they are measuring and why, so that claims about engagement can be interpreted in light of those choices. In this article, we view engagement as a multidimensional construct that describes varied qualities of students' participation in learning activities and explore how a focus on different dimensions shapes our understanding of engagement (Fredricks et al., 2004).

In addition to diversity in definitions, there are also a range of ways in which student engagement has been measured in science education, ranging from surveys (Testa et al., 2022), to experience sampling (Schmidt et al., 2018), to interviews (Godec et al., 2018), to observational rubrics (Hall and Miro, 2016), to face-based engagement detection (Bosch, 2016). Studying engagement requires determining the level (individual or collective) and grain size (person-oriented or context-oriented; Sinatra et al., 2015) at which it is measured. Henrie et al. (2015) conducted a literature review on engagement in technology-mediated environments at both the K-12 and undergraduate levels. They found that the majority of studies (61.1%) utilized quantitative self-reports to measure engagement. Although self-report instruments are practical, relatively easy to administer, and capture student perceptions of their own engagement, they tend to be worded generally (e.g., I work hard); therefore, they may not provide accounts of contextual factors, like specific task, curriculum, or pedagogical choices that influence engagement (Fredricks and McColskey, 2012). Given that student engagement is produced dynamically through interactions between learners and environments, understanding these contextual features is vital for a more complete picture of how engagement unfolds during learning activities (Ryu and Lombardi, 2015).

Therefore, in this study, we apply two observational engagement coding schemes to video data of students participating in CSCL science contexts.

Recent years have seen increasing calls for incorporating multimodal data streams to capture student engagement (Chango et al., 2022; D’Mello et al., 2017). Multimodal approaches such as multimodal learning analytics, which combine data streams such as eye-tracking, physiological sensors, log-file data, or speech analytics, have been proposed in order to triangulate across channels and achieve more accurate and complete measurement of engagement (D’Mello et al., 2017; Wiedbusch et al., 2023). However, collecting multimodal data introduces its own complications. Synchronizing multiple data streams in authentic classroom settings is logistically demanding, and fusing disparate channels into a coherent account of engagement is not always straightforward (Guerrero-Sosa et al., 2025; Jewitt, 2015). Moreover, some multimodal indicators are derived from low-level physiological or sensor signals such as heart rate variability or skin conductance whose relationship to higher-order constructs like engagement remains difficult to interpret and independently verify (Blikstein and Worsley, 2016).

While multimodal data offers richer triangulation in principle, it has limited feasibility in naturalistic classroom settings (Chango et al., 2022; Sharma and Giannakos, 2020). Video data, which preserves multimodal signals (e.g., gaze, gesture, discourse), offers a practical and analytically powerful middle ground (Nassauer and Legewie, 2022). It is among the most common and feasible data sources in authentic classrooms (Foster et al., 2024). While observational method through video data can be labor intensive, it offers the situated, interactional character of engagement as it unfolds, and is replayable, allowing multiple coders to apply different analytical frameworks to identical footage (Lu et al., 2025). This study aims to compare how two observational frameworks can be applied to understand student engagement differently through videos. We focus specifically on engagement as participation because it is directly observable in video, grounding claims about engagement in concrete interactional evidence rather than inferred internal states.

We initially selected two observational frameworks of engagement because of our interest in understanding the unique affordances of and potential synergies between approaching engagement from a cognitive versus a sociocultural perspective (two dominant classes of theories in CSCL; Danish and Gresalfi, 2018). However, as we will share in the findings, differences in theory were not the only ways that these coding schemes oriented us to the data differently.

Broadly speaking, cognitive theories of learning and engagement posit that knowledge is represented within the mind of an individual (Case, 1996). Therefore, research guided by these theories generally focuses on an individual's internal mental processes and on supporting them in gaining knowledge. In contrast, sociocultural theories of learning build on the belief that knowledge exists in social contexts; knowing is seen as a joint accomplishment between an individual and their context (Greeno et al., 1996). While cognitive and sociocultural perspectives on learning are often presented in opposition to one another, looking for synergies across these traditions can reveal opportunities to build on both that only become visible in their contrast (Danish and Gresalfi, 2018; Sfard, 1998). In the next section, we describe two coding schemes that are associated with each observational framework that were selected to guide the current study.

2.1 The coding schemes

Both selected coding schemes were recently published, grounded their framework in a coherent theoretical approach to engagement, and were designed to measure student engagement using video or observation, but they differ in theoretical orientation. For the cognitive coding scheme, we selected the Classroom Observation Protocol for Interactive Engagement in STEM (COPIE-STEM), an observation protocol designed for use in both hands-on and lecture-based STEM classrooms based on the ICAP framework of engagement (Hsiao et al., 2022). For the sociocultural coding scheme, we selected Rogat et al. (2022)'s measure of group disciplinary engagement (GDE), which was originally designed for use with small groups of students in middle school science, math, and engineering contexts. Table 1 outlines the ways that each coding scheme operationalized different modes/dimensions of engagement. Please refer to the original papers for the full coding schemes.

2.1.1 COPIE-STEM

COPIE-STEM is grounded in ICAP, a well-known cognitive framework for the measurement of engagement (Chi, 2009; Chi and Wylie, 2014). According to ICAP, engagement behaviors fall into four modes that exist along a spectrum, Interactive, Constructive, Active, and Passive, and that as learners move from passive toward interactive engagement, their learning also increases. Students are considered passively engaged when they receive information without noticeably doing anything with it. Active engagement occurs when students manipulate materials in some way, such as underlining or taking notes. In both passive and active engagement, students do not move beyond the given learning materials. In contrast, constructive and interactive engagement are characterized by deep processing strategies and require students to move beyond the provided materials. In constructive engagement, students generate outputs, such as by asking questions or taking notes in their own words. In interactive engagement, students participate in dialogue with others in ways that move beyond the provided learning materials. COPIE-STEM (Hsiao et al., 2022) translates ICAP's four conceptual modes into classroom-observable behaviors that researchers can code from video data post hoc. Specifically, COPIE-STEM gives concrete examples of what counts as each mode in a STEM classroom, for instance treating listening or reading as passive engagement and asking elaborative questions as constructive engagement (see Table 1).

2.1.2 Group disciplinary engagement

Rogat et al. (2022) offer a socioculturally-oriented framework to analyze the disciplinary engagement of groups. They divide group disciplinary engagement (GDE) into five dimensions: behavioral, socioemotional, collaborative, metacognitive, and disciplinary. As presented in Table 1, behavioral engagement refers to the extent to which the group jointly participates in the learning task; socioemotional engagement to the quality of the group's interpersonal interactions and climate; collaborative engagement to the group's conceptual coordination; metacognitive engagement to the group's use of regulative and planning strategies; and disciplinary engagement to the group's activity with regards to the academic content as defined by the discipline. This framework was influenced by several engagement frameworks that build on sociocultural theories of learning (Agarwal and Sengupta-Irving, 2019; Engle and Conant, 2002; Gresalfi et al., 2009; Gresalfi and Barnes, 2016; Sinatra et al., 2015). Each of these sociocultural approaches to engagement focus on the social, interactional, and contextual nature of engagement and often use collective group interactions as the unit of analysis (Ryu and Lombardi, 2015).

3 Methodology

In this paper, we explore the two observational frameworks described above to understand how our decisions about coding schemes and coding orient us to data and/or analysis differently. To do so, we apply both coding schemes to two cases of multifaceted engagement in CSCL classrooms: elementary school students (fifth grade) participating in mixed-reality embodied science modeling (GEM-STEP; Danish et al., 2022) and middle school students (seventh grade) participating in a problem-based science game (ECOJOURNEYS Saleh et al., 2019). In both contexts, learners worked together in small groups of four students each to learn about scientific ecosystems. While both contexts were CSCL learning environments, they differed in instructional approach (modeling versus problem-based learning), the age of the students (upper elementary versus middle school), the scientific content (garden versus aquatic ecosystems), and the technology (embodied mixed-reality versus a laptop-based 3D game). We used small subsets of data from each context (12 min each) because our goal was not to thoroughly analyze engagement throughout each context but, rather, to explore how different approaches to the analysis of engagement adapt to and are influenced by different types of learning contexts. In this sense, the data we analyze in this study is not the video data, but the coding decisions that we made. We describe each context in detail below. Given this limited and purposefully selected dataset, we treat this study as exploratory in scope. Our aim is to surface methodological decision points for researchers rather than to draw generalizable conclusions about student engagement or to seek an optimal way to measure engagement.

3.1 Small group embodied science modeling in GEM-STEP

GEM-STEP is a mixed-reality modeling software that enables students to engage in collaborative embodied scientific modeling (Danish et al., 2022). Students control agents in the system (e.g., worms; bunnies) either with full body movements by wearing hats or lanyards with tracking tags or with their fingers by using iPads (see Figure 1).

Figure 1

We coded data from a Fall 2022 implementation of GEM-STEP. The participants (n = 18) were fifth grade students from a classroom at a public elementary school in the Midwestern United States. Thirteen students self-reported demographic information: students were between 10 and 12 years old and majority identified as white (92.3%). Approximately 30% used he/him pronouns, 54% used she/her pronouns, and 15% used they/them pronouns. The students participated in nine lessons using the software, lasting approximately 45 min to one hour. The curriculum focused on understanding processes of energy transfer within pond and garden ecosystems. Students engaged in full class discussions, worked in small groups to plan how they would move as a part of the model and how they would change the script that guided how characters interact in the model, and tested the model in small groups. We selected GEM-STEP as a context for studying engagement because we believe that moving from planning in small groups to standing up and collectively embodying scientific phenomena would offer a range of observable, collaborative engagement that would be generative for analysis.

We focus on one group of students on the sixth day of the implementation, selected because the size of their group on this day (4 students) matched that of the second context (described below) and because of the quality of their data. This group included one boy (Bob^[1]) and three girls (Peyton, Max, and Sarah) who worked with one researcher to define the functionality of an ecosystem health meter, which was displayed on the side of the model as a bar graph. The group spent approximately ten minutes making a plan for what the health meter should track (e.g., just the average health of consumers, just producers, some combination, etc.), and how they would act during the embodied modeling round in order to keep the ecosystem health high. During this planning session, students had access to a shared iPad showing the GEM-STEP model and individual paper handouts with snippets of the code that programmed the ecosystem health meter (Figure 2; Steinberg and Danish, 2024). After this conversation, students participated in a two-minute round of embodied modeling where they acted as worms and bunnies and tried to keep the ecosystem balanced (i.e., all organisms were alive and healthy) within the GEM-STEP software. We refer to the first activity as the “planning” episode, and the second as the “embodied modeling” episode. Data sources included classroom video and screen recordings of the projected GEM-STEP software. The planning episode lasted for ten minutes, and the embodied modeling episode lasted for two minutes.

Figure 2

3.2 Small group problem-based learning in a narrative game: ECOJOURNEYS

ECOJOURNEYS is a problem-based learning environment that engages students with a game-based narrative to collaboratively investigate a complex problem: uncovering the mystery of why tilapia are getting sick on a remote island in the Philippines (Saleh et al., 2019; Zou et al., 2024; Figure 3). The game consists of one tutorial and three quests, each of which contains individual investigation and collaborative problem-solving activities. Students worked on an individual laptop in groups of three to four, interacting with in-game non-player characters (NPCs) and learning materials such as videos and notes. Upon collecting information, students meet at a collaborative learning space, Deduce, where they answer a set of questions and reach an agreement by justifying their ideas to their peers.

Figure 3

We coded video data from a Fall 2022 implementation of ECOJOURNEYS. The participants were seventh grade students (12–13 years old) from a science classroom at a public middle school in the Midwestern United States. The school was located in a rural area, with approximately 18–20 students per class. The student body was 85.1% White, 9.3% Black, 2% Hispanic/Latino, and the remainder identified with other racial or ethnic groups. The science teacher had 17 years of teaching experience in the content area. The Deduce activity is one of the central collaborative activities in ECOJOURNEYS which requires sustained joint reasoning and visible coordination among group members. It affords direct observation of group-level engagement across behavioral, collaborative, and disciplinary dimensions, making it a particularly appropriate context for comparing how different coding schemes capture collective participation. We selected two video clips of a small group who worked in Deduce on two different days (see Figure 4). Specifically, we selected clips that demonstrated best possible video quality with faces and gestures observable majority of the time. The group consisted of two girls (Mary and Lisa) and two boys (Jack and Tom). This group was selected because they exhibited variability in their engagement, and the video quality was good. We refer to these video clips as Episode 1 and Episode 2 in the following sections. Episode 1 lasted for five minutes, whereas Episode 2 lasted for seven and half minutes. In Episode 1, students worked on Deduce during Quest 1, in which the teacher instructed the class to be quiet. In Episode 2, students worked on Deduce during Quest 2, and there were no stated expectations for silence.

Figure 4

3.3 Analysis

The videos were segmented into one-minute segments for COPIE-STEM, and 2.5-minute segments for GDE. GDE was originally used by its authors on five-minute segments (though later reduced to 2.5-minute segments), but we chose to reduce the size of the segments due to the small amount of data we were coding, so as to give us more coding decision points. COPIE-STEM requires that for each minute, coders mark whether indicators of each dimension were present or absent. GDE requires that coders rate the quality of each dimension on a scale from low to high at each time interval.

Two coders (the first and second authors) discussed their understanding of the two coding schemes in relation to both GEM-STEP and ECOJOURNEYS contexts. The first author was involved in the design of, and was present at, the GEM-STEP implementation, while the second author was part of the design of, and was present at the ECOJOURNEYS implementation. Neither of the coders was involved in the development of the chosen Rogat et al. (2022) coding scheme. Together and separately, they viewed additional video data from both contexts that was not coded for this paper to orient themselves to each dimension of engagement. Both schemes were applied to the same video segments. Because of difficulties with achieving high agreement on some dimensions during training, all the data was double coded. Krippendorff's alpha was calculated (Table 2) and discrepancies were discussed and resolved until 100% agreement was reached. To examine differences between the two codebooks empirically, we compared the resulting codes (Figure 5) and discussed decision points, sharing the interactional evidence each coder drew on to assign them. Based on these discussions, amongst the coders and the broader author team, we noted specific interactions where engagement looked different across the coding schemes and identified several features of each scheme that seemed to shape our interpretations, which form the basis of the comparisons we present in the Findings below.

Figure 5

4 Findings

The chosen coding schemes differed in the theoretical alignment (our reason for selecting them), but also the level at which engagement was measured (individual versus group), the number and type of dimensions included in the coding scheme, the way that video was segmented, and how context was considered. Figure 5 presents the engagement ratings by the two coding schemes side by side. These coding schemes provided information about multiple dimensions of engagement from both a group and an individual level. For the most part, higher levels of individual COPIE-STEM engagement (constructive and interactive) seemed to align with higher levels of GDE disciplinary engagement in both contexts. However, there were also complex socioemotional, collaborative, and behavioral moment-to-moment interactions that shaped these dimensions, which each coding scheme highlighted or obscured in particular ways. Table 3 summarizes these differences across the two coding schemes. We unpack the sources of this divergence by detailing how each coding scheme made different components of students’ engagement visible. Finally, we present some of the challenges we experienced in applying these coding schemes to different learning contexts.

4.1 How researchers' choices shape understanding of group work (RQ1)

4.1.1 Level of analysis

COPIE-STEM measured engagement at an individual level, while GDE measured engagement at a full-group level. This meant that procedurally, as we coded using COPIE-STEM, we went through the video at least the number of times as there were students in the group, with a focus on a different student in the group each time. In comparison, while using GDE, we focused on the interactions between the group members, which often led to fewer watches.

Taking these individual and group level lenses to the data often resulted in different perceptions of students and what was going on from a learning perspective in given segments (see Figure 5 for the individual- and group-level codes assigned to this segment). COPIE-STEM made visible differences in individual's contributions to the group. Across the GEM-STEP data, Bob had the highest levels of constructive and interactive engagement in the group, followed by Sarah, and then Max, and then Peyton, who did not show any evidence of these dimensions (Figure 5). This was evident in the following exchange, where Bob and Sarah both engaged interactively with each other as they thought constructively about what happens during decomposition in order to determine which agents were producers, consumers, and decomposers (Table 4).

During this exchange, while Bob and Sarah were the only ones speaking, Max was also actively engaged: she looked at the speakers and at the picture of the model they were referencing. Peyton showed signs of both active and passive engagement– at times she listened and watched intently, while at other times she was turned away. COPIE-STEM made visible these individual contributions that each of the four students made to the sensemaking and the ways that the quality of their verbal contributions shifted over the course of the episode.

Looking at this same episode from a group level using GDE, engagement appeared to be quite high. Behavioral engagement was coded as high (because the content of the group's talk was on task), as was collaborative and socioemotional engagement (because the group was respectfully debating an idea amongst themselves). Therefore, taking a group level approach provided analysts with a broad idea/general feel of how things were working without unpacking any apparent differences across students. Ignoring these differences might be acceptable depending on the research question, but obscured Max and Peyton's lack of verbal contributions. Using GDE might lead analysts to explore how features of the task supported this high group-level engagement, while using COPIE-STEM might lead analysts to investigate how the task enabled these differences in engagement across the students.

4.1.2 Multidimensionality

COPIE-STEM measured one dimension of engagement (cognitive), while GDE measured five dimensions (behavioral, socioemotional, collaborative, metacognitive, and disciplinary). This meant that looking at codes alone, the information that analysts could perceive about students’ engagement during any given time segment was very different, and sometimes contradictory. An analyst using only COPIE-STEM would conclude that a group was disengaged or less engaged, while an analyst using only GDE would conclude that the same group was actively collaborating, and each would pursue a different next analytic step or claim about the group's collaboration.

For example, in EcoJourneys Episode 1, there were two segments where some students were coded as passively engaged (the lowest level of cognitive engagement), but there were high ratings on some GDE dimensions. Tom had opened a seemingly irrelevant picture on his computer, and nudged Jack to look. Since the questions in Deduce require all students in the group to express agreement, without Tom's participation, the group could not proceed, and this led Mary and Lisa to stop actively playing the game. Thus, based on COPIE-STEM's focus on the cognitive dimension, all the students were tagged as “passively engaged” at some point.

However, the majority of the group did attempt to maintain the flow of collaborative work, which was missed by attending only to cognitive engagement. For example, Jack tried to redirect Tom's attention by reminding him of the question, “Tom, greenish-brown and cloudy! This question!” The GDE framework noted that there was both a drop in behavioral, collaborative, and disciplinary engagement, and a rise in metacognitive engagement (as students tried to redirect Tom's attention) and socioemotional engagement (there were signs of good-natured and friendly interactions during and after the off-task behavior). This highlights the importance of recognizing the complex interplay between different dimensions of engagement. Returning to our goal of ensuring equitable participation, only examining cognitive engagement missed the contextual reasons why Mary and Lisa had stopped playing, as well as their efforts to redirect Tom's attention, and might lead to a potentially unfair deficit view of these students' engagement. Additionally, it suggests that brief off-task interactions should not always be seen as a negative aspect of group work as they might become an opportunity for students to build social connections, which can ultimately enhance their collaborative performance. Therefore, in this segment, taking a multidimensional approach to engagement enabled us to have a more contextualized view of “passive engagement”.

Returning to RQ1, this means that researchers' understanding of group work in CSCL environments depends on which dimensions and assumptions about engagement their coding scheme includes. A single-dimension scheme like COPIE-STEM risks missing the collaborative or socioemotional context that explains why a student appears disengaged, while a multidimensional group-level scheme like GDE risks masking individual struggle that a single-dimension scheme would catch.

4.1.3 Segmenting

Both COPIE-STEM and GDE required segmenting the videos using a fixed amount of time (1 min and 2.5 min segments respectively). While using GDE, we found that the longer segments often led to many moderate ratings because both low and high indicators were present in one segment. This is partly a consequence of how quickly engagement shifts in collaborative settings within a single 2.5-minute window, the result is often a moderate code that reflects the average of what occurred, which risk of flattening meaningful transitions into a single rating. This was also evident in coding collaborative engagement in EcoJourneys, which was coded as moderate for the entirety of Episode 2 (Figure 5). This meant that segments that looked qualitatively different to us in terms of collaboration ended up with the same moderate code. In the second segment, all four students in the group worked to decide on answers to questions. For example, Tom proposed an answer, “It's more”, which Lisa confirmed “more”, and then Mary requested elaboration “more and what?”, to which Jack responded “more and mixing”. This interaction was characteristic of their collaboration patterns across the segment, which was coded as moderate because although the students did not provide rationales for their answers, they made efforts to solicit perspectives and consensus. Contrastingly, in the third segment, which also received a moderate code for collaboration, there was limited coordination/checking of answers with the group. Instead, Mary, who the computer assigned to type the answer to the question, was just given and took up answers from the other students (e.g., “just put oxygen”). The first response being taken as the group response was one of the indicators of moderate on the rubric. Therefore, while collaboration looked quite different in these two segments, they were both labeled as moderate. We believe that with shorter time segments, it would have been easier to identify low and high collaborative engagement, rather than all the segments having intermittent or mixed interactions that led to moderate codes.

While both coding schemes used fixed-interval a priori segmentation, our analysis highlighted how the kind of engagement present was related to the task structure and the facilitation. For example, in the beginning of the GEM-STEP planning episode, as the facilitator was framing the task, engagement from a COPIE-STEM perspective was limited to active and some constructive dimensions, consistent with what we might expect as this portion of the activity was facilitator-led. As the conversation continued, students began to engage interactively with each other; however, interactive engagement did not occur in segments 6–9 for any of the students, perhaps because students were answering the facilitator rather than reasoning with each other. This raises questions about what contextual and task factors mediated students' level of cognitive engagement, what prompted change between levels, and whether segmenting by task structure may reveal different patterns than segmenting arbitrarily. We argue that pinpointing where these moments of change occur in the data via coding can offer analysts jumping-off points for deeper qualitative exploration of the moment-by-moment interactions that may have contributed to increased quality of cognitive engagement in discussions.

4.1.4 Theoretical framework

According to the authors of the two coding schemes, COPIE-STEM was designed from a cognitive perspective, while GDE was designed from a sociocultural one (Hsiao et al., 2022; Rogat et al., 2022). This difference manifested primarily in how the two coding schemes treated context. The indicators of engagement from a cognitive perspective in COPIE-STEM did not consider the activity or interactional context. In order for students to be coded at the highest level of cognitive engagement (interactive engagement), they needed to ask or answer questions regarding the learning materials. However, in portions of both GEM-STEP and EcoJourneys, the activity design did not afford opportunities for this kind of participation. In the GEM-STEP embodied modeling episode, students were oriented towards creating the model with their movements rather than discussing it. In EcoJourneys Episode 1, students were expected to remain quiet during individual investigation segments of the game. As such, students received low cognitive engagement ratings, even though they may have been engaged cognitively in ways that were not visible through their talk. Neglecting to take context into account may lead to deficit views of learners, in which their apparent “failure” is due to their own abilities rather than the ways that the contexts in which they are situated influence their participation and achievement (Lee, 2008).

Thus, in embodied contexts, indicators of high-quality cognitive engagement may need to include embodied elements like gaze, gesture, or movements that correspond to inquiry. The primacy of verbal contributions as indicators of cognitive engagement somewhat limited how much this coding scheme could reveal about the conceptual work students were engaged in during embodied phases of activity. Had coding focused on the ways in which students' movements demonstrated understanding, prompted conversation, or impacted the projected model, engagement would likely have looked very different. For example, as Bob controlled the bunny, he noticed that the plants that he was eating were not healthy, and asked Sarah, Max, and Peyton (the worms) to visit the plants. This utterance was coded as constructive, but not as interactive, because the other students did not respond to him verbally, but instead adjusted their movements to try to give nutrients to the plants. However, students were interacting through their actions. In contrast, students' disciplinary engagement (as rated by GDE) was considered high during embodied modeling, as the coding scheme required understanding what counted as disciplinary in context. Seeing low versus high engagement with the scientific content would lead researchers in very different directions for next analysis steps. This contrast reflects what each scheme was built to track guided by their theoretical orientation. COPIE-STEM centers on individual responses, while GDE centers on group-level disciplinary activity that heavily depends on how the task was structured.

4.2 Challenges of adapting engagement coding schemes across contexts (RQ2)

In this section, we explore some of the challenges that arose by using the coding schemes across different CSCL contexts for which they were not initially implemented.

4.2.1 Inconsistency in coding across activity structures

First, there were challenges in maintaining consistency in how dimensions were coded across activity structures, which continues to raise questions about the universality versus context specificity of coding schemes and their indicators. For example, student-led collaborative talk may not have the neat question-and-answer format that characterizes traditional teacher-led discussions, and so indicators of productive engagement may need to be adjusted to capture what high-quality engagement looks like in these more dialogic problem-solving interactions. Even though GDE was developed for student-centered inquiry contexts, we still needed to discuss what counted as the presence of different dimensions in each project context. This was particularly evident when coding disciplinary engagement across contexts, which requires coders to have working knowledge of the disciplinary content students are engaging with at the moment of interaction. Indicators of disciplinary engagement, such as the use of scientific terminology, the application of evidence to support a claim, or the construction of causal explanations, are inherently context-specific, because what counts as evidence-based reasoning in one task may look quite different from what counts as such in another.

4.2.2 Reliability challenges in the metacognitive dimension

Achieving reliability was also challenging, especially in measuring engagement in the metacognitive dimension. Metacognitive engagement refers to the degree to which students are aware of and able to regulate their own learning processes, such as setting goals, monitoring progress, and evaluating their own learning. During the training process, we had disagreements about whether students could demonstrate their metacognition through talk or observable behaviors. This disagreement was especially salient in episodes 1 and 2 from EcoJourneys, where student talk was often minimal or absent for stretches of the activity (Figure 5). In these moments, coders had to decide whether nonverbal indicators, such as a student's gaze shifting to a previous step or a gesture toward an upcoming task, were sufficient evidence that the student was aware of and monitoring their own learning process, or whether metacognition required some verbal trace to be coded with confidence. While we were able to clarify and agree on the final coding, it was still challenging to decide on codes.

Moreover, metacognitive engagement was used in this analysis as part of a group-level coding scheme, however, it was rare to observe indicators of metacognitive engagement from all students in a group. For example, in one segment from EcoJourneys, a single student demonstrated clear metacognitive regulation by announcing a shift in the group's process, saying “next we are going to do Deduce”, but no other group member verbally took up or responded to this statement. In a case like this, it was unclear whether the group as a whole should receive credit for metacognitive engagement on the strength of one student's contribution, or whether the absence of uptake from other members meant the group's shared awareness and regulation of the task could not actually be confirmed. This limitation can make it difficult to accurately measure metacognitive engagement across all group members, which may impact the validity and reliability of this dimension of the coding scheme. These disagreements also point to how coders' own assumptions about what counts as metacognition shaped the codes we ultimately applied.

4.2.3 Mismatch between fixed-interval segmentation and activity structure

Another challenge concerned the fixed-interval segmentation that both coding schemes required. Because COPIE-STEM and GDE were each developed with a different fixed segment length in mind, reflecting the activity structures of their original contexts, applying either scheme's native interval to a new context risked cutting across the activity's own structure. Figure 5 makes this mismatch visible. This mismatch also limited direct comparison between the two schemes within our own data. Because COPIE-STEM and GDE segment time differently, a single classroom episode could be parsed into different units depending on which scheme was applied, so an engagement code assigned under one scheme did not necessarily correspond to the same stretch of activity as a code assigned under the other. In our GEM-STEP data, segments that spanned the boundary between facilitator-led framing and student-led discussion mixed engagement signals from both portions, which made it harder to attribute a code to either part of the activity with confidence, and harder still to align that judgment with how the other scheme had segmented the same moment. Adapting a coding scheme to a new context may therefore require revisiting not just its indicators, but also its assumptions about how the activity itself is organized in time, particularly when the goal is to compare engagement across schemes rather than apply just one.

4.2.4 Theoretical grounding and portability across contexts

As noted above, the two coding schemes’ theoretical groundings also created different cross-context risks. Because COPIE-STEM is grounded in a cognitive perspective, it treats its indicators of engagement as properties of the individual learner that should be visible regardless of setting, which is what made it portable enough to apply directly to both GEM-STEP and EcoJourneys, but also what made it insensitive to the embodied and computer-mediated activity designs in which talk was not the primary mode of participation. GDE's sociocultural grounding, by contrast, treats engagement as constituted by the interactional context, which made it more sensitive to context-specific group activity but meant that applying it to a new setting required us to first determine what counted as disciplinary or collaborative in that particular activity, a determination that the framework itself does not specify in advance. As Table 4 summarizes, the cognitively-grounded scheme travelled across contexts more easily but flattened context-specific meaning, while the socioculturally-grounded scheme preserved context-specific meaning but required more interpretive work from coders before it could be applied at all.

5 Discussion

In this article, we coded the same data from two CSCL contexts with both a cognitively-grounded engagement coding scheme (COPIE-STEM) and a socioculturally-grounded coding scheme (GDE), with the intention of exploring the unique affordances of each and their potential synergies. As expected, COPIE-STEM oriented us towards the extent to which students engaged with surface level versus deep processing strategies. GDE also provided a level of disciplinary engagement with the content but expanded engagement to be distributed across the group and to include socioemotional and collaborative dimensions, echoing arguments that disciplinary engagement is jointly constructed rather than held by any one student (Agarwal and Sengupta-Irving, 2019; Engle and Conant, 2002; Gresalfi et al., 2009). However, in addition to differences in theory, the coding schemes also told different (sometimes complementary, sometimes contradicting) stories about engagement because of their differences in level of analysis, number of dimensions, and way of segmenting the video. This divergence speaks directly to an open debate in the engagement literature. Some researchers argue that engagement's dimensions are so intertwined that separating them is not analytically productive (Bergdahl et al., 2018; Renninger and Bachrach, 2015), while others treat them as distinct enough to measure independently (Fredricks et al., 2004). Our comparison suggests the answer is less a matter of which position is correct than a demonstration of Azevedo's (2015) caution that two studies invoking “engagement” may not be describing the same construct at all, even when coding identical footage. Together, the coding schemes provided a fuller picture of how individuals functioned within the group and how different dimensions of engagement interacted to produce moments of high-quality and low-quality engagement. We see here a connection to and the promise of exploring ways of traversing and synthesizing levels of analysis (Stahl, 2012; Stahl and Öner, 2013) and how we might productively pair coding schemes together during analysis.

It is important to note that the metacognitive dimension showed the lowest inter-rater reliability among all coded dimensions (Krippendorff's α = 0.05; Table 2), indicating effectively chance-level agreement between coders. We therefore treat this result as a methodological caution rather than a substantive finding. Patterns involving metacognitive engagement in this dataset should not be interpreted as stable or dependable, and we highlight this reliability challenge for researchers considering the application of GDE's metacognitive dimension, particularly in new contexts (see Section 4.2.2 for a discussion of the specific sources of coder disagreement). One possible explanation for this low reliability is the study's reliance on video data alone. Metacognitive engagement often involves processes that may not be directly observable through students' verbal and behavioral activity, making it particularly difficult to infer consistently from video (Järvelä et al., 2023). Research (e.g., Järvelä et al., 2023; Sharma et al., 2024) on adjacent constructs, such as socially shared regulation, similarly suggests that capturing regulatory and metacognitive processes may require integrating multiple sources of evidence such as log-trace and self-report data, alongside video to support more reliable identification of metacognitive engagement.

Additionally, there were significant challenges in applying both coding schemes to each context, raising questions about the context-dependency of indicators of engagement, consistent with arguments that what counts as engagement cannot be read off behavior alone but depends on the field of practice in which that behavior occurs (Godec et al., 2018), and that engagement is produced dynamically through the interaction of learner and environment rather than residing in the learner alone (Ryu and Lombardi, 2015). For example, in COPIE-STEM, constructive and interactive engagement required extensive student discussion. Yet, in computer-mediated environments like EcoJourneys and embodied environments like GEM-STEP, verbal talk may not be sufficient as an indicator of high-quality engagement. GDE included indicators like physicality and gaze, but it was difficult to obtain high reliability across coders for several dimensions. We noticed that the coder who was present at the data collection was able to rely upon extra contextual information and understanding of the activity in order to make coding decisions. This is consistent with Elliott's (2018) accounts of qualitative coding as a decision-making process in which choices about how to segment and interpret data are made by individual researchers in light of their methodological background, research design, and research questions, rather than by applying a fixed, study-independent rule set. This also highlights how the understanding and analysis of engagement is situated in particular interactional contexts, and that the researcher is also a part of this dynamic context, even as sideline observers. Consequently, researchers attempted to reconstruct some of these contextual factors during analysis, and the choices we made as researchers in selecting, adapting, and applying each coding scheme were themselves part of what shaped the engagement we were able to see.

In respond to the calls for equitable participation (e.g., Culver et al., 2024; Jennings et al, 2025; Neill et al., 2019) that motivated our study, our findings suggest that the two coding schemes captured, missed, and represented different aspects of how students engaged in small groups. COPIE-STEM captured individual variation in verbal contribution but, on its own, could miss group-level processes such as the socioemotional work that supports participation, while GDE captured the collaborative climate of a group but could mask individual students whose contributions fell away from view, a concern consistent with calls to attend to how power and positioning within a group shape whose disciplinary engagement gets recognized as such (Agarwal and Sengupta-Irving, 2019). For researchers interested in equitable participation specifically, this means that a coding scheme chosen without attention to these tradeoffs risks either overlooking students who are quietly excluded within an apparently engaged group or overstating the significance of an individual's silence without considering the group context that produced it.

Our findings also speak back to the broader engagement literature and the sociocultural perspectives raised in our literature review. Where Fredricks et al. (2004) and others treat engagement's dimensions as conceptually distinct, our results align more closely with sociocultural arguments that knowing, and by extension engagement, is a joint accomplishment of learner and context rather than a property held within an individual mind (Greeno et al., 1996; Sfard, 1998), and that engagement is constituted in and through interaction (Ryu and Lombardi, 2015). The same moment of group activity supported different readings depending on whether the coding scheme attended to individual cognition (Case, 1996) or group-level disciplinary participation, a divergence that echoes Azevedo's (2015) point that “engagement” is not a single construct waiting to be measured more or less accurately, but a family of related constructs whose boundaries are drawn differently by different theoretical commitments.

Putting these coding schemes side-by-side made it clear that engagement at a group level is not equivalent to the sum or average of each individual's engagement. We therefore argue that how we label moments matters. Rather than using words like “off-task” or “disengaged”, we suggest that it is important to take an asset-based approach to inferring and interpreting activity in the video (Jennings et al., 2025; Ocumpaugh et al., 2024). As we look toward future measures of engagement, we argue that it is important to understand both individual and group levels, and how learners' engagement flows between these levels across different dimensions.

Finally, looking across theoretical frameworks reminds us that some of the assumptions we make when we develop coding schemes are not “required” by the framework (e.g., foregrounding embodiment, using codes that background history, prior knowledge, or the sociocultural context). Our analysis demonstrated that different ways of coding engagement in video obscure or make visible different understandings of the group work, and looking across these understandings makes the analysis richer.

5.1 Limitations and future directions

There are several limitations and future directions for this work. Most fundamentally, this study is exploratory in scope. We coded only 12 min of video from each context, a sample selected specifically to allow a close, contrastive comparison of how two coding schemes handled the same stretches of activity. Given this small, purposively selected sample, the frequencies, dimension scores, and reliability statistics reported throughout this article should be read as illustrative of the methodological and conceptual tensions we set out to investigate, not as stable, generalizable estimates of how students engage in these two contexts, let alone in similar learning environments more broadly. The brevity of the dataset also constrained the range of behaviors coders had the opportunity to observe within each dimension: several dimensions, including metacognitive engagement, showed little variability across the 12-minute segments, which in turn depressed the reliability statistics we were able to compute and made it difficult to distinguish genuine coding disagreement from the effects of a small number of codable instances. Future work applying and comparing these coding schemes should draw on longer and more varied samples, ideally spanning multiple sessions per context, both to establish more stable reliability estimates and to determine whether the patterns we observed here hold at scale.

Additionally, in this study, we selected only one sociocultural and one cognitive coding scheme, and we wonder how tied the findings are to the coding schemes and theoretical groundings themselves. For example, COPIE-STEM only considers cognitive engagement, but there are cognitively grounded coding schemes that also include emotional engagement (e.g., Sun et al., 2022). It would be interesting to explore dimensions in addition to cognitive engagement at an individual level. Moreover, using these two frameworks in an embodied learning context helped to highlight that embodiment is present to varying degrees in all contexts, but often not attended to in coding. This suggests that it could be useful to explore a framework designed for use in these contexts (e.g., Danish et al., 2022), but also that researchers using non-embodied frameworks think about what might be getting hidden by excluding embodied indicators of engagement.

6 Conclusion

References

• 1

  AgarwalP.Sengupta-IrvingT. (2019). Integrating power to advance the study of connective and productive disciplinary engagement in mathematics and science. Cogn. Instr.37, 349–366. 10.1080/07370008.2019.1624544

  • CrossRef
  • Google Scholar
  • View reference in article

• 2

  AppletonJ. J.ChristensonS. L.FurlongM. J. (2008). Student engagement with school: critical conceptual and methodological issues of the construct. Psychol. Sch.45, 369–386. 10.1002/pits.20303

  • CrossRef
  • Google Scholar
  • View reference in article

• 3

  AzevedoR. (2015). Defining and measuring engagement and learning in science: conceptual, theoretical, methodological, and analytical issues. Educ. Psychol.50, 84–94. 10.1080/00461520.2015.1004069

  • CrossRef
  • Google Scholar
  • View reference in article

• 4

  BangM.BrownB.Calabrese BartonA.RoseberyA. S.WarrenB. (2017). “Toward more equitable learning in science,” in Helping Students Make Sense of the World Using Next Generation Science and Engineering Practices, eds. SchwarzC.PassmoreC.ReiserB. J. (Arlington, VA: National Science Teachers Association), 33–58.

  • Google Scholar
  • View reference in article

• 5

  BergdahlN.ForsU.HernwallP.KnutssonO. (2018). The use of learning technologies and student engagement in learning activities. Nord. J. Digit. Lit.13, 113–130. 10.18261/issn.1891-943x-2018-02-04

  • CrossRef
  • Google Scholar
  • View reference in article

• 6

  BliksteinP.WorsleyM. (2016). Multimodal learning analytics and education data mining: using computational technologies to measure complex learning tasks. J. Learn. Anal.3, 220–238. 10.18608/jla.2016.32.11

  • CrossRef
  • Google Scholar
  • View reference in article

• 7

  BoschN. (2016). “Detecting student engagement: human versus machine,” in Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization, 317–320. 10.1145/2930238.2930371

  • CrossRef
  • Google Scholar
  • View reference in article

• 8

  CaseR. (1996). “Changing views of knowledge and their impact on educational research and practice,” in The Handbook of Education and Human Development: New Models of Learning, Teaching and Schooling, eds. OlsonD. R.TorranceN. (Cambridge, MA: Blackwell Publishing), 75–99.

  • Google Scholar
  • View reference in article

• 9

  ChangoW.LaraJ. A.CerezoR.RomeroC. (2022). A review on data fusion in multimodal learning analytics and educational data mining. Wiley Interdiscip. Rev.12, e1458. 10.1002/widm.1458

  • CrossRef
  • Google Scholar
  • View reference in article

• 10

  ChenY.-C.TeradaT. (2021). Development and validation of an observation-based protocol to measure the eight scientific practices of the next generation science standards in K-12 science classrooms. J. Res. Sci. Teach.58, 1489–1526. 10.1002/tea.21716

  • CrossRef
  • Google Scholar

• 11

  ChiM. T. (2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Top. Cogn. Sci.1, 73–105. 10.1111/j.1756-8765.2008.01005.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 12

  ChiM. T.WylieR. (2014). The ICAP framework: linking cognitive engagement to active learning outcomes. Educ. Psychol.49, 219–243. 10.1080/00461520.2014.965823

  • CrossRef
  • Google Scholar
  • View reference in article

• 13

  CulverK. C.PerezR. J.KitchenJ. A.ColeD. G. (2024). Fostering equitable engagement: a mixed-methods exploration of the engagement of racially diverse students in a comprehensive college transition program. J. Divers. High. Educ.17, 200. 10.1037/dhe0000408

  • CrossRef
  • Google Scholar
  • View reference in article

• 14

  DanishJ. A.GresalfiM. (2018). “Cognitive and sociocultural perspective on learning: tensions and synergy in the learning sciences,” in International Handbook of the Learning Sciences, eds. FischerF.Hmelo-SilverC. E.GoldmanS. R.ReimannP. (New York, NY: Routledge), 34–43.

  • Google Scholar
  • View reference in article

• 15

  DanishJ. A.AntonG.MathayasN.JenT.VickeryM.LeeS. (2022). Designing for shifting learning activities. J. Appl. Instruct. Des.11, 169–185. 10.51869/114/jdabc

  • CrossRef
  • Google Scholar
  • View reference in article

• 16

  De LoofH.StruyfA.Boeve-de PauwJ.Van PetegemP. (2021). Teachers’ motivating style and students’ motivation and engagement in STEM: the relationship between three key educational concepts. Res. Sci. Educ.51, 109–127. 10.1007/s11165-019-9830-3

  • CrossRef
  • Google Scholar
  • View reference in article

• 17

  D’MelloS.DieterleE.DuckworthA. (2017). Advanced, analytic, automated (AAA) measurement of engagement during learning. Educ. Psychol.52, 104–123. 10.1080/00461520.2017.1281747

  • CrossRef
  • Google Scholar
  • View reference in article

• 18

  ElliottV. (2018). Thinking about the coding process in qualitative data analysis. Qual. Rep.23, 2850–2861. 10.46743/2160-3715/2018.3560

  • CrossRef
  • Google Scholar
  • View reference in article

• 19

  EngleR. A.ConantF. R. (2002). Guiding principles for fostering productive disciplinary engagement: explaining an emergent argument in a community of learners classroom. Cogn. Instr.20, 399–483. 10.1207/S1532690XCI2004_1

  • CrossRef
  • Google Scholar
  • View reference in article

• 20

  FormanE. A. (2018). “The practice turn in learning theory and science education,” in Constructivist Education in an Age of Accountability, ed. KrittD. (Cham: Palgrave Macmillan), 97–111.

  • Google Scholar

• 21

  FosterJ. K.KorbanM.YoungsP.WatsonG. S.ActonS. T. (2024). Automatic classification of activities in classroom videos. Comput. Educ. Artif. Intell.6, 100207. 10.1016/j.caeai.2024.100207

  • CrossRef
  • Google Scholar

• 22

  FredricksJ. A.McColskeyW. (2012). “The measurement of student engagement: a comparative analysis of various methods and student self-report instruments,” in Handbook of Research on Student Engagement, ChristensonS. LReschlyA. L.WylieC. (New York, NY: Springer US), 763–782.

  • Google Scholar

• 23

  FredricksJ. A.BlumenfeldP. C.ParisA. H. (2004). School engagement: potential of the concept, state of the evidence. Rev. Educ. Res.74, 59–109. 10.3102/00346543074001059

  • CrossRef
  • Google Scholar

• 24

  FredricksJ.McColskeyW.MeliJ.MordicaJ.MontrosseB.MooneyK. (2011). Measuring Student Engagement in Upper Elementary Through High School: A Description of 21 Instruments. Issues & Answers. REL 2011-No. 098. Washington, DC: Regional Educational Laboratory Southeast.

  • Google Scholar

• 25

  FriedL. J.KonzaD. M. (2013). Using self-determination theory to investigate student engagement in the classroom. Int. J. Pedagogy Curric.19, 27–40. 10.18848/2327-7963/CGP/v19i02/48898

  • CrossRef
  • Google Scholar

• 26

  GodecS.KingH.ArcherL.DawsonE.SeakinsA. (2018). Examining student engagement with science through a bourdieusian notion of field. Sci. Educ.27, 501–521. 10.1007/s11191-018-9988-5

  • CrossRef
  • Google Scholar
  • View reference in article

• 27

  GreenoJ. G.CollinsA. M.ResnickL. B. (1996). “Cognition and learning,” in Handbook of Educational Psychology, eds. BerlinerD. C.CalfeeR. C. (New York: NY: Macmillan), 15–45.

  • Google Scholar

• 28

  GresalfiM. S.BarnesJ. (2016). Designing feedback in an immersive videogame: supporting student mathematical engagement. Educ. Technol. Res. Dev.64, 65–86. 10.1007/s11423-015-9411-8

  • CrossRef
  • Google Scholar

• 29

  GresalfiM.BarabS.SiyahhanS.ChristensenT. (2009). Virtual worlds, conceptual understanding, and me: designing for consequential engagement. On Horizon17, 21–34. 10.1108/10748120910936126

  • CrossRef
  • Google Scholar
  • View reference in article

• 30

  Guerrero-SosaJ. D.RomeroF. P.Menéndez-DomínguezV. H.Serrano-GuerreroJ.Montoro-MontarrosoA.OlivasJ. A. (2025). A comprehensive review of multimodal analysis in education. Appl. Sci.15, 5896. 10.3390/app15115896

  • CrossRef
  • Google Scholar
  • View reference in article

• 31

  HallA.MiroD. (2016). A study of student engagement in project-based learning across multiple approaches to STEM education programs. Sch. Sci. Math.116, 310–319. 10.1111/ssm.12182

  • CrossRef
  • Google Scholar
  • View reference in article

• 32

  HenrieC. R.HalversonL. R.GrahamC. R. (2015). Measuring student engagement in technology-mediated learning: a review. Comput. Educ.90, 36–53. 10.1016/j.compedu.2015.09.005

  • CrossRef
  • Google Scholar
  • View reference in article

• 33

  HsiaoJ.-C.ChenS.-K.ChenW.LinS. S. J. (2022). Developing a plugged-in class observation protocol in high-school blended STEM classes: student engagement, teacher behaviors and student-teacher interaction patterns. Comput. Educ.178, 104403. 10.1016/j.compedu.2021.104403

  • CrossRef
  • Google Scholar
  • View reference in article

• 34

  JärveläS.NguyenA.VuorenmaaE.MalmbergJ.JärvenojaH. (2023). Predicting regulatory activities for socially shared regulation to optimize collaborative learning. Comput. Human. Behav.144, 107737. 10.1016/j.chb.2023.107737

  • CrossRef
  • Google Scholar
  • View reference in article

• 35

  JenningsK.DayerA.ChavesW. (2025). Building equitable engagement: strategies for enhancing diverse engagement in participatory science. Conserv. Sci. Pract.7, e70129. 10.1111/csp2.70129

  • CrossRef
  • Google Scholar
  • View reference in article

• 36

  JewittC. (2015). “Multimodal analysis,” in The Routledge Handbook of Language and Digital Communication, eds. GeorgakopoulouA.SpiliotiT. (New York: NY: Routledge), 69–84.

  • Google Scholar
  • View reference in article

• 37

  LeeC. D. (2008). The centrality of culture to the scientific study of learning and development: how an ecological framework in education research facilitates civic responsibility. Educ. Res.37, 267–279. 10.3102/0013189X08322683

  • CrossRef
  • Google Scholar
  • View reference in article

• 38

  LuW.YangY.SongR.ChenY.WangT.BianC. (2025). A video dataset for classroom group engagement recognition. Sci. Data12, 644. 10.1038/s41597-025-04987-w

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 39

  MiyakeN. (2007). “Computer supported collaborative learning,” in Sage Handbook of E-Learning Research, eds. AndrewR.HaythornwaiteC. (London: Sage), 248–268.

  • Google Scholar
  • View reference in article

• 40

  NassauerA.LegewieN. (2022). Video Data Analysis: How to Use 21st Century Video in the Social Sciences. London: SAGE Publications.

  • Google Scholar

• 41

  National Research Council (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Washington, DC: National Academies Press.

  • Google Scholar
  • View reference in article

• 42

  NeillC.CotnerS.DriessenM.BallenC. J. (2019). Structured learning environments are required to promote equitable participation. Chem. Educ. Res. Pract.20, 197–203. 10.1039/C8RP00169C

  • CrossRef
  • Google Scholar
  • View reference in article

• 43

  OcumpaughJ.RoscoeR. D.BakerR. S.HuttS.AguilarS. J. (2024). Toward asset-based instruction and assessment in artificial intelligence in education. Int. J. Artif. Intell. Educ., 1–40. 10.1007/s40593-023-00382-x

  • CrossRef
  • Google Scholar
  • View reference in article

• 44

  RenningerK. A.BachrachJ. E. (2015). Studying triggers for interest and engagement using observational methods. Educ. Psychol.50, 58–69. 10.1080/00461520.2014.999920

  • CrossRef
  • Google Scholar
  • View reference in article

• 45

  RogatT. K.Hmelo-SilverC. E.ChengB. H.TraynorA.AdeoyeT. F.GomollA.et al (2022). A multidimensional framework of collaborative groups’ disciplinary engagement. Front. Learn. Res.10, 1–21. 10.14786/flr.v10i2.863

  • CrossRef
  • Google Scholar
  • View reference in article

• 46

  RyuS.LombardiD. (2015). Coding classroom interactions for collective and individual engagement. Educ. Psychol.50, 70–83. 10.1080/00461520.2014.1001891

  • CrossRef
  • Google Scholar
  • View reference in article

• 47

  SalehA.Hmelo-SilverC. E.GlazewskiK. D.MottB.ChenY.RoweJ. P.et al (2019). Collaborative inquiry play: a design case to frame integration of collaborative problem solving with story-centric games. Inf. Learn. Sci.120, 547–566. 10.1108/ILS-03-2019-0024

  • CrossRef
  • Google Scholar
  • View reference in article

• 48

  SchmidtJ. A.RosenbergJ. M.BeymerP. N. (2018). A person-in-context approach to student engagement in science: examining learning activities and choice. J. Res. Sci. Teach.55, 19–43. 10.1002/tea.21409

  • CrossRef
  • Google Scholar
  • View reference in article

• 49

  SfardA. (1998). On two metaphors for learning and the dangers of choosing just one. Educ. Res.27, 4–13. 10.3102/0013189X027002004

  • CrossRef
  • Google Scholar
  • View reference in article

• 50

  SharmaK.GiannakosM. (2020). Multimodal data capabilities for learning: what can multimodal data tell US about learning?Br. J. Educ. Technol.51, 1450–1484. 10.1111/bjet.12993

  • CrossRef
  • Google Scholar
  • View reference in article

• 51

  SharmaK.NguyenA.HongY. (2024). Self-regulation and shared regulation in collaborative learning in adaptive digital learning environments: a systematic review of empirical studies. Br. J. Educ. Technol.55, 1398–1436. 10.1111/bjet.13459

  • CrossRef
  • Google Scholar
  • View reference in article

• 52

  SinatraG. M.HeddyB. C.LombardiD. (2015). The challenges of defining and measuring student engagement in science. Educ. Psychol.50, 1–13. 10.1080/00461520.2014.1002924

  • CrossRef
  • Google Scholar
  • View reference in article

• 53

  SkinnerE.KindermannT.FurrerC. (2009). A motivational perspective on engagement and disaffection: conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educ. Psychol. Meas.69, 493–525. 10.1177/0013164408323233

  • CrossRef
  • Google Scholar
  • View reference in article

• 54

  StahlG. (2012). Traversing planes of learning. Int. J. Comput. Support. Collab. Learn.7, 467–473. 10.1007/s11412-012-9159-7

  • CrossRef
  • Google Scholar
  • View reference in article

• 55

  StahlG.ÖnerD. (2013). “Resources for connecting levels of learning,” in To See the World and a Grain of Sand: Learning Across Levels of Space, Time, and Scale: CSCL 2013 Conference Proceedings Volume 1 — Full Papers & Symposia, eds. RummelN.KapurM.NathanM.PuntambekarS. (Madison, WI: International Society of the Learning Sciences), 454–461.

  • Google Scholar
  • View reference in article

• 56

  StahlG.KoschmannT.SuthersD. D. (2006). “Computer-supported collaborative learning: a historical perspective,” in Cambridge Handbook of the Learning Sciences, ed. SawyerR. K. (New York: Cambridge University Press), 409–426.

  • Google Scholar
  • View reference in article

• 57

  SteinbergS.DanishJ. (2024). “We were doing science, not just talking about science”: embodied learning and science identity development,” in Proceedings of the 18th International Conference of the Learning Sciences-ICLS 2024 (International Society of the Learning Sciences), 306–313.

  • Google Scholar

• 58

  SteinbergS.Hmelo-SilverC. E.ZouV.DanishJ.LesterJ. (2023). “Seeing student engagement in classroom video: affordances of cognitive and sociocultural frameworks,” in Proceedings of the 16th International Conference on Computer-Supported Collaborative Learning-CSCL 2023 (International Society of the Learning Sciences), 123–130.

  • Google Scholar
  • View reference in article

• 59

  SunJ.AndersonR. C.LinT.-J.MorrisJ. A.MillerB. W.MaS.et al (2022). Children’s engagement during collaborative learning and direct instruction through the lens of participant structure. Contemp. Educ. Psychol.69, 102061. 10.1016/j.cedpsych.2022.102061

  • CrossRef
  • Google Scholar
  • View reference in article

• 60

  TestaI.CostanzoG.CrispinoM.GalanoS.ParlatiA.TaralloO.et al (2022). Development and validation of an instrument to measure students’ engagement and participation in science activities through factor analysis and Rasch analysis. Int. J. Sci. Educ.44, 18–47. 10.1080/09500693.2021.2010286

  • CrossRef
  • Google Scholar
  • View reference in article

• 61

  WiedbuschM.DeverD.LiS.AmonM. J.LajoieS.AzevedoR. (2023). “Measuring multidimensional facets of SRL engagement with multimodal data,” in Unobtrusive Observations of Learning in Digital Environments: Examining Behavior, Cognition, Emotion, Metacognition and Social Processes Using Learning Analytics, eds. KovanovicV.AzevedoR.GibsonD. C.lfenthalerD. (Cham: Springer International Publishing), 141–173.

  • Google Scholar
  • View reference in article

• 62

  ZangoriL.PinnowR. J. (2020). Positioning participation in the NGSS era: what counts as success?J. Res. Sci. Teach.57, 623–648. 10.1002/tea.21607

  • CrossRef
  • Google Scholar
  • View reference in article

• 63

  ZouX.HongD.ChenF.Hmelo-SilverC. E.GlazewskiK.WangT.et al (2024). “Examine student resource uses in a game-based CSCL,” in Proceedings of the 17th International Conference on Computer-Supported Collaborative Learning-CSCL 2024 (International Society of the Learning Sciences), 185–188.

  • Google Scholar
  • View reference in article