Enhancing the coding experience in primary school with pepper: learning gains, cognitive workload, and student engagement

Abstract

Introduction:

This study examines a Pepper-based coding activity in primary education through three complementary dimensions: changes in coding performance, perceived cognitive workload, and pupils' post-activity engagement responses. Rather than testing the causal effectiveness of Pepper in isolation, the study investigates how a robot-mediated coding activity can be implemented and evaluated in an authentic classroom context.

Methods:

Twenty-nine third-grade pupils (8–9 years old) from an Italian primary school participated in a teacher-orchestrated coding activity in which they programmed the Pepper robot to navigate a floor grid. Coding performance was assessed using curriculum-aligned pre- and post-tests scored on a 0–10 scale. The perceived workload was measured with a child-adapted NASA Task Load Index (NASA-TLX) questionnaire. Post-activity engagement responses were collected using an adapted version of the User Engagement Scale-Short Form (UES-SF), with particular caution in interpretation due to the limited internal consistency observed for several subscales.

Results:

The coding scores increased significantly from the pre-test (M = 5.83, SD = 3.09) to the post-test (M = 7.97, SD = 2.54), t(28) = 3.75, p < 0.001, with a size of the effect size of medium-to-large paired-samples (d_z = 0.70). NASA-TLX responses indicated low mental and temporal demand, moderate physical demand, high perceived performance, and low frustration, suggesting that the activity was perceived as manageable by the pupils. The adapted engagement questionnaire provided limited descriptive evidence of positive post-activity responses, especially on the Reward subscale, while the other UES-SF dimensions were interpreted cautiously due to weak reliability.

Discussion:

The findings provide preliminary classroom-based evidence that a Pepper-mediated coding activity was associated with increased coding scores, manageable perceived workload, and positive descriptive indications of perceived reward. However, because of the single-group pre–post design, small sample size, unequal pre- and post-test formats, short-term scope, and limitations of the adapted engagement measure, the results should not be interpreted as causal or generalizable evidence of Pepper's effectiveness. Instead, the study offers methodological and design insights for future comparative research on robot-mediated coding activities in primary education.

1 Introduction

In the last decade, Computational Thinking (CT) has been increasingly recognized as a foundational competence, on par with literacy and numeracy, for full participation in a digital society (Wing, 2006; Grover and Pea, 2013; Shute et al., 2017). CT encompasses a set of interrelated skills, such as abstraction, decomposition, pattern recognition, and algorithm design, that support problem solving across domains, beyond Computer Science alone (Grover and Pea, 2013; Barr and Stephenson, 2011). Therefore, integrating CT into compulsory education has become a policy priority in many countries, and the European Commission explicitly stresses its role in preparing citizens for future labor markets and democratic participation (Bocconi et al., 2016).

Despite this growing attention, a substantial proportion of European citizens still lack basic digital skills: recent data show that only 56% of people aged 16–74 have at least basic competencies, well-below the targets set by the Digital Decade of the EU (Eurostat, 2023; Vuorikari et al., 2016). Primary school represents a crucial window of opportunity to address this gap: children aged 7–9, who typically transition from preoperative to concrete operational stages of cognitive development, benefit from structured hands-on activities that make abstract concepts more concrete (Brennan and Resnick, 2012; Piaget and Inhelder, 1973). Early exposure to CT through age-appropriate tasks is associated with improvements in logical reasoning, creativity, and performance in STEM subjects (Lye and Koh, 2014; Sengupta et al., 2013; Bers et al., 2014).

In this context, coding has become one of the most widely adopted approaches to foster CT in schools. Block-based environments such as Scratch allow children to translate algorithmic ideas into executable instructions while working with manipulable visual representations (Resnick et al., 2009; Kafai and Burke, 2014). Research shows that well-designed programming activities can support not only technical skills, but also collaboration, perseverance, and resilience, especially when teachers scaffold the learning process and gradually remove their support (Barr and Stephenson, 2011; Werner et al., 2012; Denner et al., 2014). However, teacher preparedness, unrestricted access to resources, and gender-based participation patterns still represent major challenges for the large-scale and inclusive adoption of coding in formal education (Denner et al., 2014; Yadav et al., 2016; Sullivan and Bers, 2016).

Alongside software-based tools, the toy and educational technology market has introduced a growing number of programmable artifacts designed to make CT tangible. Floor robots and simple programmable toys allow children to encode routes using directional commands, providing immediate feedback on the correctness of their “programs.” Bee-Bot^Ⓡ is one of the most widely distributed examples and has been successfully adopted in primary classrooms to introduce basic sequencing and programming concepts (Lanzilotti et al., 2021; Pérez-Marín et al., 2022). These solutions lower the entry barrier for young learners and have been associated with positive effects on engagement and basic coding skills (Atmatzidou and Demetriadis, 2016; Ragusa and Leung, 2023).

More recently, research in educational robotics has begun to explore the role of social humanoid robots as learning companions. Systematic reviews show that social robots can function as tutors, peers, or novices, supporting language learning, STEM activities, and collaborative tasks in various educational settings (Mubin et al., 2013; Belpaeme et al., 2018; Hein and Nathan-Roberts, 2018). Studies with platforms such as NAO and RoboThespian report positive effects on motivation, attention, and classroom climate, especially when robots are used to provide short structured activities that complement teacher-guided instruction (Verner et al., 2016; Feidakis et al., 2023; Buchem, 2023). At the same time, the appearance, interaction style, and level of autonomy of the robot can influence the engagement and cognitive performance of learners, as highlighted by work on adaptive and non-compliant behaviors in social robots (Ahmad et al., 2017; Maggi et al., 2021).

Pepper is a humanoid social robot originally produced by SoftBank Robotics and equipped with a tablet, speech synthesis, movement capabilities, and sensors that enable multimodal interaction. Although Pepper is no longer commercially available, it remains a relevant research platform to examine the broader design principles of robot-mediated learning, particularly when the learning activity relies on embodiment, visible movement in physical space, and immediate feedback. In the present study, Pepper was not selected as a commercially unique or irreplaceable solution, but rather as an available humanoid platform that allowed pupils to translate directional commands into observable robot movements on a floor grid.

Previous studies have used Pepper in educational contexts to explore collaborative learning scenarios, language activities, and playful applications in which children “teach” the robot or learn alongside it (Tanaka et al., 2015; Ramachandran et al., 2018). However, empirical evidence on humanoid social robots explicitly supporting coding activities in primary school remains limited, especially when coding performance, perceived workload, and engagement-related responses are examined together. The study by Lanzilotti et al. (2021) introduced Pepper as a social robot to teach coding in primary school, showing promising results in terms of feasibility and motivational impact. This earlier work was also documented and extended in the field study reported by Labollita (2020), which described the classroom implementation of Pepper-based coding activities with primary school pupils. However, these previous contributions left open questions about how robot-supported coding activities are associated with pupils' coding performance, perceived task demands, and post-activity engagement responses.

The work presented in this article builds on that initial trial and on the more extensive analysis reported in Labollita (2020)'s thesis, which investigated Pepper use in a structured coding course for 8–10-year-old students. In particular, our study adopts a pre–post test design to measure students' coding competences before and after a Pepper-supported intervention and complements these data with validated instruments to capture cognitive workload and engagement. Cognitive load is assessed using the NASA Task Load Index (NASA-TLX), widely used to evaluate perceived workload in human-technology interaction (Hart and Staveland, 1988), while engagement is measured through the User Engagement Scale (UES-SF) short form, which has been adapted to learning and interactive media contexts (O'Brien and Toms, 2010).

From a theoretical perspective, the relevance of a robot-mediated coding activity can be understood through the lens of cognitive load and embodied interaction. For young learners, coding requires the coordination of abstract instructions, spatial reasoning, sequencing, and error checking. A humanoid robot moving on a physical grid may help make these abstract relations more concrete by linking each command to an observable action in space. In this sense, embodiment and immediate feedback may reduce some forms of extraneous cognitive load by externalizing the consequences of pupils' instructions. At the same time, the robot may also introduce additional demands, such as interpreting the robot's orientation, managing the novelty of the interaction, and coordinating attention between the tablet interface, the grid, and the robot's movements. For this reason, coding performance alone is insufficient to evaluate the educational value of such activities: workload and engagement need to be considered as complementary but distinct dimensions of the learning experience. Consequently, the study does not aim to demonstrate the causal effectiveness of Pepper in isolation but to provide an exploratory multidimensional account of a Pepper-mediated coding activity in an authentic classroom setting.

By integrating coding performance with perceived workload and engagement-related responses, this study addresses a gap in existing research on social robots and coding education. To structure the investigation, we formulate the following research questions:

• RQ1: What changes in pupils' coding performance are observed between the pre-test and the post-test following a Pepper-based coding activity?

• RQ2: How do pupils perceive the workload associated with the Pepper-based coding activity, as measured through a child-adapted NASA-TLX questionnaire?

• RQ3: What descriptive indications of post-activity engagement and perceived reward emerge from the adapted UES-SF responses, considering the psychometric limitations of the adapted instrument in this child sample?

The contribution of this study is threefold. First, it provides preliminary classroom evidence of changes in student coding scores after a Pepper-mediated activity, extending previous work on robot-supported CT and STEM education with a post quantitative perspective (Atmatzidou and Demetriadis, 2016; Ragusa and Leung, 2023). Second, it integrates learning scores with perceived workload and post-activity engagement responses, highlighting the importance of examining whether robot-mediated coding activities are not only motivating but also cognitively manageable for young learners. Third, it derives design implications for teachers and instructional designers interested in embedding humanoid or tangible robots into primary school coding curricula, while explicitly acknowledging the exploratory nature of the evidence and the need for future comparative studies.

The remainder of the paper is organized as follows. Section 2 reviews related work on CT, educational robotics, social robots, and the role of cognitive load and engagement in technology-enhanced learning. Section 3 describes the study context, participants, instruments, procedure, and data analysis. Section 4 reports the results of the coding performance, perceived workload, engagement responses, and exploratory associations among the measured variables. Section 5 discusses the findings in relation to the research questions and derives design implications for robot-mediated coding activities. Section 6 outlines the main limitations and directions for future research, and Section 7 concludes the paper.

2 Background and related work

2.1 Computational thinking and coding in primary education

The term CT was first mentioned by Papert in his 1980 book “Mindstorms: Children, Computers, and Powerful Ideas,” where he described how children could use programming as a “microworld” to explore mathematical ideas and develop powerful mental models through constructionist activities (Papert, 1980). However, its modern formulation and widespread adoption come from Wing (2006)'s article, which framed CT as a fundamental skill that should be taught to everyone, not only computer scientists, highlighting problem decomposition, abstraction, and algorithmic reasoning as key components of literacy of the 21st century.

In recent decades, these ideas have progressively influenced school curricula, including primary education, where coding activities are introduced to support CT and general problem-solving skills. Visual and block-based programming environments were developed to lower the barrier to entry for young learners, building on Logo and its graphical successors (Resnick et al., 2009). By manipulating blocks that represent commands and control structures, children can focus on the logic of their programs without being overwhelmed by syntax.

However, integrating CT and coding into primary school still presents practical challenges. Teachers must design tasks that are cognitively appropriate for 7–10-year-old pupils, provide sufficient scaffolding, and connect programming activities to curricular goals. For younger learners, activities that are concrete, hands-on and closely tied to the physical world tend to be more effective than purely screen-based tasks (Strawhacker and Bers, 2015). This has motivated an increasing interest in educational robotics, where coding is directly linked to the behavior of a tangible object.

2.2 Educational robotics and tangible programming

Educational robotics uses programmable artifacts, ranging from simple floor robots to more complex kits, to help children understand sequences, loops, and conditional structures by observing how their programs control physical devices. Several studies have shown that such activities can positively influence CT skills, as well as collaboration and motivation, particularly when robotics is integrated into regular classroom practice (Atmatzidou and Demetriadis, 2016).

Early floor robots such as Bee-Bot popularized the idea of tangible programming in primary school. Pupils enter sequences of directional commands (e.g., forward, left, right) and then watch the robot execute the program on a mat or grid. This concrete visualization of abstract instructions supports the development of spatial reasoning and step-by-step planning. Similar approaches have been extended to more advanced platforms, where children design and debug programs that control robot movements, sensors, and actuators.

Within this line of work, the design of the learning activity is crucial: tasks must be challenging enough to stimulate higher-order thinking, but not so complex as to overload pupils' limited working memory. Moreover, the role of the teacher (as facilitator, co-designer of tasks, and mediator of discussion) remains central to ensure that robotics is not perceived merely as a toy, but as a meaningful learning tool.

2.3 Social and humanoid robots in education

A specific strand of educational robotics focuses on social robots that engage in social interaction through speech, gestures, facial expressions, and other cues. Reviews of social robots in education highlight their potential to act as tutors, peers, or learning companions, capable of sustaining attention, providing feedback, and fostering inclusion (Mubin et al., 2013; Belpaeme et al., 2018). Social robots have been deployed in various domains, including language learning, storytelling, and STEM subjects, with promising results in terms of motivation and classroom participation.

At the same time, the introduction of a social robot in the classroom can introduce new cognitive and interactional demands. The students must interpret the robot's behavior, adapt to its perspective in space, and coordinate their own actions with those of the robot and their peers. Research on adaptive and non-compliant robot behaviors has shown that the way a robot responds to students' actions can influence their engagement and learning strategies, for example, by encouraging self-regulation when the robot deliberately refuses to provide immediate help (Maggi et al., 2021).

These findings suggest that social robots should not be treated merely as “attractive gadgets,” but as pedagogical tools whose design must carefully balance engagement with cognitive demands. Therefore, studies are needed that go beyond anecdotal reports of enjoyment and systematically evaluate coding performance, perceived workload, and engagement-related responses.

2.4 Pepper in coding and STEM education

Pepper is a humanoid robot equipped with a tablet on its chest, speech synthesis, and a range of sensors that enable rich interaction. Its physical embodiment and expressive capabilities make it suitable for scenarios in which the robot acts as a social partner rather than a purely functional device. In educational settings, Pepper has been used in activities such as storytelling, language learning, and collaborative games, often with children in primary or secondary school (Tanaka et al., 2015; Ramachandran et al., 2018).

In the domain of STEM and CT, Pepper has been explored as a mediator to introduce basic programming concepts. For example, studies have described scenarios in which children program Pepper to move in a physical environment or perform simple tasks, using graphical interfaces on the robot's tablet (Diyas et al., 2016; Tazhigaliyeva et al., 2016). These activities draw on the same principles as floor-robot programming, but extend them through the presence of a humanoid body and a more complex interaction space.

However, empirical evidence on Pepper explicitly supporting coding in primary school, with a clear focus on curriculum-aligned learning goals, remains limited. Previous work conducted with the same research group that developed this study investigated Pepper as a tool to support coding and CT, highlighting pupils' enthusiasm and the feasibility of integrating the robot into classroom practice, but without fully quantifying the pre–post changes in coding performance and perceived cognitive workload (Lanzilotti et al., 2021; Labollita, 2020). The present work builds on that foundation by adopting a pre–post evaluation of coding performance and by systematically examining pupils' perceived workload and post-activity engagement-related responses during a Pepper-based coding activity. At the same time, Pepper should be understood as one possible instance of a broader class of embodied and programmable learning artifacts. Other humanoid robots, floor robots, programmable boards, and tangible or graphical environments may support similar coding objectives through different forms of embodiment, feedback, and interaction. Therefore, examining Pepper-based activities can provide design insights that are potentially transferable beyond the specific platform, while also requiring caution because the technical and interactional affordances of each platform may shape pupils' experience differently.

2.5 Cognitive load, embodied interaction, and engagement in robot-supported coding

In technology-enhanced learning, the effectiveness of an activity cannot be evaluated only in terms of motivation or immediate performance. This is particularly relevant in primary education, where pupils are still developing the cognitive resources required to coordinate abstract reasoning, spatial orientation, task planning, and self-regulation. Cognitive load theory distinguishes between the intrinsic complexity of the learning task, the extraneous load imposed by the way information and interaction are designed, and the cognitive resources productively devoted to learning (Sweller, 2011; Chakraborty et al., 2023; Laurie-Rose et al., 2014). From this perspective, coding activities for young learners should be designed so that core programming concepts are made accessible without overwhelming pupils' working memory.

Robot-supported coding activities are especially interesting in this regard because they combine abstract symbolic instructions with spatial and embodied feedback. When pupils program a robot to move on a physical grid, commands such as forward, left, and right are no longer only symbolic elements in a sequence: they become observable actions performed by an embodied artifact in shared space. This may help pupils externalize the consequences of their instructions, compare expected and actual results, and review their programs through concrete feedback. In this sense, robot-mediated coding can reduce some forms of extraneous cognitive load by making the relationship between instruction, execution, and error more visible (O'Brien and Toms, 2018; Andrade et al., 2019).

At the same time, embodiment can also introduce additional demands. Pupils may need to coordinate their own perspective with the robot's orientation, monitor the tablet interface, follow the robot's movement on the grid, and interpret the social cues produced by the robot. In social robot contexts, these demands are not only cognitive but also interactional. The robot's behavior, interaction style, and degree of autonomy may shape how learners allocate attention, regulate their actions, and respond to moments of uncertainty. Previous work has shown that social robot interaction styles can influence user cognitive performance, acceptance, and non-compliant behavior (Tazhigaliyeva et al., 2016). This suggests that engagement with a robot should not be interpreted as a purely motivational phenomenon but as part of a broader interactional configuration that may either support or burden learning.

The relationship between engagement and learning is therefore complex. Social robots can attract attention, generate curiosity, and make classroom activities more memorable, but enjoyment and perceived reward do not necessarily imply deeper learning. In short-term studies, positive engagement responses can also reflect novelty effects, social desirability, or the unusual nature of the activity rather than stable educational impact. For this reason, engagement indicators must be interpreted together with coding performance measures and workload data, and with particular caution when self-report instruments are adapted for children.

Within this framework, the present study examines a Pepper-based coding activity by considering three complementary dimensions: changes in coding performance, perceived cognitive workload, and post-activity engagement responses. This activity builds on the classroom implementation documented in previous Pepper-based coding work (Labollita, 2020), but extends it by explicitly integrating coding scores, workload, and engagement-related responses. The aim is not to isolate the causal effect of Pepper as a technology, but to understand how an embodied social robot can be integrated into a teacher-orchestrated coding activity and how pupils experience such an activity in terms of performance, perceived effort, and reward.

3 Methodology

3.1 Participants

The study was conducted in two third-grade classes (III A and III B) of the “Giovanni Falcone” primary school in Adelfia (Bari, Italy). The sample included 29 pupils (13 boys, 16 girls), all between 8 and 9 years old. All participants had previously acquired basic knowledge and skills related to coding through traditional unplugged activities conducted by the same team of teachers.This ensured that the two classes shared comparable teaching conditions (same teachers with the same teaching approach and the same syllabus), helping to minimize variability due to differences in teaching style or curricular coverage. Participation was voluntary and required parental consent. No incentives were provided. All pupils who were present on the days of data collection and whose parents had provided their consent participated in the study.

3.2 Instruments

3.2.1 Demographic and technology-usage questionnaire

Before the learning activities, pupils completed a brief paper questionnaire consisting of eight open and closed-ended questions. The questionnaire collected information on:

• age, class, and gender;

• types of digital devices available at home (e.g., smartphone, tablet, computer, game console);

• frequency of use of each device;

• perceived competence in using each device;

• previous experiences with robots (e.g., toy robots, educational robots).

This instrument was used to characterize the digital profile of the sample and to contextualize subsequent analyzes. The descriptive results (reported in the Results section) showed, for example, that all pupils used smartphones regularly, tablets were the second most used device, and computers were the least used.

3.2.2 Coding pre-test and post-test

Learning performance was examined using a domain-specific pre- and post-testing design aligned with the coding objectives addressed in the classroom activities. The tests were designed by the research team in collaboration with classroom teachers and focused on basic coding and CT skills appropriate for third-grade students, including sequencing, path planning on a grid, decomposition of a route into discrete movements, and reasoning about alternative paths.

The pre-test consisted of nine exercises. These included tasks in which pupils had to identify or complete sequences of directional commands, plan a route on a grid from a starting point to a target position, and reason about possible corrections when a proposed route did not reach the target. The pre-test was administered in the pupils' classroom a few days before the Pepper-based activity, lasted approximately 30 min, and was completed individually.

The post-test consisted of three structured exercises targeting the same curricular constructs as the pre-test: problem solving on a grid, coding comprehension, and the application of directional commands in a novel scenario. The post-test was administered a few days after the Pepper-based activity, under conditions comparable to the pre-test, lasted approximately 30 min, and was completed individually in the classroom.

For both tests, the pupils' responses were scored according to a rubric prepared before data analysis. The rubric assigned credit for the accuracy of the planned path, the appropriate ordering of commands, and the ability to decompose the route into discrete steps. The raw scores were then rescaled to a common scale of 0–10, where higher values indicated better coding performance. The same scoring criteria were applied to all pupils. The Pre-test and post-test responses were matched using anonymous pupil codes, so individual learning trajectories could be analyzed without using identifiable information.

Although the two tests targeted the same curriculum-aligned coding objectives and were normalized on the same 0–10 scale, they differed in the number of items (nine in the pre-test and three in the post-test). Therefore, they should be considered classroom-based assessment instruments rather than psychometrically equivalent parallel forms. This design choice limits the precision with which pre–post score increases can be estimated and is acknowledged as a methodological limitation. Consequently, pre–post differences are interpreted as indicative changes in coding scores rather than as definitive evidence of stable competence acquisition.

3.2.3 Cognitive workload questionnaire (NASA-TLX)

Consistent with the theoretical framework of workload as a key dimension of robot-supported coding activities, the perceived workload of the pupils was collected using an adapted version of the NASA-TLX (Tanaka et al., 2015). The instrument was used to capture pupils' post-activity perception of task demands rather than to provide an objective measure of cognitive load. Because the questionnaire was completed after the post-test, responses may reflect pupils' overall recollection of the Pepper-based activity and the subsequent assessment session, rather than the robot interaction alone. The questionnaire included six items corresponding to the standard NASA-TLX dimensions:

• Mental Demand

• Physical Demand

• Temporal Demand

• Performance

• Effort

• Frustration

Each item was presented on a ten-point Likert-type scale adapted for children (0 = very low, 10 = very high). Age-appropriate explanations were provided orally and in written form to clarify the meaning of each dimension. For example, Mental Demand referred to how much pupils felt they had to think during the task, Temporal Demand referred to whether they felt hurried, and Frustration referred to whether they felt stressed or annoyed. When necessary, teachers or researchers clarified the meaning of the scale without suggesting any response.

For analysis, we distinguished between:

• A Core Workload index, computed as the mean of the five dimensions related to workload (Mental Demand, Physical Demand, Temporal Demand, Effort and Frustration);

• Performance, treated separately as an indicator of self-reported satisfaction with one's own performance, with higher scores indicating higher perceived satisfaction.

The internal consistency of the adapted scale was assessed using Cronbach's alpha. For the five dimensions related to workload, alpha was acceptable (α≈0.73), indicating that it captured a coherent perception of workload. Including Performance in the composite index yielded a lower alpha (α≈0.59), which is why Performance was analyzed separately. Descriptive statistics are reported for each dimension and for the core workload index.

3.2.4 User Engagement Scale–Short Form (UES-SF)

Post-activity engagement responses were collected using an adapted version of the User Engagement Scale–Short Form (UES-SF) (Ramachandran et al., 2018). The instrument was used as a descriptive post-activity questionnaire rather than as a fully validated engagement scale for this child sample. This distinction is important because UES-SF was originally developed and validated mainly in adult digital interaction contexts, and its direct use with primary school pupils requires caution.

The adapted version comprised 11 items grouped into four indices:

• Focused Attention (FA) - Items 7–9, capturing the extent to which pupils felt absorbed in the activity and focused on the interaction.

• Perceived Usability (PU) - Items 10–12, focusing on perceived ease of interaction, sense of control, and absence of confusion while using the interface.

• Aesthetic Appeal (AE) - Items 13–14, assessing pupils' perception of the attractiveness of the robot and the tablet interface.

• Reward (RW) - Items 15–17, assessing perceived enjoyment, interest, and perceived value of the activity.

Items were linguistically simplified and explained using age-appropriate language. The response format was adapted to make it understandable for 8–9-year-old pupils. During questionnaire completion, teachers and researchers were allowed to clarify the meaning of items or response options, but were instructed not to guide pupils toward any specific answer.

Mean scores were calculated for each index. However, because the adapted version differed from the original UES-SF and because reliability was weak for several subscales, the resulting scores were primarily interpreted as descriptive response patterns. In particular, the Reward subscale was considered the most interpretable engagement-related indicator because it showed acceptable internal consistency in this sample, while the other indices were treated with substantial caution.

The internal consistency of the adapted UES-SF was heterogeneous. Cronbach's alpha for the Reward subscale was acceptable (α≈0.70), but the alpha values were lower for the remaining subscales and for the overall 11-item scale. The Aesthetic Appeal index, based on only two items, showed a negative alpha, indicating inconsistent responses across those items. Because Cronbach's alpha is also limited when applied to very short subscales, especially two-item indices, these reliability results further support a cautious descriptive interpretation of the UES-SF data. The internal consistency indices for the adapted NASA-TLX and UES-SF scales are summarized in Table 1.

3.3 Procedure

The study followed a structured sequence of activities designed to integrate the Pepper robot into an authentic primary school context.

3.3.1 Pre-test and demographic/technology-usage questionnaire

In the first session, all pupils completed the coding pre-test (nine exercises, 0–10 score) and the demographic/technology-usage questionnaire in their classroom. The session lasted approximately 30 min and was conducted under the supervision of the teacher of the class and the research team.

3.3.2 Pepper-based coding activity

Two days after the pre-test, the Pepper-based coding activity took place in the school gymnasium. A large grid was drawn on the floor to serve as the physical environment for Pepper's movements (see Figure 1). The grid provided a shared spatial representation in which pupils could observe the relationship between directional commands and the robot's movements.

Figure 1

Each pupil, in turn, was asked to program Pepper using directional arrows displayed on the tablet mounted on the robot's chest (see Figure 2). The goal was to move Pepper from a fixed starting position to a target square marked with an “X” on the grid. The target position was changed over turns, so that pupils had to plan a route rather than simply reproduce the same sequence of commands. The task required pupils to decompose the path into a sequence of discrete movements, order the commands correctly, and take into account the robot's orientation on the grid.

Figure 2

The activity was conducted as a teacher-orchestrated classroom session rather than as an isolated individual laboratory task. Pupils were present in the gymnasium with their classmates and could observe the activity while each child directly interacted with Pepper. To preserve the individual nature of each programming attempt, classmates were not expected to provide the solution to the pupil currently interacting with the robot. Teachers and researchers managed turn-taking, clarified procedural instructions when needed, and ensured that pupils understood how to use the tablet interface. Assistance was limited to procedural clarification and classroom management; adults did not provide the sequence of commands required to solve the task.

As the pupil selected each directional arrow, the corresponding instruction was appended to the program shown on the tablet, allowing the learner to visualize the emerging sequence of commands before execution. After the program was launched, Pepper ran the sequence on the grid, providing immediate embodied feedback. If the robot did not reach the target, the mismatch between the expected and actual movement could be observed directly and discussed in terms of sequence planning, orientation, and debugging.

The overall activity lasted approximately 3 h, during which all 29 pupils had the opportunity to interact directly with Pepper. At the end of the session, a brief brainstorming discussion was conducted to collect spontaneous reflections and impressions of the pupils about the activity, as well as informal feedback from the teachers on feasibility and perceived educational value (see Figure 3).

Figure 3

3.3.3 Post-test and questionnaires

A few days after the Pepper session, pupils completed the coding post-test in their classroom, under conditions comparable to those of the pre-test. Immediately after completing the post-test, they filled in the combined NASA-TLX and adapted UES-SF questionnaires, for a total of 17 items. The questionnaire session lasted approximately 30 min.

Items were read aloud or clarified when necessary to ensure comprehension of the response scales. Clarifications were limited to explaining the meaning of the items and the response options; teachers and researchers did not suggest specific answers. Because the workload- and engagement-related questionnaires were administered after the post-test rather than immediately after the robot interaction, responses may reflect pupils' overall recollection of the Pepper-based activity and the assessment experience. This timing is therefore considered when interpreting workload- and engagement-related results.

3.4 Data analysis

Quantitative analyzes focused on three main aspects: (i) changes in coding performance, operationalized through pre–post coding scores, (ii) perceived cognitive workload, and (iii) post-activity engagement-related responses, as well as exploratory relations between these variables.

• Changes in coding performance (RQ1): pre-test and post-test scores (0–10 scale) were summarized using means and standard deviations, collapsed across the two classes. The difference between the pre-test and post-test scores was analyzed using a paired-samples t-test, with a significance level of α = 0.05. The distribution of difference scores was assessed using the Shapiro–Wilk test to evaluate the appropriateness of the paired-samples t test. As a robustness check, a Wilcoxon signed-rank test was also conducted to verify whether the direction and significance of the pre–post change were consistent under a non-parametric approach. The effect size was estimated using Cohen's d_z, calculated as the mean paired difference divided by the standard deviation of the difference scores. The 95% confidence interval for the mean difference was also reported.

• Cognitive workload (RQ2): for each NASA-TLX dimension and for the core workload index (mean of Mental, physical, and temporal demand, effort, and frustration), we computed means and standard deviations. These statistics were used to characterize perceived workload and to examine whether the activity was perceived as cognitively manageable. The Core Workload index was also used in exploratory correlation analyzes with coding performance indicators.

• Post-activity engagement responses (RQ3): for each adapted UES-SF index (FA, PU, AE, and RW), we computed the mean and standard deviations. In light of the heterogeneous internal consistency of the adapted instrument, these results were interpreted as descriptive response patterns rather than as validated engagement scores. Particular attention was paid to the Reward subscale, which showed acceptable reliability, while the remaining indices were interpreted with substantial caution.

• Exploratory relations between workload, engagement-related responses, and learning performance: as a secondary, exploratory step, we computed Pearson correlations between the Core Workload index, the Reward subscale, and coding performance indicators (pre-test scores, post-test scores, and gain scores). Spearman correlations were also inspected as a robustness check. Given the small sample size, the absence of correction for multiple exploratory associations, and the limited reliability of several adapted engagement indices, these correlations were interpreted cautiously and are reported mainly to generate hypotheses for future research rather than to support causal claims. The main exploratory correlations between workload, reward, and coding performance are reported in Table 2.

Missing values were minimal and handled listwise in each analysis. All analyzes were conducted at the student level.

3.5 Ethical considerations

The study involved 29 third-grade pupils aged 8–9 years of age from the “Giovanni Falcone's primary school in Adelfia, Bari, Italy. The activity was conducted in collaboration with school administration and classroom teachers as a low-risk educational coding activity using the Pepper robot. Formal ethics committee approval was not obtained because, under the applicable local institutional procedures at the time of data collection, the study was treated as a school-based educational activity that did not involve medical, clinical, psychological, or invasive procedures, deception, sensitive personal data, or risks exceeding those normally encountered in classroom learning activities.

Written informed consent for participation was obtained from the parents or legal guardians of the pupils. Pupils were informed in an appropriate age that participation was voluntary and that they could withdraw at any time without consequences. To match pre-test and post-test responses while preserving confidentiality, pupils were assigned anonymous codes used only for data linkage. Identifying information was not included in the analysis dataset. All data were anonymized before analysis and no identifiable information is reported about individual pupils. In the figures included in the manuscript, pupils' faces were blurred to avoid disclosure of identifiable visual data.

The study was conducted according to the principles of the Declaration of Helsinki, the European Code of Conduct for Research Integrity, the UN Convention on the Rights of the Child, and applicable institutional and national requirements for research involving minors.

4 Results

4.1 Technology profile of the sample

Descriptive analyzes of the demographic and technology-usage questionnaire showed that all students reported regular access to digital devices at home. Smartphones were the most used device, followed by tablets, whereas computers were the least used. Most pupils reported feeling confident using smartphones and tablets, while self-reported computer skills were more variable. Only a minority of pupils had previously interacted with robots (mainly toy robots), and none had prior experience with Pepper. These results suggest that the sample can be characterized as digitally active, with everyday exposure to mobile devices but limited prior experience with educational robots.

4.2 Changes in coding performance (RQ1)

The descriptive statistics for the pre-test and post-test coding scores are reported in Table 3. Across the two classes, the mean pre-test score was 5.83 (SD = 3.09), while the mean post-test score was 7.97 (SD = 2.54) on a scale of 0–10 (see Figure 4). These results describe changes in classroom-based coding test scores following the Pepper-based activity; given the single-group design, they should not be interpreted as isolating the causal effect of Pepper.

Figure 4

A paired-samples t-test on the difference scores showed a statistically significant increase from pre-test to post-test, t(28) = 3.75, p < 0.001. The mean gain was 2.14 points (SD_diff = 3.07), corresponding to a medium-to-large paired-samples effect size (Cohen's d_z = 0.70). The 95% confidence interval for the mean difference ranged from 0.97 to 3.30 points. The distribution of the difference scores did not deviate significantly from normality according to the Shapiro–Wilk test (p = 0.249), supporting the use of the paired samplet-test. A Wilcoxon signed-rank test was also conducted as a non-parametric robustness check and confirmed the pre–post increase, with the sum of positive ranks exceeding the sum of negative ranks (V = 282.00, p = 0.001). Taken together, these results indicate a measurable increase in coding scores after the activity, while remaining compatible with the exploratory and non-controlled nature of the study.

An exploratory analysis by class suggested that pupils in Class III B exhibited a clearer increase in mean scores between the pre- and post-test, whereas pupils in Class III A started from a higher baseline and showed little observable gain. This pattern is consistent with a possible ceiling effect in Class III A, where some pupils already achieved relatively high scores at pre-test. No inferential comparison between classes was used to support the main claims, because these class-level patterns were exploratory and based on very small subgroups. At the same time, the marked difference between the two classes reinforces the need for caution when interpreting the aggregate pre–post change. At the same time, the marked difference between the two classes reinforces the need for caution when interpreting the aggregate pre–post change. Overall, the results address RQ1 by showing that coding scores increased after the Pepper-based activity, without allowing the increase to be attributed uniquely to Pepper.

4.3 Cognitive workload (RQ2)

The mean scores for the six NASA-TLX dimensions are reported in Table 4 and illustrated in Figure 5. In general, the responses of the pupils after the activity suggest that the coding task was perceived as cognitively accessible.

Figure 5

Mental Demand received low scores (M≈2.21 on a 10-point scale), suggesting that the pupils did not experience the task as mentally taxing. The temporal demand was also low (M≈3.31), indicating that the activity pace was not perceived as hurried or stressful. In contrast, Physical Demand obtained somewhat higher scores (M≈3.90), reflecting the embodied and spatial nature of the task, which required pupils to reason about movements on the grid and align their own perspective with the robot's orientation.

In the Performance dimension, pupils reported high satisfaction (M≈8.38), indicating that they generally felt that they had performed well. Effort and Frustration ratings were relatively low, suggesting that although the task required concentration, it did not trigger a substantial negative affect or perceived stress.

In addition to the six individual dimensions, a Core Workload index was calculated by averaging Mental Demand, Physical Demand, Temporal Demand, Effort, and Frustration. The mean value of this index indicated an overall low-to-moderate level of perceived workload, consistent with the pattern observed on individual dimensions. Given its acceptable internal consistency (Cronbach's α≈0.73), this index was used in exploratory correlational analyzes with coding performance indicators.

Taken together, these results address RQ2 by suggesting that the pupils perceived the activity as manageable: the task required some effort, particularly in relation to its embodied and spatial components, but did not appear to generate high levels of perceived mental workload or frustration. Because the questionnaire was administered after the post-test, these findings should be interpreted as post-activity perceptions rather than as a direct isolated measure of the Pepper interaction alone.

4.4 Post-activity engagement responses (RQ3)

The descriptive statistics for the four adapted UES-SF indices, Focused Attention (FA), Perceived Usability (PU), Aesthetic Appeal (AE), and Reward (RW), are presented in Table 5 and summarized in Figure 6. Given the heterogeneous internal consistency of the adapted instrument, these scores are interpreted as descriptive post-activity response patterns rather than as validated engagement measures.

Figure 6

The Reward index showed the highest mean score (M = 4.47, SD = 0.91), suggesting that pupils tended to describe the activity as enjoyable, interesting, and worthwhile. This interpretation is the most reliable among the adapted UES-SF indices because the Reward subscale showed acceptable internal consistency in this sample. Aesthetic Appeal also showed a relatively high mean score (M = 4.03, SD = 1.04), but this index should be interpreted with particular caution because of its weak psychometric performance. Focused Attention and Perceived Usability showed lower mean values and should also be treated as descriptive indicators rather than as reliable subscale scores.

These findings address RQ3 by providing limited descriptive evidence of positive post-activity responses, especially in relation to perceived reward. However, they do not allow strong conclusions about engagement as a validated multidimensional construct, nor do they imply that enjoyment or perceived reward translated directly into learning. Qualitative comments collected during the final brainstorming are consistent with this descriptive interpretation: pupils described the session as “fun,” “different from usual lessons,” and “interesting because the robot really moved,” while teachers appreciated the way Pepper helped make abstract coding concepts more concrete.

4.5 Exploratory relations between workload, reward, and coding performance

To explore potential relations between perceived workload, reward, and coding performance, correlation analyzes were conducted between the Core Workload index, the Reward subscale, and coding performance indicators (pre-test scores, post-test scores, and gain scores). Given the small sample size, the adapted nature of self-report measures, and the exploratory nature of these analyzes, the correlations were interpreted as hypothesis-generating indicators rather than as evidence of causal or generalizable relationships. The descriptive statistics and correlation coefficients for these exploratory analyzes are summarized in Table 2.

Exploratory analyzes involving the Core Workload index and gain scores indicated a negative association, in which pupils who reported a higher overall workload tended to show smaller pre–post increases. Given the small sample size, the self-report nature of the workload measure, and the absence of a control group, this result is interpreted cautiously and considered hypothesis-generating rather than conclusive.

The correlations involving the Reward subscale were positive but weak with respect to the post-test scores and gains. Because Reward was the only adapted UES-SF subscale with acceptable internal consistency and because the remaining engagement-related indices showed weak reliability, the exploratory analysis was limited to Reward rather than used to support broader claims about engagement. Therefore, these results do not provide evidence that perceived reward systematically improved learning within this sample.

Overall, the results provide a cautious and preliminary picture: coding scores increased after the Pepper-based activity, pupils' post-activity workload responses suggested a manageable perceived task demand, and the adapted engagement questionnaire offered limited descriptive evidence of positive perceived reward. These findings should be interpreted in light of the exploratory design, the lack of a control group, and the measurement limitations discussed below.

5 Discussion

This study examined a Pepper-based coding activity in primary education, focusing on three research questions related to changes in coding performance (RQ1), perceived workload (RQ2), and responses related to post-activity participation (RQ3). The findings provide a preliminary and cautious picture of a robot-mediated coding activity implemented in an authentic classroom context. Coding scores increased from pre-test to post-test, pupils' workload responses suggested that the activity was perceived as manageable, and the adapted engagement questionnaire provided limited descriptive evidence of positive perceived reward. Given the exploratory single-group design, these findings should be interpreted as associations observed after the activity rather than as causal evidence of Pepper's effectiveness.

5.1 Changes in coding performance (RQ1)

The pre–post analysis of coding scores indicates a statistically significant increase after the Pepper-based activity, with a medium-to-large paired-sample effect size. This result suggests that the activity was associated with measurable changes in classroom-based coding performance of pupils, particularly in tasks that involve sequencing, path planning, and decomposition of a route into discrete steps. However, because the study did not include a control group, the observed increase cannot be uniquely attributed to Pepper. It should instead be interpreted as preliminary evidence that a teacher-orchestrated robot-mediated coding activity can be feasibly associated with positive pre–post score changes in an authentic classroom context. These results are consistent with previous work on educational robotics, which reports positive but context-sensitive effects of tangible programming activities on CT and problem-solving skills in primary school (Resnick et al., 2009; Atmatzidou and Demetriadis, 2016; Papert, 1980).

It is important to note that pupils in both classes had already been exposed to unplugged coding activities before the Pepper session. In this sense, Pepper did not introduce coding “from scratch,” but was embedded in a broader instructional trajectory. The observed score increase may therefore reflect consolidation and application of previously introduced concepts, rather than the acquisition of entirely new competences during a single robot-based session. This interpretation is aligned with a progressive view of CT education, in which different representations, including unplugged, screen-based, tangible, and robotic activities, are combined to support increasingly sophisticated forms of reasoning (Wing, 2006; Brennan and Resnick, 2012).

The descriptive differences between the two classes also require caution. Class III B showed a clearer pre–post increase, whereas Class III A started from a higher baseline and showed little observable gain. This pattern may be compatible with a ceiling effect in Class III A, but it may also reflect differences in prior competence, classroom dynamics, or other contextual factors that could not be controlled in the present design. This reinforces the importance of using larger samples, comparison conditions, and more robust assessment instruments in future studies, rather than relying on aggregate prevpost differences alone.

5.2 Cognitive workload in a robot-supported coding task (RQ2)

NASA-TLX results suggest that pupils perceived the Pepper-based task as cognitively accessible: mental and temporal demands were rated low, perceived performance was rated high, and frustration was minimal. The only dimension with moderately higher scores was physical demand, which appears to reflect the embodied and spatial nature of the task, including the need to interpret Pepper's orientation and to map directional commands onto movements in the physical grid. The acceptable internal consistency of the Core Workload index (α≈0.73) supports its use as a descriptive indicator of the demands of perceived tasks in this sample. However, because the questionnaire was administered after the post-test, these responses should be interpreted as post-activity perceptions rather than as a direct isolated measure of the Pepper interaction alone.

From a cognitive load perspective, this profile is compatible with the idea that the task was sufficiently structured to remain manageable for 8–9-year-old pupils. The physical grid, fixed starting point, visible target, and immediate robot movement may have helped externalize the relationship between commands and results, thus supporting pupils in planning and revising paths. At the same time, the present data do not allow us to determine whether Pepper reduced cognitive load compared with a non-robotic or less embodied activity. The results therefore indicate perceived manageability within this specific classroom configuration, rather than a comparative advantage of Pepper over alternative tools.

The finding that physical demand is somewhat higher invites careful consideration in the design of similar activities. While embodied interaction is one of the strengths of educational robotics, tasks that require frequent perspective shifts (e.g., interpreting left/right from the robot's point of view) can be demanding for younger children. In our setting, the use of a large floor grid, clear visual markers, and teacher support likely helped mitigate these difficulties. Future designs could further scaffold orientation by, for example, explicitly discussing frames of reference with pupils or providing additional visual cues on the grid.

Exploratory correlations did not reveal strong systematic relationships between individual workload dimensions and post-test scores. A negative association emerged between the Core Workload index and gain scores, suggesting that pupils who reported higher overall workload tended to show smaller pre–post increases. However, this result should only be treated as hypothesis-generating, given the small sample size, the nature of self-reporting the workload measure, the timing of questionnaire administration and the absence of a control group. Future comparative studies are needed to clarify whether perceived workload moderates learning in robot-mediated coding activities.

5.3 Post-activity engagement responses and perceived reward (RQ3)

The adapted UES-SF data provide limited descriptive evidence of positive post-activity responses, especially in relation to perceived reward. Pupils tended to describe the activity as pleasant, interesting and worthwhile, and these responses are consistent with previous studies that emphasize the motivational potential of social robots in educational settings (Feidakis et al., 2023; Buchem, 2023; Hart and Staveland, 1988). However, the present findings should not be interpreted as robust evidence of engagement as a validated multidimensional construct, because most adapted UES-SF subscales showed weak internal consistency in this child sample (Mubin et al., 2013; Belpaeme et al., 2018; Tanaka et al., 2015).

At the same time, the internal consistency of the adapted UES-SF was heterogeneous: the reliability was acceptable for the Reward subscale, but lower for the other subscales and for the overall scale, with a negative alpha for the two-item Aesthetic Appeal index. Consequently, the adapted UES-SF should be treated as a descriptive post-activity questionnaire rather than as a validated measure of engagement in this sample. The Reward dimension provides the most interpretable engagement-related evidence, while Focused Attention, Perceived Usability, and Aesthetic Appeal should be discussed as descriptive response patterns.

The combination of positive reward responses and increased coding scores is encouraging, but it does not rule out novelty effects. Interacting with a humanoid robot in a gym-based coding activity was an unusual experience for the pupils, and their positive responses may partly reflect curiosity, social desirability, or the short-term motivational wave generated by the activity. At the same time, the task required real problem solving because pupils had to plan command sequences, consider orientation, and observe whether Pepper reached the target. Future longitudinal studies are needed to examine whether perceived reward and attention remain stable once the novelty of interacting with a humanoid robot diminishes.

The qualitative feedback collected during the final brainstorming is consistent with the descriptive rewards results: pupils described the session as more stimulating and “different” from usual lessons, and teachers valued the way Pepper helped make abstract concepts more concrete. These comments should be interpreted as contextual evidence of pupils' and teachers' perceptions, not as independent proof of improved coding performance. Nevertheless, they align with previous observations that social robots can catalyze classroom discussion and provide a shared, tangible focus for collaborative reasoning (Lanzilotti et al., 2021; Labollita, 2020).

5.4 Design implications for robot-supported coding in primary school

Taken together, the findings suggest several design implications for teacher-orchestrated robot-mediated coding activities in primary school. These implications should be understood as design-oriented reflections derived from an exploratory classroom study, rather than as general prescriptions about the effectiveness of Pepper or social robots.

First, robot-mediated coding activities may be more meaningful when built on prior unplugged experiences. In our study, Pepper was introduced after the pupils had already participated in unplugged coding activities. This sequencing may have helped pupils apply previously introduced concepts in a new embodied context: unplugged tasks can introduce core ideas in a low-tech, low-stakes environment, while the robot-based activity provides a concrete setting for applying and refining these ideas. Second, designers should use embodied tasks with clear spatial scaffolds. The floor grid, fixed starting position, and clearly marked targets helped pupils reason about the paths and see the consequences of their programs. Such scaffolds are particularly important when learners must coordinate between the robot's perspective and their own. Third, it is crucial to calibrate task difficulty to maintain a manageable perceived workload. In the present study, low mental demand and high perceived performance suggest that the task was perceived as accessible by many pupils. Future implementations should consider both the complexity of the grid and the number of steps required, avoiding configurations that would require excessively long or error-prone sequences, especially when pupils must also coordinate their own perspective with the robot's orientation. Fourth, teachers and designers should leverage the robot's social presence without over-automation. In this study, Pepper did not “solve” tasks for pupils, but served as a responsive artifact whose behavior depended on their command sequences. This aligns with previous work suggesting that the interaction style and autonomy of social robots can influence users'cognitive performance, acceptance, and behavior, and that robots should therefore support, rather than replace, learners' agency (Maggi et al., 2021; Papert, 1980). Finally, it is important to position teachers as co-orchestrators. The presence and guidance of teachers was essential for classroom management, clarification of instructions, and mediation of discussions. Successful integration of robots in education therefore requires viewing the robot as one element in a broader pedagogical orchestration, not as a standalone solution or as an automatic driver of coding performance.

6 Limitations and future work

Several limitations of this exploratory classroom study should be acknowledged. These limitations are important for interpreting the findings as preliminary evidence of Pepper-mediated coding activity, rather than as causal or generalizable evidence of Pepper's effectiveness.

First, the research involved a relatively small sample (29 pupils) from a single primary school, and the design did not include a control group. As a result, the observed pre–post increase in coding scores cannot be attributed uniquely to Pepper. Alternative explanations include previous unplugged instruction, familiarity with the test, maturation, classroom dynamics, novelty of the activity, and the enthusiasm generated by interacting with a humanoid robot in the gymnasium. Future work should therefore adopt quasi-experimental or experimental designs with larger samples and comparison conditions, such as a similar coding task without a robot, a task mediated by a non-humanoid artifact, or an activity using a different educational robotics platform. Such designs would make it possible to examine more precisely whether and how humanoid embodiment contributes to pupils' coding performance.

Second, although both pre- and post-tests were aligned with the same curricular objectives and their scores were normalized on a common 0–10 scale, the two instruments differed in the number of items (nine in the pre-test and three in the post-test). This limits their psychometric comparability and the precision with which pre–post score increases can be estimated. The tests should therefore be understood as classroom-based curriculum-aligned assessment instruments rather than as equivalent standardized forms. Subsequent studies should refine the assessment design by using parallel forms with matched difficulty and length, explicitly reporting reliability indices, and, where possible, incorporating item-level analyzes, such as item response theory, to better characterize test properties.

Third, the measures of perceived workload and engagement-related responses relied on adapted self-report instruments for children. The adapted NASA-TLX showed acceptable internal consistency for the Core Workload index (α≈0.73), supporting its use as a descriptive indicator of perceived tasks' demands in this sample. By contrast, the psychometric properties of the adapted UES-SF were weak and heterogeneous. Reliability was acceptable only for the Reward subscale, while the other subscales and the overall scale showed low internal consistency, and the two-item Aesthetic Appeal index showed a negative alpha. These results substantially limit the strength of any inference about engagement as a multidimensional construct. Consequently, the UES-SF findings in this study should be interpreted only as descriptive post-activity response patterns, with Reward as the most interpretable indicator. Future research should invest in the design and validation of engagement measures tailored to primary school pupils and should triangulate self-reports with behavioral indicators, such as on-task behavior, error rates during programming, interaction logs, peer interaction patterns and, where feasible, physiological measures.

Fourth, the timing of the workload and engagement-related questionnaires represents an additional limitation. Pupils completed the NASA-TLX and adapted UES-SF after the post-test rather than immediately after the Pepper interaction. As a result, their responses may have reflected not only the robot-mediated coding activity, but also the subsequent assessment experience and their general recollection of the session. This makes it difficult to isolate pupils' perceptions of the Pepper event alone. Future studies should consider administering short measures immediately after the robot interaction, or collecting repeated micro-measures during multi-session activities, in order to distinguish perceptions of the robot-mediated task from perceptions of the assessment phase.

Fifth, the study captured only short-term responses, based on a single Pepper-based session and post-testing conducted shortly after the activity. We do not know whether the observed score increase would persist over time, nor how pupils' perceived workload and reward would evolve after repeated exposure to the robot. The novelty of interacting with Pepper, the unusual gym-based setting, and the social desirability of responding positively in a school context may all have contributed to pupils' post-activity responses. Longitudinal studies, in which robot-mediated coding is integrated into a sequence of lessons, would be valuable to examine sustained learning trajectories, delayed retention, the evolution of pupils' attitudes, and potential changes in the relationships between workload, reward, and coding performance. In this respect, the exploratory correlations reported in this study, such as the negative association between Core Workload and gain scores, should be considered as hypotheses to be tested in larger longitudinal samples rather than as definitive evidence.

Sixth, the study was conducted with the SoftBank Pepper robot, which is no longer commercially available. This limits the direct replicability of the activity with the same platform. However, the broader educational issue addressed by the study is not restricted to Pepper: it concerns how embodied and programmable artifacts can support CT in primary education. Future research should therefore examine whether similar design principles can be implemented with other currently available humanoid or social robots, such as NAO, Tega, QTrobot, or other classroom-suitable platforms. It should also compare humanoid robots with alternative and complementary tools, including floor robots, programmable boards, tangible interfaces, graphical coding environments, and game-based learning environments. Such comparisons would help clarify which aspects of the activity are related to embodiment, which are related to social presence, and which can be achieved through simpler or more scalable educational technologies.

Finally, the specific implementation reported here involved a structured, teacher-orchestrated activity in which pupils controlled Pepper individually and in turn, while the class observed the activity. This format facilitated classroom management and ensured that all pupils had direct interaction with the robot, but it may differ from more open-ended, collaborative, or autonomous scenarios that leverage social robots' capabilities in different ways. Future work could explore variations in the robot's role, such as peer, tutee, novice, or tutor, as well as different degrees of learner autonomy and collaboration. It would also be useful to examine how robot-mediated tasks can be integrated with unplugged activities, block-based programming, tangible interfaces, and classroom discussion, and how these design choices influence both coding performance and socio-emotional outcomes across diverse educational contexts.

This broader perspective is also consistent with recent work on emerging educational technologies and alternative platforms for coding and CT. Tangible and graphical experimentation with programmable boards such as Arduino and Micro:bit can support hands-on learning and may influence educators' intention to integrate such tools into classroom practice (Tselegkaridis et al., 2025). More broadly, future work should situate social robots within a wider ecosystem of coding and CT technologies, including tangible interfaces, graphical programming environments, floor robots, and other scalable classroom tools, rather than treating humanoid robots as isolated solutions.

7 Conclusion

This paper has presented an exploratory classroom-based study on a Pepper-mediated coding activity with pupils in two third-grade classes in an Italian primary school. By combining curriculum-aligned pre–post coding tests with adapted measures of perceived workload (NASA-TLX) and post-activity engagement-related responses (UES-SF), the study offers a multidimensional account of how young learners experienced a teacher-orchestrated robot-supported programming activity in an authentic school setting. The contribution of the study lies not in demonstrating the causal effectiveness of Pepper, but in documenting how coding performance, perceived task demands, and reward-related responses can be examined together in the evaluation of robot-mediated coding activities.

In relation to RQ1, the results showed a measurable increase in coding scores from pre-test to post-test, with a medium-to-large paired-sample effect size. Because the study used a single-group design without a control condition, this increase should be interpreted as a pre–post score change observed after the Pepper-based activity, rather than as conclusive evidence that Pepper caused the improvement. The fact that pupils had previously engaged in unplugged coding activities further suggests that the robot-mediated session may have supported the application and consolidation of already introduced concepts. In relation to RQ2, the responses to the post-activity workload of the students suggested that the task was perceived as manageable, with low mental and temporal demand, somewhat higher physical demand, low frustration, and high perceived performance. The Core Workload index showed acceptable internal consistency, supporting its use as a descriptive indicator of perceived task demands in this sample. However, because the questionnaire was completed after the post-test, these responses should be interpreted as post-activity perceptions rather than as an isolated measure of the Pepper interaction alone. In relation to RQ3, the adapted UES-SF provided limited descriptive evidence of positive post-activity responses, especially on the Reward subscale, suggesting that pupils tended to perceive the activity as enjoyable, interesting, and worthwhile. At the same time, the weak internal consistency of several adapted UES-SF indices prevents strong conclusions about engagement as a validated multidimensional construct in this child sample.

These findings extend previous work on educational robotics and social robots in education by providing preliminary classroom-based evidence that combines pre–post coding scores, perceived workload, and reward-related responses in a single evaluation framework. The study also suggests several design-oriented considerations for robot-mediated coding activities in primary school: building on previous unplugged experiences, using embodied tasks with clear spatial scaffolds, calibrating task difficulty to maintain manageable perceived workload, leveraging the robot's social presence without over-automation, and positioning teachers as co-orchestrators rather than as mere supervisors of a technological artifact. These considerations should be interpreted as provisional design insights rather than as general prescriptions about the effectiveness of social robots.

At the same time, the study is limited by its small sample, single-school context, lack of a control group, unequal pre- and post-test formats, short-term scope, timing for questionnaire administration, and the weak psychometric performance of several adapted engagement-related indices. The exploratory correlations observed between workload, reward, and coding performance should therefore be interpreted with caution and viewed primarily as hypotheses for future research.

Future work with larger samples, comparison conditions, and longitudinal designs will be essential to clarify whether, how, and under what conditions humanoid or tangible robots can support the development of CT and coding competencies in early education. Such research should compare robot-mediated activities with non-robotic, tangible, graphical, and game-based alternatives; use parallel and psychometrically stronger assessment instruments; and collect age-appropriate measures of workload, reward, engagement, and behavioral interaction. In this way, future studies can go beyond preliminary evidence from single classroom implementations and provide a stronger basis to understand the educational potential, limits, and design requirements of robot-mediated coding in primary school.

References

• 1

  AhmadM.MubinO.OrlandoJ. (2017). A systematic review of adaptivity in human-robot interaction. Multimodal Technol Interact. 1:14. doi: 10.3390/mti1030014

  • CrossRef
  • Google Scholar
  • View reference in article

• 2

  AndradeA. Q.BeleigoliA. M. R.SilvaT. M. S.DinizM. F. H.RibeiroA. L. P. (2019). Exploring the user engagement scale short form as a determinant of adherence in digital health interventions. Stud. Health Technol. Inform. 264, 1901–1902. doi: 10.3233/SHTI190704

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 3

  AtmatzidouS.DemetriadisS. (2016). Advancing students' computational thinking skills through educational robotics: a study on age and gender relevant differences. Rob. Auton. Syst. 75, 661–670. doi: 10.1016/j.robot.2015.10.008

  • CrossRef
  • Google Scholar
  • View reference in article

• 4

  BarrV.StephensonC. (2011). Bringing computational thinking to K-12: what is involved and what is the role of the computer science education community?ACM Inroads2, 48–54. doi: 10.1145/1929887.1929905

  • CrossRef
  • Google Scholar
  • View reference in article

• 5

  BelpaemeT.KennedyJ.RamachandranA.ScassellatiB.TanakaF. (2018). Social robots for education: a review. Sci. Robot. 3:eaat5954. doi: 10.1126/scirobotics.aat5954

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 6

  BersM. U.FlanneryL.KazakoffE. R.SullivanA. (2014). Computational thinking and tinkering: exploration of an early childhood robotics curriculum. Comput. Educ. 72, 145–157. doi: 10.1016/j.compedu.2013.10.020

  • CrossRef
  • Google Scholar
  • View reference in article

• 7

  BocconiS.ChioccarielloA.DettoriG.FerrariA.EngelhardtK. (2016). Developing Computational Thinking in Compulsory Education-Implications for Policy and Practice. Technical report, European Commission, Joint Research Centre, Luxembourg.

  • Google Scholar
  • View reference in article

• 8

  BrennanK.ResnickM. (2012). “New frameworks for studying and assessing the development of computational thinking,” in Proceedings of the AERA annual meeting (Vancouver), 1–25.

  • Google Scholar
  • View reference in article

• 9

  BuchemI. (2023). Scaling-up social learning in small groups with robot supported collaborative learning (RSCL): effects of learners' prior experience in the case study of planning poker with the robot NAO. Appl. Sci. 13:4106. doi: 10.3390/app13074106

  • CrossRef
  • Google Scholar
  • View reference in article

• 10

  ChakrabortyS. S.RahmanS. S.RahmanM. M.RahmanM. M. (2023). “NASA-TLX based workload assessment of learning tasks for primary school children,” in Proceedings of the future technologies conference (FTC) 2023, eds K. Arai volume 814 of Lecture Notes in Networks and Systems (Vol. 2, Cham: Springer), 473–486. doi: 10.1007/978-3-031-47451-4_33

  • CrossRef
  • Google Scholar
  • View reference in article

• 11

  DennerJ.WernerL.CampeS.OrtizE. (2014). Pair programming: under what conditions is it advantageous for middle school students?J. Res. Technol. Educ. 46, 277–296. doi: 10.1080/15391523.2014.888272

  • CrossRef
  • Google Scholar
  • View reference in article

• 12

  DiyasY.BrakkD.AimambetovY.SandygulovaA. (2016). “Evaluating peer versus teacher robot within educational scenario of programming learning,” in Proceedings of the 2016 11th ACM/IEEE international conference on human-robot interaction (HRI) (Christchurch), 425–426. doi: 10.1109/HRI.2016.7451788

  • CrossRef
  • Google Scholar
  • View reference in article

• 13

  Eurostat (2023). 56% of EU People Have Basic Digital Skills. Available online at: https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20231215-3 (Accessed March 13, 2026).

  • Google Scholar
  • View reference in article

• 14

  FeidakisM.GkolompiaI.MarnelakiA.MarathakiK.EmmanouilidouS.AgrianitiE. (2023). “NAO robot, an educational assistant in training, educational and therapeutic sessions,” in Proceedings of the 2023 IEEE global engineering education conference (EDUCON) (Kuwait), 1–6. doi: 10.1109/EDUCON54358.2023.10125229

  • CrossRef
  • Google Scholar
  • View reference in article

• 15

  GroverS.PeaR. (2013). Computational thinking in K-12: a review of the state of the field. Educ. Res. 42, 38–43. doi: 10.3102/0013189X12463051

  • CrossRef
  • Google Scholar
  • View reference in article

• 16

  HartS. G.StavelandL. E. (1988). “Development of NASA-TLX: results of empirical and theoretical research,” in Human mental workload (North-Holland, Amsterdam), 139–183. doi: 10.1016/S0166-4115(08)62386-9

  • CrossRef
  • Google Scholar
  • View reference in article

• 17

  HeinM.Nathan-RobertsD. (2018). “Socially interactive robots can teach young students language skills: a systematic review,” in Proceedings of the human factors and ergonomics society annual meeting (Vol. 62, Thousand Oaks, CA: SAGE), 1083–1087. doi: 10.1177/1541931218621249

  • CrossRef
  • Google Scholar
  • View reference in article

• 18

  KafaiY. B.BurkeQ. (2014). Connected Code: Why Children Need to Learn Programming. Cambridge, MA: MIT Press. doi: 10.7551/mitpress/9992.001.0001

  • CrossRef
  • Google Scholar
  • View reference in article

• 19

  LabollitaC. (2020). Pepper e attività di coding nella scuola primaria: Uno studio di caso. Bachelor's thesis, Università di Bari, Bari, Italy.

  • Google Scholar
  • View reference in article

• 20

  LanzilottiR.PiccinnoA.RossanoV.RoselliT. (2021). “Social robot to teach coding in primary school,” in Proceedings of the 2021 international conference on advanced learning technologies (ICALT) (Tartu), 102–104. doi: 10.1109/ICALT52272.2021.00038

  • CrossRef
  • Google Scholar
  • View reference in article

• 21

  Laurie-RoseC.FreyM.EnnisA.ZamaryA. (2014). Measuring perceived mental workload in children. Am. J. Psychol. 127, 107–125. doi: 10.5406/amerjpsyc.127.1.0107

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 22

  LyeS. N.KohJ. H. L. (2014). Review on teaching and learning of computational thinking through programming. Comput. Human Behav. 41, 51–61. doi: 10.1016/j.chb.2014.09.012

  • CrossRef
  • Google Scholar
  • View reference in article

• 23

  MaggiG. Dell'Aquila, E.CuccinielloI.RossiS. (2021). ‘Don't get distracted!': the role of social robots' interaction style on users' cognitive performance, acceptance, and non-compliant behavior. Int. J. Soc. Robot. 13, 2057–2069. doi: 10.1007/s12369-020-00702-4

  • CrossRef
  • Google Scholar
  • View reference in article

• 24

  MubinO.StevensC. J.ShahidS.MahmudA. A.DongJ. (2013). A review of the applicability of robots in education. Technol. Educ. Learn. 1, 1–7. doi: 10.2316/Journal.209.2013.1.209-0015

  • CrossRef
  • Google Scholar
  • View reference in article

• 25

  O'BrienA.TomsB. (2018). A practical approach to measuring user engagement with the refined user engagement scale (UES) and new ues short form. Int. J. Hum. Comput. Stud. 112, 28–39. doi: 10.1016/j.ijhcs.2018.01.004

  • CrossRef
  • Google Scholar
  • View reference in article

• 26

  O'BrienH. L.TomsE. G. (2010). The development and evaluation of a survey to measure user engagement. J. Am. Soc. Inf. Sci. Technol. 61, 50–69. doi: 10.1002/asi.21229

  • CrossRef
  • Google Scholar
  • View reference in article

• 27

  PapertS. (1980). Mindstorms: Children, Computers, and Powerful Ideas. New York, NY: Basic Books.

  • Google Scholar
  • View reference in article

• 28

  Pérez-MarínD.Hijón-NeiraR.PizarroC. (2022). Coding in early years education: which factors influence the skills of sequencing and plotting a route, and to what extent?Int. J. Early Years Educ. 30, 969–985. doi: 10.1080/09669760.2022.2037076

  • CrossRef
  • Google Scholar
  • View reference in article

• 29

  PiagetJ.InhelderB. (1973). The Psychology of the Child. New York, NY: Basic Books.

  • Google Scholar
  • View reference in article

• 30

  RagusaG.LeungL. (2023). The impact of early robotics education on students' understanding of coding, robotics design, and interest in computing careers. Sensors23:9335. doi: 10.3390/s23239335

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 31

  RamachandranA.HuangC.-M.GartlandE.ScassellatiB. (2018). “Thinking aloud with a tutoring robot to enhance learning,” in Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction (HRI '18) (New York), 59–68. doi: 10.1145/3171221.3171250

  • CrossRef
  • Google Scholar
  • View reference in article

• 32

  ResnickM.SilvermanB.KafaiY.MaloneyJ.Monroy-HernándezA.RuskN.et al. (2009). Scratch: programming for all. Commun. ACM52, 60–67. doi: 10.1145/1592761.1592779

  • CrossRef
  • Google Scholar
  • View reference in article

• 33

  SenguptaP.KinnebrewJ. S.BasuS.BiswasG.ClarkD. (2013). Integrating computational thinking with k-12 science education using agent-based computation: a theoretical framework. Educ. Inf. Technol. 18, 351–380. doi: 10.1007/s10639-012-9240-x

  • CrossRef
  • Google Scholar
  • View reference in article

• 34

  ShuteV. J.SunC.Asbell-ClarkeJ. (2017). Demystifying computational thinking. Educ. Res. Rev. 22, 142–158. doi: 10.1016/j.edurev.2017.09.003

  • CrossRef
  • Google Scholar
  • View reference in article

• 35

  StrawhackerA.BersM. U. (2015). ‘I want my robot to look for food': comparing kindergartners' programming comprehension using tangible, graphic, and hybrid user interfaces. Int. J. Technol. Design Educ. 25, 293–319. doi: 10.1007/s10798-014-9287-7

  • CrossRef
  • Google Scholar
  • View reference in article

• 36

  SullivanA.BersM. U. (2016). Girls, boys, and bots: gender differences in young children's performance on robotics and programming tasks. J. Inf. Technol. Educ15, 145–165. doi: 10.28945/3547

  • CrossRef
  • Google Scholar
  • View reference in article

• 37

  SwellerJ. (2011). Cognitive load theory. Psychol. Learn. Motiv. 55, 37–76. doi: 10.1016/B978-0-12-387691-1.00002-8

  • CrossRef
  • Google Scholar
  • View reference in article

• 38

  TanakaF.IsshikiK.TakahashiF.UekusaM.SeiR.HayashiK. (2015). “Pepper learns together with children: development of an educational application,” in Proceedings of the 2015 IEEE-RAS 15th international conference on humanoid robots (Humanoids) (Seoul: IEEE), 270–275. doi: 10.1109/HUMANOIDS.2015.7363546

  • CrossRef
  • Google Scholar
  • View reference in article

• 39

  TazhigaliyevaN.DiyasY.BrakkD.AimambetovY.SandygulovaA. (2016). “Learning with or from the robot: exploring robot roles in educational context with children,” in Social Robotics, eds A. Agah, J.-J. Cabibihan, A. M. Howard, M. A. Salichs, and H. He (Cham: Springer International Publishing), 650–659. doi: 10.1007/978-3-319-47437-3_64

  • CrossRef
  • Google Scholar
  • View reference in article

• 40

  TselegkaridisS.SapounidisT.TriantafyllouS. A. (2025). Educators' intention to use tangible and graphical experimentation with arduino and micro:bit. Discove. Educ. 4:7. doi: 10.1007/s44217-025-00398-6

  • CrossRef
  • Google Scholar
  • View reference in article

• 41

  VernerI. M.PolishukA.KraynerN. (2016). Science class with robothespian: using a robot teacher to make science fun and engage students IEEE Robot.Autom. Magaz. 23, 74–80. doi: 10.1109/MRA.2016.2515018

  • CrossRef
  • Google Scholar
  • View reference in article

• 42

  VuorikariR.PunieY.Carretero GomezS.Van den BrandeG. (2016). DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: The Conceptual Reference Model. Luxembourg: Publications Office of the European Union.

  • Google Scholar
  • View reference in article

• 43

  WernerL.DennerJ.CampeS.KawamotoD. C. (2012). “The fairy performance assessment: measuring computational thinking in middle school,” in Proceedings of the 43rd ACM technical symposium on computer science education (SIGCSE '12) (New York), 215–220. doi: 10.1145/2157136.2157200

  • CrossRef
  • Google Scholar
  • View reference in article

• 44

  WingJ. M. (2006). Computational thinking. Commun. ACM49, 33–35. doi: 10.1145/1118178.1118215

  • CrossRef
  • Google Scholar
  • View reference in article

• 45

  YadavA.HongH.StephensonC. (2016). Computational thinking for all: pedagogical approaches to embedding 21st century problem solving in K-12 classrooms. TechTrends60, 565–568. doi: 10.1007/s11528-016-0087-7

  • CrossRef
  • Google Scholar
  • View reference in article