Exploring gender as a determinant of game element effectiveness

Abstract

Student diversity, for example, in terms of gender, characterizes higher education learning settings and presents a challenge for optimally supporting all learners. Digital learning settings are increasingly employed to provide a personalized learning experience. To enhance motivation, game elements, such as points, badges, leaderboards, avatars, and feedback, are commonly used in these settings. However, their impact on performance and motivation appears to vary across genders, and empirical evidence on the effects of individual game elements remains limited. This study systematically examines gender differences in performance, motivation, self-efficacy, and anxiety in response to specific game elements. Therefore, participants solved logical problems in a digital learning setting featuring different combinations of game elements. Results reveal a significant gender difference in the influence of game elements on experienced external regulation, a dimension of motivation. Related research suggests that cultural factors and the learning context might shape gender differences, underscoring the importance of context-sensitive gamification design in digital learning settings.

1 Introduction

Higher education is characterized by students' diversity, including variation in gender. This heterogeneity poses challenges for educators in addressing individual learners' needs. Therefore, digital learning settings (DLSs) are used to support educators in individualizing and adapting learning content based on learners' characteristics and preferences (Chen et al., 2016; Hsu et al., 2012; Zhang and Zhang, 2014). Characteristically, DLSs often incorporate game elements, such as points, to improve motivation (Hamari et al., 2014; Saleem et al., 2021; Zainuddin et al., 2020). The implementation of such game-inspired elements in contexts not associated with games is referred to as gamification (Deterding et al., 2011).

Current research indicates that gamification in education can enhance learners' performance, motivation, and enjoyment (Alsawaier, 2018; Bai et al., 2020; Huang et al., 2020; Li et al., 2023; Sailer and Homner, 2020; Zhang and Yu, 2022; Zeng et al., 2024). The beneficial effects of gamification are explained by various theories, which identify mechanisms such as goal setting, feedback, reinforcement, rewards, social comparison, and social support as factors contributing to its effects (Krath et al., 2021). Among them, the most popular theory is the Self-Determination Theory (SDT; Ryan and Deci, 2000), which indicates that game elements enhance motivation by satisfying one or more of the three fundamental human needs for competence, autonomy, and relatedness (Krath et al., 2021). However, gamification may also have no or even negative effects, such as increased anxiety, task demands, and stress (Almeida et al., 2023; Bai et al., 2020; Dichev and Dicheva, 2017; Hanus and Fox, 2015; Sailer and Homner, 2020; Toda et al., 2018).

To effectively support learners and prevent negative effects, it is important to understand the mechanisms underlying these consequences. Yet, existing gamification research faces several limitations that make it challenging to draw clear conclusions about those mechanisms. Exemplary challenges include differences in definitions (Hallifax et al., 2023), sets of game elements (e.g., Toda et al., 2019a; Tondello et al., 2017), educational levels (e.g., Dichev and Dicheva, 2017), and subjects (e.g., Dichev and Dicheva, 2017; Dicheva et al., 2020). To achieve a clearer understanding of divergent impacts, individual game elements must be investigated separately within the same participant samples (Dichev and Dicheva, 2017; Mekler et al., 2017; Seaborn and Fels, 2015).

Thereby, points, badges, and leaderboards are the most commonly used game elements in education, making them key to understanding the impact of gamification on learning (Hamari et al., 2014; Saleem et al., 2021; Zainuddin et al., 2020). While points are awarded for achievements (Codish and Ravid, 2017), badges are visual indicators of achieved goals, learned skills, or acquired knowledge (Fanfarelli and McDaniel, 2019; Gibson et al., 2015). Leaderboards show learners' rankings based on their scores (Denden et al., 2021). Avatars are visual representations of the learner in DLSs (Hallifax et al., 2023) and enhance social relatedness (Sailer et al., 2017). The game element feedback provides information about learners' performance (Denden et al., 2021) and offers social support (Hernandez et al., 2021).

Not only do game elements influence performance and motivation, but recent reviews on gamification also show that learner characteristics such as gender, culture, age, user type, or personality constitute important moderators (Bai et al., 2020; Klock et al., 2020; Oliveira et al., 2023). This highlights the need to design learning systems that effectively support diverse learners' needs and preferences. Gender is an especially relevant variable for DLS adaptation in higher education contexts, as student populations are typically gender-diverse and gender can be assessed with a single self-report item, requiring minimal extra effort from learners. Therefore, understanding whether and how gender influences the impact of game elements on motivational and performance-related outcomes is crucial.

In addition to performance and motivation, self-efficacy and anxiety also play a critical role in educational contexts. Motivational and performance-related aspects control whether and how effectively someone learns. Furthermore, self-efficacy influences motivation and performance (Zimmerman, 2000), while anxiety can hinder learning (Cassady, 2004; Soltanlou et al., 2019). Taking a look into related research, existing evidence indeed suggests that the effects of gamification on motivation and performance differ between genders (Denden et al., 2017; Mellado et al., 2024; Rodrigues et al., 2022).

Regarding performance, studies report either an increase (Rodrigues et al., 2022) or decrease (Mellado et al., 2024) only for female participants, performance increase only for male participants (Pedro et al., 2015), or performance gains for both genders (Zahedi et al., 2021) due to combinations of game elements. Regarding motivation, sometimes subsumed under the broader concept of user experience, male participants report being more motivated by gamification than female participants (Mellado et al., 2024; Pedro et al., 2015). Male participants also report greater enjoyment and perceived usefulness of gamification compared to female participants (Mellado et al., 2024; Pedro et al., 2015). Similarly, female participants mostly did not enjoy the gamification (Zahedi et al., 2021). Because these studies employed different combinations of game elements, it is impossible to draw conclusions about the effects of individual elements. However, there is some evidence on gender differences for the individual game elements (see Table 1 for a summary).

For points, no consistent differences between genders regarding enjoyment, perceived usefulness and relevance, or intention to use points were found (Codish and Ravid, 2017; Denden et al., 2017, 2021; Toda et al., 2019b).

Badges were disliked by male learners, and working with badges negatively influenced their perceived need-satisfying (Schürmann and Quaiser-Pohl, 2022). For female learners, no comparable patterns were found. Even more, in two of three semesters, female learners enjoyed badges more than male learners (Codish and Ravid, 2017). Similarly, Denden et al. (2017) conclude that female learners have a higher perception of badges than male learners. In contrast, studies indicated no differences in enjoyment, usefulness of, and intention to use badges (Denden et al., 2017, 2021). Similarly, visualizing the progress, for example, with badges, was perceived as equally relevant for different genders (Toda et al., 2019b).

The presentation of leaderboards increased the performance of male but not of female learners (Lovász et al., 2023). Similarly, competition, including a leaderboard, was perceived as more relevant by male learners than by female learners (Toda et al., 2019b). Moreover, no differences between genders were found in enjoyment, perceived usefulness, and the intention to use a leaderboard (Denden et al., 2017, 2021). Codish and Ravid (2017) found varying patterns of perceived playfulness across three student cohorts, with female learners scoring higher in one, male learners in another, and no gender differences in the third cohort.

For avatars, no differences between genders in preferences for avatars compared to other game elements were found (Denden et al., 2017).

Feedback was perceived as more useful by female learners compared to male learners, but for enjoyment, no difference between genders was found (Denden et al., 2021). Similarly, encouragement increased performance for female learners and decreased it for male learners (Lovász et al., 2022). In contrast, Denden et al. (2017) found no gender differences in the preference for feedback compared to other game elements.

These results may, among other explanations, reflect gender differences in motivational aspects. For instance, female learners were found to be generally less attracted to competitive game elements than male learners (Buckley et al., 2017; Hartmann and Klimmt, 2006; Toda et al., 2019b), whereas male learners tend to associate competition with more positive outcomes (Kesebir et al., 2019). Leaderboards, which introduce competition (Michinov and Michinov, 2025), may thus particularly benefit male learners. Male learners also tended to be more performance-oriented, whereas female learners showed a stronger mastery orientation (D'Lima et al., 2014). Similarly, male learners' flow experiences was shown to be largely driven by competence, whereas female learners' flow was more strongly linked to positive emotions and playfulness (Rodríguez-Ardura and Meseguer-Artola, 2021). This orientation toward mastery and enjoyment offers a potential explanation for female learners' preference for feedback as it supports self-improvement processes, and for badges as they function as visible indicators of skill mastery and learning progress (Abramovich et al., 2013). Importantly, these gender differences may reflect socially constructed norms rather than inherent traits and other individual, contextual, or cultural factors may also contribute to the observed patterns.

Taken together, existing research indicates that female and male learners differ in the perception of gamification in general (Denden et al., 2017; Mellado et al., 2024; Rodrigues et al., 2022), badges (Codish and Ravid, 2017; Denden et al., 2021; Schürmann and Quaiser-Pohl, 2022), leaderboards (Lovász et al., 2023; Toda et al., 2019b), and feedback (Denden et al., 2021; Lovász et al., 2022). While few studies systematically examined how specific game elements impact learning (Codish and Ravid, 2017; Denden et al., 2017, 2021; Toda et al., 2019b), the results are inconsistent. Furthermore, most of those studies focused on aspects of user experience, such as perceived usefulness and enjoyment of individual elements, thereby lacking a more concise inspection of motivational and/or performance-related outcome variables. Most limiting, we could not find any research investigating the influences of individual game elements on self-efficacy or anxiety. Consequently, the limited body of research highlights the need for a more systematic investigation of the differential impacts of individual game elements on learners identifying with different genders.

To address this research gap, the current experiment systematically examines whether gender moderates the effects of game elements (i.e., points, badges, leaderboards, avatars, and feedback) on performance, motivation, self-efficacy, and anxiety. Due to the exploratory nature of the present investigation, no specific hypotheses were formulated. Points, badges, and leaderboards were selected because they represent the most commonly used game elements in education (Hallifax et al., 2023; Hamari et al., 2014; Saleem et al., 2021; Zainuddin et al., 2020). Avatars were included for their potential to enhance social relatedness (Sailer et al., 2017), while feedback was considered for its role in fostering social support (Hernandez et al., 2021). Performance and motivation control whether and how effectively someone learns, while self-efficacy influences performance and motivation (Zimmerman, 2000), and anxiety can hinder learning (Cassady, 2004; Soltanlou et al., 2019). The current experiment employed a controlled design to minimize confounding factors and isolate the effects of gamification. To this end, participants engaged in a simple logical reasoning task, which reduced variability arising from task complexity. Game elements were systematically varied, while distracting elements were minimized. This setup ensured that any observed differences in outcomes could be attributed primarily to the presence of the game elements. Additionally, rather than asking participants directly about their perceptions of the game elements without having them experience these elements, they interacted with different game elements and subsequently indicated their motivation, self-efficacy, and anxiety through questionnaires based on their experience. This approach reduced social desirability bias (Edwards, 1957).

2 Methods

2.1 Participants

The data sets from 117 students (M = 21.91 years, SD = 2.38 years, range: 18–27 years), including 83 (70.94%) men and 34 (29.06%) women, collected at the University of Stuttgart, were included in the data analysis. Participants were recruited through on-campus direct address and were mostly enrolled in computer science (n = 68), engineering (n = 37), or other technical and non-technical programs (n = 12). Each participant gave informed consent prior to participation and received €15 as compensation for spending 30 to 60 minutes on the experiment.

2.2 Design

The randomized controlled experiment employed a 3 × 8 factorial within-between-subjects design, examining the effects of participants' gender and game elements on performance, motivation, self-efficacy, and anxiety in a digital setting. Gender was assessed categorically, with participants identifying as male, female, or other. To examine the effects of individual game elements systematically, participants were randomly assigned to three of eight sets of game elements, including no gamification (None), Points (P), Badges (B), Leaderboards (L), Avatars (A), Feedback (F), Points-Badges-Leaderboards-Avatars (PBLA), and Points-Badges-Leaderboards-Avatars-Feedback (PBLAF).

2.3 Materials

To ensure comparability with existing findings, this experiment was based on the digital setting and tasks from Albuquerque et al. (2017), with only minor adaptations. It represents a possible design for task or quiz components in tutoring systems or digital learning setting. Figure 1 illustrates an exemplary configuration of the setting using the PBLA set and an example logical problem. Further examples of logical problems are available in the Supplementary material. Logical problem-solving tasks were chosen for their relevance across a wide range of exercises and for requiring no prior knowledge. The implementation of the game elements represents one possible design and is expected to engage learners solely through visual presentation. Participants solved three blocks of 20 logical problems. The problems from Albuquerque et al. (2017) were altered for rounds two and three. Alterations included changes in the position or rotation of matrix elements, as well as adjustments to the five answer options.

Figure 1

The game elements were implemented as follows. One point was awarded for each correctly solved problem. Badges were awarded for 1, 5, 10, and 18 correctly solved problems as a visual recognition of achievement. The leaderboard displayed fictitious learners with fixed scores and the learner's ranking. In the leaderboard condition, the avatars on the leaderboard were hidden. In our experiment, each learner always received feedback regarding the correctness of their responses directly after providing an answer (knowledge of results feedback; Attali and van der Kleij, 2017). In the feedback condition, a speech bubble containing a randomly selected piece of praise or encouragement was also presented at the bottom right. For example, after incorrect answers, the feedback might have said, “You're learning, and that's what matters!” while after correct answers, it could have said, “Great job! I knew you could do it.” Hence, this feedback represented supportive feedback from other persons. In conditions with only one game element, the logic problems and the respective game element were presented while all other game elements were hidden. In PBLA and PBLAF conditions, avatars were presented on the leaderboard. In the PBLAF condition, the feedback message was assigned to an avatar. The avatars were generated using ChatGPT (OpenAI, 2023) to represent individuals of different genders, cultures, and ages.

Motivation was assessed using the 16-item Situational Motivation Scale (SIMS, Guay et al., 2020). It uses a 7-point Likert scale from “does not correspond at all” to “corresponds exactly”. The SIMS measures four subscales: Intrinsic motivation describes the extent to which a participant engages in an activity for the pleasure and satisfaction of the task itself. Identified regulation describes how much a behavior is interpreted as chosen and valued by oneself. In contrast, external regulation is experienced when behavior is expected to be rewarded or to avoid negative consequences. Amotivation describes a lack of contingencies between behavior and outcomes (Guay et al., 2020). In previous research, the subscales all showed high reliability, (intrinsic motivation: Cronbach's α =.93; identified regulation: Cronbach's α =.81; external regulation: Cronbach's α =.75; amotivation: Cronbach's α =.78; Guay et al., 2020). The 8-item New General Self-Efficacy Scale (NGSE), with reliability of Cronbach's α>.85 in previous studies, captured self-efficacy on a 5-point Likert scale reaching from “strongly disagree” to “strongly agree” (Chen et al., 2001). State anxiety was measured using the 6-item shortened form (Court et al., 2010) of the State-Trait Anxiety Inventory (STAI-6, Marteau and Bekker, 1992). Items are rated on a 4-point Likert scale from “not at all” to “very much so.” The STAI has shown a reliability of Cronbach's α =.82 in previous research (Marteau and Bekker, 1992).

2.4 Procedure

In the experiment, participants first provided informed consent and indicated their gender, age, and the study program to which they enrolled to. Subsequently, they began the main part of the experiment, consisting of three blocks, each featuring a different set of game elements. If the condition included an avatar, participants began the block by selecting one from 15 available avatars. They completed the indicated set of 20 logical problems in each block, followed by assessments of STAI, NGSE, and SIMS. In conditions involving feedback, participants received an appropriate message after every three problems. Afterward, participants were debriefed and received compensation for their participation.

2.5 Scoring

Performance was measured by the percentage of correct responses to logical problems. Motivation was assessed via average SIMS subscale scores (Chen et al., 2001). Our sample showed high reliabilities for intrinsic motivation (α =.89), external regulation (α =.88), and amotivation (α =.81), but a lower reliability for identified regulation (α =.60). Anxiety scores were derived from STAI item responses. These were weighted following the procedure described by Court et al. (2010) to create a continuous scale. Mean scores were then calculated for each participant, with higher values indicating higher levels of anxiety. We observed a medium scale reliability in our sample (α =.70). Finally, self-efficacy scores were computed as averages of NGSE items (Chen et al., 2001), with high reliability in our sample (α =.83).

3 Results

For data analysis, data sets from two participants who identified as “other” were excluded due to insufficient sample size for meaningful statistical inference, resulting in the analysis of gender across two rather than three conditions. All remaining participants performed beyond chance level (20% with five response alternatives), so no further exclusions were necessary. This resulted in 117 participants, with 10 to 18 female participants and 25 to 37 male participants per condition. The descriptive results are shown in Table 2. The data, the analysis scripts, and details on the distribution of participants across conditions are provided in the Supplementary material.

Despite model assumptions being violated (e.g., non-Gaussian distribution of residuals), we employed linear mixed-effects models (LMMs) for all dependent variables (i.e., performance, motivation, self-efficacy, and anxiety), given their demonstrated robustness to such violations (Schielzeth et al., 2020). The LMMs specify gender, game element, their interaction, and block as fixed effects, while participant was included as a random effect. Block was integrated to account for position effects.

We investigated the interaction effects of gender and game elements using F-tests based on restricted Maximum Likelihood estimation of the parameters (Kuznetsova et al., 2017) and type II sums of squares (Langsrud, 2003; Smith and Cribbie, 2014). Post-hoc t-tests were calculated based on estimated marginal means using the emmeans R-package (Denden et al., 2017). We corrected the degrees of freedom for LMMs using Kenward-Roger correction (Kenward and Roger, 1997) as suggested by Luke (2017) and controlled the false discovery rate using the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995) for post-hoc analyses.

As we focus on gender differences in the effects of game elements on outcome variables, we subsequently focus on the interaction between gender and game elements. Detailed results are provided in the Supplementary material. We found a significant interaction between gender and game element influencing external regulation (F(7, 225.35) = 2.24, p = .032, ). Post-hoc analyses comparing genders for each game element showed more external regulation perceived by male participants compared to female participants in the PBLAF condition (β = –1.03, SE = 0.33, t(231) = –3.16, p = .014, 95%-CI = [–1.67, –0.39]). Although some beta weights were substantial, no other comparisons reached significance (none: b = –0.14, SE = 0.31, t(210) = –0.45, p = .654, 95%-CI = [–0.76, 0.48], points: b = –0.32, SE = 0.31, t(211) = –1.02, p = .654, 95%-CI = [–0.94, 0.30], badges: b = 0.20, SE = 0.33, t(234) = –0.61, p = .654, 95%-CI = [–0.84, 0.45], leaderboard: b = –0.29, SE = 0.33, t(236) = -0.88, p = .654, 95%-CI = [–0.94, 0.36], avatar: b = –0.17, SE = 0.33, t(230) = –0.52, p = .654, 95%-CI = [–0.81, 0.47], feedback: b = –0.51, SE = 0.32, t(216) = –1.61, p = .440, 95%-CI = [–1.13, 0.12], PBLA: b = –0.19, SE = 0.33, t(201) = –0.62, p = .654, 95%-CI = [–0.80, 0.42]). Further, the external regulation of female participants exposed to PBLAF was lower compared to none (b = 0.73, SE = 0.23, t(225) = 3.24, p = .003, 95%-CI = [0.29, 1.18]), points (b = 0.82, SE = 0.24, t(229) = 3.39, p = .003, 95%-CI = [0.34, 1.30]), badges (b = 0.74, SE = 0.26, t(229) = 2.82, p = .007, 95%-CI = [0.22, 1.26]), leaderboard (b = 0.53, SE = 0.25, t(226) = 2.10, p = .043, 95%-CI = [0.032, 1.02]), avatar (b = 0.81, SE = 0.25, t(227) = 3.21, p = .003, 95%-CI = [0.31, 1.30]) and PBLA (b = 0.82, SE = 0.21, t(223) = 3.91, p = .001, 95%-CI = [0.40, 1.24]). No significant difference was found between feedback and PBLAF (b = 0.45, SE = 0.23, t(225) = 1.93, p = .055, 95%-CI = [–0.01, 0.91). For male participants, no differences were found between the PBLAF and other game element sets (|b| ≤ 0.22; p ≤.821; see Supplementary material for complete results). Figure 2 shows estimated marginal means and standard errors for each game element separated by gender.

Figure 2

Interactions between gender and game elements regarding performance (F(7, 237.96) = 1.65, p = .122, ), intrinsic motivation (F(7, 227.55) = 0.74, p = .636, ), identified regulation (F(7, 231.61) = 0.21, p = .982, ), amotivation (F(7, 231.44) = 1.66, p = .121, ), self-efficacy (F(7, 227.24) = 1.39 p = .209, ), and anxiety (F(7, 248.72) = 1.28, p = .259, ) did not reach significance. Taken together, the present findings suggest that game elements may influence external regulation differently across genders. No significant effects were detected for the remaining dependent variables.

4 Discussion

Gamification is frequently used in DLS to motivate learners (Saleem et al., 2021; Zainuddin et al., 2020), yet its effect on performance and motivation varies across genders (Denden et al., 2017; Mellado et al., 2024; Rodrigues et al., 2022). Building on this, the current experiment examined how specific game elements (i.e., points, badges, leaderboards, avatars, feedback, and selected combinations of these elements) influence performance, motivation, self-efficacy, and anxiety of female and male students. Therefore, participants solved logical problems while different sets of game elements were presented. We found a significant interaction between genders and game elements on external regulation as a motivational dimension. Interaction effects of gender and game element on performance, anxiety, self-efficacy, intrinsic motivation, intrinsic regulation, and amotivation did not reach significance.

Post-hoc analyses show lower external regulation among female participants compared to male participants in the PBLAF condition. Similarly, for female participants, external regulation was lower when presenting this set of game elements than when presenting other sets, except for feedback. For male participants, no difference was found. Feedback might satisfy the need for social relatedness, as described by SDT. When feedback is combined with an avatar, it may strengthen the sense of relatedness. Although mixed findings regarding gender differences were reported (Chen et al., 2015; Martela and Ryan, 2015; Mouratidis et al., 2015), our interpretation aligns with those suggesting that female participants may be more sensitive to relatedness than male participants (Chen et al., 2015). The increased feelings of relatedness when feedback was presented, combined with avatars, may, in turn, have diminished the perceived extrinsic reward.

The non-significant results align with previous studies showing no gender differences regarding perception, enjoyment, relevance, and usefulness of points, badges, leaderboards, avatars, and feedback (Codish and Ravid, 2017; Denden et al., 2017, 2021; Toda et al., 2019b). At the same time, they contradict studies observing gender differences in performance, perceived motivation, or usability for badges, leaderboard, or feedback (e.g., Lovász et al., 2022; Lovász et al., 2023; Toda et al., 2019b).

However, there are important differences between previous studies and the current one. Several earlier studies examined courses including a combination of different game elements and used retrospective questionnaires to assess perceptions of each game element (Codish and Ravid, 2017; Denden et al., 2017, 2021). Similarly, Toda et al. (2019b) asked participants in a questionnaire how relevant they considered different game elements, without presenting them visually. In contrast, Schürmann and Quaiser-Pohl (2022) used a controlled setting and compared the need satisfaction and frustration in a seminar with and without badges. Lovász et al. (2022) investigated the effect of adding leaderboards and feedback individually and in combination. Similarly, we also conducted a controlled experiment by presenting no game element, each game element individually and combinations of game elements. Such a controlled setting allowed participants to experience game elements independently. Unlike the other controlled experiments, we observed gender differences only for feedback, but no differences for badges or leaderboards.

The contrasting results regarding gender differences in the perception of game elements could also be due to differences in the samples. For example, previous studies included participants from various regions, including Germany (Schürmann and Quaiser-Pohl, 2022), Europe (Lovász et al., 2023; Lovász et al., 2022), Israel (Codish and Ravid, 2017), and Tunisia (Denden et al., 2017, 2021). While studies from Europe reported gender differences, those from Israel and Tunisia did not. This could be explained by the use of experimental manipulations in studies from Europe. In contrast, studies outside Europe used retrospective questionnaires. Additionally, the differences could arise from cultural factors, as previous research indicates that culture plays a role in the effect of gamification (Klock et al., 2020; Silva Barbosa et al., 2023), and this is also empirically confirmed (Bai et al., 2020; Toda et al., 2022). As we did not collect information on participants' cultural backgrounds, we cannot analyze potential cultural effects in this sample. Hence, future research should also investigate cultural influence on gender differences in the impact of game elements on learning outcomes.

Additionally, two studies included participants with broader educational backgrounds (Lovász et al., 2023; Lovász et al., 2022), whereas Schürmann and Quaiser-Pohl (2022) and the current experiment investigated only students. Schürmann and Quaiser-Pohl (2022) examined education students, whereas our sample consisted primarily of computer science and engineering students. As Schürmann and Quaiser-Pohl (2022) found evidence of gender differences in badges, this could indicate the influence of study areas on these differences. Students in STEM fields (science, technology, engineering, and mathematics) have been shown to exhibit distinct interest profiles, including lower social orientation compared to students in most other disciplines (Ertl and Hartmann, 2019). Consequently, perceptions of and responses to game elements may differ between STEM students and students from other academic fields. In summary, contextual factors such as culture and educational background may moderate gender differences in the perception of game elements, which could explain the heterogeneous findings across studies. Future research should systematically investigate these effects.

Statistical power to detect gender-related interaction effects was limited by the gender imbalance in our sample (71% male participants), resulting in only ten female participants in some conditions. However, sensitivity analyses support the robustness of our results (see Supplementary material). Future research should replicate these findings with larger and more gender-balanced samples across diverse academic fields.

Even more, our experiment relies on self-reports to assess motivation, self-efficacy, and anxiety. Although such instruments are often used in psychological research, their results may be influenced by response biases, like social desirability (Edwards, 1957). Future work could strengthen robustness by adding behavioral or physiological measures. Still, our instruments were adequate to achieve the aim of this experiment, namely, conducting a controlled experiment providing initial insights into gender differences while minimizing influences beyond gender.

Additionally, we implemented only one version of each game element, though different versions may yield different outcomes. For example, leaderboards were presented with all other fictive learners (absolute leaderboard) instead of only those immediately above and below the participant (relative leaderboard) (Ćwil, 2020). Future studies should compare alternative implementations of the same elements to identify the aspects responsible for the effects of gamification. Similarly, our experiment examined the short-term effects of game elements. As the effects of gamification on motivation change over time (Dichev and Dicheva, 2017; Sailer and Homner, 2020; Van Roy and Zaman, 2018), future studies should investigate the long-term evolution of gender differences in the impact of game elements on motivation.

In conclusion, our experiment suggests that the impact of game elements on external regulation differs significantly between genders. Although previous research reports mixed findings, closer examination reveals that these discrepancies may arise from variations in cultural and contextual factors. Consistent with prior studies, our discussion underscores that both the learner's cultural background and the study program might play a role in shaping whether gender differences emerge. Future research should examine how game elements interact with gender, culture, and context to influence learning outcomes. Moreover, the design of gamified DLS should account for these factors to enhance the support for learning for all users.

References

• 1

  AbramovichS.SchunnC.HigashiR. M. (2013). Are badges useful in education? It depends upon the type of badge and expertise of learner. Educ. Technol. Res. Dev. 61, 217–232. doi: 10.1007/s11423-013-9289-2

  • CrossRef
  • Google Scholar

• 2

  AlbuquerqueJ.BittencourtI. I.CoelhoJ. A.SilvaA. P. (2017). Does gender stereotype threat in gamified educational environments cause anxiety? An experimental study. Comput. Educ. 115, 161–170. doi: 10.1016/j.compedu.2017.08.005

  • CrossRef
  • Google Scholar

• 3

  AlmeidaC.KalinowskiM.UchǒaA.FeijóB. (2023). Negative effects of gamification in education software: systematic mapping and practitioner perceptions. Inf. Softw. Technol. 156:107142. doi: 10.1016/j.infsof.2022.107142

  • CrossRef
  • Google Scholar

• 4

  AlsawaierR. S. (2018). The effect of gamification on motivation and engagement. Int. J. Inf. Learn. Technol. 35, 56–79. doi: 10.1108/IJILT-02-2017-0009

  • CrossRef
  • Google Scholar

• 5

  AttaliY.van der KleijF. (2017). Effects of feedback elaboration and feedback timing during computer-based practice in mathematics problem solving. Comput. Educ. 110, 154–169. doi: 10.1016/j.compedu.2017.03.012

  • CrossRef
  • Google Scholar

• 6

  BaiS.HewK. F.HuangB. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educ. Res. Rev. 30:100322. doi: 10.1016/j.edurev.2020.100322

  • CrossRef
  • Google Scholar

• 7

  BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B57, 289–300. doi: 10.1111/j.2517-6161.1995.tb02031.x

  • CrossRef
  • Google Scholar

• 8

  BuckleyP.DoyleE.DoyleS. (2017). Game on! Students' perceptions of gamified learning. J. Educ. Technol. Soc. 20, 1–10. Available online at: https://www.jstor.org/stable/26196115 (Accessed May 9, 2025).

  • Google Scholar

• 9

  CassadyJ. C. (2004). The impact of cognitive test anxiety on text comprehension and recall in the absence of external evaluative pressure. Appl. Cogn. Psychol. 18, 311–325. doi: 10.1002/acp.968

  • CrossRef
  • Google Scholar

• 10

  ChenB.VansteenkisteM.BeyersW.BooneL.DeciE. L.Van Der Kaap-DeederJ.et al. (2015). Basic psychological need satisfaction, need frustration, and need strength across four cultures. Motiv. Emot. 39, 216–236. doi: 10.1007/s11031-014-9450-1

  • CrossRef
  • Google Scholar

• 11

  ChenG.GullyS. M.EdenD. (2001). Validation of a new general self-efficacy scale. Organ. Res. Methods4, 62–83. doi: 10.1177/109442810141004

  • CrossRef
  • Google Scholar

• 12

  ChenS. Y.HuangP.-R.ShihY.-C.ChangL.-P. (2016). Investigation of multiple human factors in personalized learning. Inter. Learn. Environ. 24, 119–141. doi: 10.1080/10494820.2013.825809

  • CrossRef
  • Google Scholar

• 13

  CodishD.RavidG. (2017). “Gender moderation in gamification: does one size fit all?” in Proceedings of the 50th Hawaii International Conference on System Sciences (HICSS), 2006–2015. doi: 10.24251/HICSS.2017.244

  • CrossRef
  • Google Scholar

• 14

  CourtH.GreenlandK.MargrainT. H. (2010). Measuring patient anxiety in primary care: rasch analysis of the 6-item Spielberger state anxiety scale. Value Health13, 813–819. doi: 10.1111/j.1524-4733.2010.00758.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15

  ĆwilM. (2020). “Leaderboards – A motivational tool in the process of business education,” in Serious Games JCSG 2020. Lecture Notes in Computer Science, vol 12434, eds MaM.FletcherB.GöbelS.Baalsrud HaugeJ. and MarshT. (Cham: Springer). doi: 10.1007/978-3-030-61814-8_15

  • CrossRef
  • Google Scholar

• 16

  DendenM.TliliA.EssalmiF.JemniM. (2017). “An investigation of the factors affecting the perception of gamification and game elements,” in Proceedings of the 6th International Conference on Information and Communication Technology and Accessibility, ICTA 2017, 1–6. doi: 10.1109/ICTA.2017.8336019

  • CrossRef
  • Google Scholar

• 17

  DendenM.TliliA.EssalmiF.JemniM.ChenN.-S.BurgosD. (2021). Effects of gender and personality differences on students' perception of game design elements in educational gamification. Int. J. Hum. Comput. Stud. 154:102674. doi: 10.1016/j.ijhcs.2021.102674

  • CrossRef
  • Google Scholar

• 18

  DeterdingS.DixonD.KhaledR.NackeL. (2011). “From game design elements to gamefulness: Defining gamification,” in Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (ACM), 9–15. doi: 10.1145/2181037.2181040

  • CrossRef
  • Google Scholar

• 19

  DichevC.DichevaD. (2017). Gamifying education: what is known, what is believed and what remains uncertain: a critical review. Int. J. Educ. Technol. Higher Educ. 14:9. doi: 10.1186/s41239-017-0042-5

  • CrossRef
  • Google Scholar

• 20

  DichevaD.CaldwellR.GuyB. (2020). “Do badges increase student engagement and motivation?,” in Proceedings of the 21st Annual Conference on Information Technology Education, SIGITE '20, 81–86. doi: 10.1145/3368308.3415393

  • CrossRef
  • Google Scholar

• 21

  D'LimaG. M.WinslerA.KitsantasA. (2014). Ethnic and gender differences in first-year college students' goal orientation, self-efficacy, and extrinsic and intrinsic motivation. J. Educ. Res. 107, 341–356. doi: 10.1080/00220671.2013.823366

  • CrossRef
  • Google Scholar

• 22

  EdwardsA. L. (1957). The Social Desirability Variable in Personality Assessment and Research. Hinsdale, IL: Dryden Press.

  • Google Scholar

• 23

  ErtlB.HartmannF. G. (2019). The interest profiles and interest congruence of male and female students in STEM and Non-STEM fields. Front. Psychol. 10:897. doi: 10.3389/fpsyg.2019.00897

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  FanfarelliJ. R.McDanielR. (2019). Designing Effective Digital Badges: Applications for Learning, 1st Edn. London: Routledge, 1st Edition edition. doi: 10.4324/9780203728550

  • CrossRef
  • Google Scholar

• 25

  GibsonD.OstashewskiN.FlintoffK.GrantS.KnightE. (2015). Digital badges in education. Educ. Inf. Technol. 20, 403–410. doi: 10.1007/s10639-013-9291-7

  • CrossRef
  • Google Scholar

• 26

  GuayF.VallerandR. J.BlanchardC. (2020). On the assessment of situational intrinsicy and extrinsic motivation: the situational motivation scale (SIMS). Motiv. Emot. 24, 175–213. doi: 10.1023/A:1005614228250

  • CrossRef
  • Google Scholar

• 27

  HallifaxS.AltmeyerM.KöllnK.RauschenbergerM.NackeL. E. (2023). “From points to progression: a scoping review of game elements in gamification research with a content analysis of 280 research papers,” in Proceedings of the ACM on Human-Computer Interaction, number CHI PLAY, 748–768. doi: 10.1145/3611048

  • CrossRef
  • Google Scholar

• 28

  HamariJ.KoivistoJ.SarsaH. (2014). “Does gamification work?-A literature review of empirical studies on gamification,” in 2014 47th Hawaii International Conference on System Sciences, 3025–3034. doi: 10.1109/HICSS.2014.377

  • CrossRef
  • Google Scholar

• 29

  HanusM. D.FoxJ. (2015). Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput. Educ. 80, 152–161. doi: 10.1016/j.compedu.2014.08.019

  • CrossRef
  • Google Scholar

• 30

  HartmannT.KlimmtC. (2006). Gender and computer games: Exploring females' dislikes. J. Comput.-Med. Commun. 11, 910–931. doi: 10.1111/j.1083-6101.2006.00301.x

  • CrossRef
  • Google Scholar

• 31

  HernandezD.JacominoG.SwamyU.DonisK.EddyS. L. (2021). Measuring supports from learning assistants that promote engagement in active learning: evaluating a novel social support instrument. Int. J. STEM Educ. 8, 1–17. doi: 10.1186/s40594-021-00286-z

  • CrossRef
  • Google Scholar

• 32

  HsuC.-K.HwangG.-J.ChangC.-K. (2012). A personalized recommendation-based mobile learning approach to improving the reading performance of EFL students. Comput. Educ. 63, 327–336. doi: 10.1016/j.compedu.2012.12.004

  • CrossRef
  • Google Scholar

• 33

  HuangR.RitzhauptA. D.SommerM.ZhuJ.StephenA.ValleN.et al. (2020). The impact of gamification in educational settings on student learning outcomes: a meta-analysis. Educ. Technol. Res. Dev. 68, 1875–1901. doi: 10.1007/s11423-020-09807-z

  • CrossRef
  • Google Scholar

• 34

  KenwardM. G.RogerJ. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics53, 983–997. doi: 10.2307/2533558

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35

  KesebirS.LeeS. Y.ElliotA. J.PillutlaM. M. (2019). Lay beliefs about competition: Scale development and gender differences. Motiv. Emot. 43, 719–739. doi: 10.1007/s11031-019-09779-5

  • CrossRef
  • Google Scholar

• 36

  KlockA. C. T.GaspariniI.PimentaM. S.HamariJ. (2020). Tailored gamification: a review of literature. Int. J. Hum.-Comput. Stud. 144:102495. doi: 10.1016/j.ijhcs.2020.102495

  • CrossRef
  • Google Scholar

• 37

  KrathJ.SchürmannL.Von KorfleschH. (2021). Revealing the theoretical basis of gamification: a systematic review and analysis of theory in research on gamification, serious games and game-based learning. Comput. Human Behav. 125:106963. doi: 10.1016/j.chb.2021.106963

  • CrossRef
  • Google Scholar

• 38

  KuznetsovaA.BrockhoffP. B.ChristensenR. H. B. (2017). lmerTest package: tests in linear mixed effects models. J. Stat. Softw. 82, 1–26. doi: 10.18637/jss.v082.i13

  • CrossRef
  • Google Scholar

• 39

  LangsrudØ. (2003). ANOVA for unbalanced data: use type II instead of Type III sums of squares. Stat. Comput. 13, 163–167. doi: 10.1023/A:1023260610025

  • CrossRef
  • Google Scholar

• 40

  LiM.MaS.ShiY. (2023). Examining the effectiveness of gamification as a tool promoting teaching and learning in educational settings: a meta-analysis. Front. Psychol. 14:1253549. doi: 10.3389/fpsyg.2023.1253549

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41

  LovászA.Bat-ErdeneB.Cukrowska-TorzewskaE.RigóM.Szabó-MorvaiA. (2023). Competition, subjective feedback, and gender gaps in performance. J. Behav. Exper. Econ. 102, 1–42. doi: 10.1016/j.socec.2022.101954

  • CrossRef
  • Google Scholar

• 42

  LovászA.Cukrowska-TorzewskaE.RigóM.Szabó-MorvaiÁ. (2022). Gender differences in the effect of subjective feedback in an online game. J. Behav. Exper. Econ. 98:101854. doi: 10.1016/j.socec.2022.101854

  • CrossRef
  • Google Scholar

• 43

  LukeS. G. (2017). Evaluating significance in linear mixed-effects models in R. Behav. Res. Methods49, 1494–1502. doi: 10.3758/s13428-016-0809-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44

  MarteauT. M.BekkerH. (1992). The development of a six-item short-form of the state scale of the Spielberger State—Trait Anxiety Inventory (STAI). Br. J. Clin. Psychol. 31, 301–306. doi: 10.1111/j.2044-8260.1992.tb00997.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45

  MartelaF.RyanR. M. (2015). The benefits of benevolence: basic psychological needs, beneficence, and the enhancement of Well-Being. J. Pers. 84, 750–764. doi: 10.1111/jopy.12215

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46

  MeklerE. D.BrühlmannF.TuchA. N.OpwisK. (2017). Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput. Human Behav. 71, 525–534. doi: 10.1016/j.chb.2015.08.048

  • CrossRef
  • Google Scholar

• 47

  MelladoR.CubillosC.VicariR. M.Gasca-HurtadoG. (2024). Leveraging gamification in ICT education: examining gender differences and learning outcomes in programming courses. Appl. Sci. 14:7933. doi: 10.3390/app14177933

  • CrossRef
  • Google Scholar

• 48

  MichinovN.MichinovE. (2025). More competition, less interaction: gamifying lectures using a leaderboard reduces female students' social engagement. J. Comput. High. Educ. 2025, 1–27. doi: 10.1007/s12528-025-09438-4

  • CrossRef
  • Google Scholar

• 49

  MouratidisA.BarkoukisV.TsorbatzoudisC. (2015). The relation between balanced need satisfaction and adolescents' motivation in physical education. Eur. Phys. Educ. Rev. 21, 421–431. doi: 10.1177/1356336X15577222

  • CrossRef
  • Google Scholar

• 50

  OliveiraW.HamariJ.ShiL.TodaA. M.RodriguesL.PalominoP. T.et al. (2023). Tailored gamification in education: a literature review and future agenda. Educ. Inf. Technol. 28, 373–406. doi: 10.1007/s10639-022-11122-4

  • CrossRef
  • Google Scholar

• 51

  OpenAI. (2023). ChatGPT (June 20 version) [Large language model]. Available online at: https://chat.openai.com/chat (Accessed June 3, 2024).

  • Google Scholar

• 52

  PedroL. Z.LopesA. M. Z.PratesB. G.VassilevaJ.IsotaniS. (2015). “Does gamification work for boys and girls? An exploratory study with a virtual learning environment,” in Proceedings of the 30th Annual ACM Symposium on Applied Computing, SAC '15, 214–219. doi: 10.1145/2695664.2695752

  • CrossRef
  • Google Scholar

• 53

  RodriguesL.PereiraF.TodaA.PalominoP.OliveiraW.PessoaM.et al. (2022). Are they learning or playing? Moderator conditions of gamification's success in programming classrooms. ACM Trans. Comput. Educ. 22, 1–27. doi: 10.1145/3485732

  • CrossRef
  • Google Scholar

• 54

  Rodríguez-ArduraI.Meseguer-ArtolaA. (2021). Flow experiences in personalised e-learning environments and the role of gender and academic performance. Inter. Learn. Environ. 29, 59–82. doi: 10.1080/10494820.2019.1572628

  • CrossRef
  • Google Scholar

• 55

  RyanR. M.DeciE. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78. doi: 10.1037//0003-066X.55.1.68

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  SailerM.HenseJ. U.MayrS. K.MandlH. (2017). How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. Comput. Human Behav. 69, 371–380. doi: 10.1016/j.chb.2016.12.033

  • CrossRef
  • Google Scholar

• 57

  SailerM.HomnerL. (2020). The gamification of learning: a meta-analysis. Educ. Psychol. Rev. 32, 77–112. doi: 10.1007/s10648-019-09498-w

  • CrossRef
  • Google Scholar

• 58

  SaleemA. N.NooriN. M.OzdamliF. (2021). Gamification applications in e-learning: a literature review. Technol. Knowl. Learn. 27, 139–159. doi: 10.1007/s10758-020-09487-x

  • CrossRef
  • Google Scholar

• 59

  SchielzethH.DingemanseN. J.NakagawaS.WestneatD. F.AllegueH.TeplitskyC.et al. (2020). Robustness of linear mixed-effects models to violations of distributional assumptions. Methods Ecol. Evol. 11, 1141–1152. doi: 10.1111/2041-210X.13434

  • CrossRef
  • Google Scholar

• 60

  SchürmannL.Quaiser-PohlC. (2022). Digital badges affect need satisfaction but not frustration in males in higher education. Comput. Educ. 182:104484. doi: 10.1016/j.compedu.2022.104484

  • CrossRef
  • Google Scholar

• 61

  SeabornK.FelsD. I. (2015). Gamification in theory and action: a survey. Int. J. Hum. Comput. Stud. 74, 14–31. doi: 10.1016/j.ijhcs.2014.09.006

  • CrossRef
  • Google Scholar

• 62

  Silva BarbosaJ. F.Chalco ChallcoG.BittencourtI. I. (2023). Does gender-stereotyped gamification increase negative thinking? Results from an experimental study with logic tutoring systems. Inter. Learn. Environ. 32, 5632–5659. doi: 10.1080/10494820.2023.2229145

  • CrossRef
  • Google Scholar

• 63

  SmithC. E.CribbieR. (2014). Factorial anova with unbalanced data: a fresh look at the types of sums of squares. J. Data Sci. 12, 385–404. doi: 10.6339/JDS.201407_12(3).0001

  • CrossRef
  • Google Scholar

• 64

  SoltanlouM.ArtemenkoC.DreslerT.FallgatterA. J.EhlisA.-C.NuerkH.-C. (2019). Math anxiety in combination with low visuospatial memory impairs math learning in children. Front. Psychol. 10:89. doi: 10.3389/fpsyg.2019.00089

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 65

  TodaA.KlockA.PereiraF. D.RodriguesL. A.PalominoP. T.LopesV.et al. (2022). “Towards the understanding of cultural differences in between gamification preferences: a data-driven comparison between the US and Brazil,” in Proceedings of the 15th International Conference on Educational Data Mining, 560–564.

  • Google Scholar

• 66

  TodaA. M.KlockA. C.OliveiraW.PalominoP. T.RodriguesL.ShiL.et al. (2019a). Analysing gamification elements in educational environments using an existing gamification taxonomy. Smart Learn. Environ. 6, 1–14. doi: 10.1186/s40561-019-0106-1

  • CrossRef
  • Google Scholar

• 67

  TodaA. M.OliveiraW.ShiL.BittencourtI. I.IsotaniS.CristeaA. I. (2019b). “Planning gamification strategies based on user characteristics and DM: a gender-based case study,” in Proceedings of the 12th International Conference on Educational Data Mining, EDM '19, 438–443.

  • Google Scholar

• 68

  TodaA. M.ValleP. H. D.IsotaniS. (2018). “The dark side of gamification: an overview of negative effects of gamification in education,” in Higher Education for All: From Challenges to Novel Technology-Enhanced Solutions, eds CristeaA. I.BittencourtI. I. and LimaF. (Springer International Publishing), 143–156. doi: 10.1007/978-3-319-97934-2_9

  • CrossRef
  • Google Scholar

• 69

  TondelloG. F.MoraA.NackeL. E. (2017). “Elements of gameful design emerging from user preferences,” in Proceedings of the Annual Symposium on Computer-Human Interaction in Play, CHI PLAY '17 (New York, NY, USA: Association for Computing Machinery), 129–142. doi: 10.1145/3116595.3116627

  • CrossRef
  • Google Scholar

• 70

  Van RoyR.ZamanB. (2018). Need-supporting gamification in education: an assessment of motivational effects over time. Comput. Educ. 127, 283–297. doi: 10.1016/j.compedu.2018.08.018

  • CrossRef
  • Google Scholar

• 71

  ZahediL.BattenJ.RossM.PotvinG.DamasS.ClarkeP.et al. (2021). Gamification in education: a mixed-methods study of gender on computer science students' academic performance and identity development. J. Comput. High. Educ. 33, 441–474. doi: 10.1007/s12528-021-09271-5

  • CrossRef
  • Google Scholar

• 72

  ZainuddinZ.ChuS. K. W.ShujahatM.PereraC. J. (2020). The impact of gamification on learning and instruction: a systematic review of empirical evidence. Educ. Res. Rev. 30:100326. doi: 10.1016/j.edurev.2020.100326

  • CrossRef
  • Google Scholar

• 73

  ZengJ.SunD.LooiC.FanA. C. W. (2024). Exploring the impact of gamification on students' academic performance: a comprehensive meta-analysis of studies from the year 2008 to 2023. Br. J. Educ. Technol. 55, 2478–2502. doi: 10.1111/bjet.13471

  • CrossRef
  • Google Scholar

• 74

  ZhangG.ZhangX. (2014). “Optimization of the learning process in a new digital learning environment,” in 2014 9th International Conference on Computer Science &Education, 331–334. doi: 10.1109/ICCSE.2014.6926480

  • CrossRef
  • Google Scholar

• 75

  ZhangQ.YuZ. (2022). Meta-analysis on investigating and comparing the effects on learning achievement and motivation for gamification and game-based learning. Educ. Res. Int. 2022, 1–19. doi: 10.1155/2022/1519880

  • CrossRef
  • Google Scholar

• 76

  ZimmermanB. J. (2000). Self-efficacy: an essential motive to learn. Contemp. Educ. Psychol. 25, 82–91. doi: 10.1006/ceps.1999.1016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar