Students' Achievement Emotions and Online Learning in Teacher Education

Introduction

It is becoming increasingly common in higher education institutions to offer online-learning environments. They are considered as an important teaching approach in university that can overcome limitations related to on-campus learning. For example, the internet allows educators to provide learners independent of time and place with new and innovative virtual environments-an attempt to stimulate and enhance their learning process (Brown, 2002). It is well-known that learning environments have an impact not only on cognitive but also on emotional and motivational aspects of learning. But productivity gains and benefits to students and academic institutions promised by learning technologies cannot be realized unless they are accepted and effectively used (Iivari and Ervasti, 1994; Schmid et al., 2017; Hawlitschek and Fredrich, 2018). Studies on emotions in technology-supported learning environments have increased in recent years (Loderer et al., 2018). For example, it was shown that emotional experiences in technology-enriched learning environments are different from those in traditional on-campus courses (Daniels and Stupnisky, 2012; Regan et al., 2012; Butz et al., 2015). Nevertheless, the question how students experience online-learning environments emotionally has not been answered sufficiently, so far. Especially in teacher education there are relatively limited empirical studies on students' emotions in online-learning environments (Keengwe and Kang, 2013; Loderer et al., 2018). To contribute to this research field a study is presented that analyzes the relation between students' control and value appraisals, achievement emotions, their technology acceptance and their learning outcome in the context of an on-campus and an online course in teacher education.

Achievement Emotions, Learning Processes, and Outcomes

Emotions that are directly linked to learning processes and achievement outcomes are classified as achievement emotions (Pekrun and Stephens, 2010). Achievement emotions have an effect on learning and achievement, mediated by attention, self-regulation, and motivation (Pekrun et al., 2002; Pekrun, 2006); they direct the person toward or away from learning matters in learning situations (Ellis and Ashbrook, 1988). In the traditional academic context, Pekrun et al. (2011) explored achievement emotions, showing that positive emotions can predict creative thinking and reflecting, thereby supporting academic performance, whereas negative emotions are more associated with lower levels of performance. More precisely, positive emotions such as enjoyment, hope, and pride were positively associated with student effort, self-regulation and more elaborated learning strategies, whereas anger, shame, anxiety and boredom have been associated with lower performances and more external regulation (Pekrun et al., 2011). Furthermore, positive emotions facilitate self-regulated learning (Carver and Scheier, 1990; Boekaerts et al., 2000). Students' perceived self-regulation correlates positively with positive emotions, perceived external regulation correlates with negative emotions (Pekrun et al., 2002). In terms of motivation, the experience of competence and autonomy in learning has been emphasized as important for self-regulation and self-determination (Deci and Ryan, 1985; Ryan and Deci, 2000). Gender-specific differences, are inconsistent and domain-specific, as some studies in school and higher education reported (Frenzel et al., 2007; Zembylas, 2008; Yukselturk and Bulut, 2009; Götz et al., 2012).

Pekrun (2006) classifies achievement emotions based on the control-value theory according to valence (positive vs. negative), degree of activation (activating vs. deactivating) and object focus (activity, outcome prospective and outcome retrospective). Appraisals of control and value are critical antecedents of achievement emotions. Specifically, control is relevant to refer to the perceived causal influence of an agent of achievement. Value (according to Wigfield and Eccles, 1992) describes the perceived importance of actions and outcomes associated with four dimensions of achievement: intrinsic value (personal enjoyment in a given task), attainment value (fulfillment of one's self-schema), utility value (reaching long and short-term goals) and cost (the consequences of engaging in a particular activity) (cf. Butz et al., 2015). Previous empirical work has shown a significant relation between control and value appraisals to achievement emotions. Pekrun et al. (2002) found perceived control to have significant positive correlations with positive activating emotions and significant negative correlations with negative activating emotions. Furthermore, a significant positive correlation between high task value and students' positive activating emotions was described. Students' achievement emotions were also strongly linked to their learning outcome in traditional learning environments (Pekrun et al., 2011).

Achievement Emotions and Online-Learning in Higher Education

For online-learning environments D'Errico et al. (2016) demonstrated that students' positive emotions across different online-learning activities are higher than negative emotions, particularly during synchronous activities with a teacher and with peers. They also found that experiencing positive emotions during exam preparation strongly correlates with students' motivation supporting students learning process and learning outcome.

Some studies were carried out in technology-enriched environments based on Pekrun's (2006) control-value theory. For example, students' emotions were analyzed in virtual tutoring systems (Lehman et al., 2012) and self-paced online courses (Artino and Jones, 2012). Specifically, Marchand and Gutierrez (2012) showed that students' utility value significantly predicted frustration, in both online and on-campus courses. In virtual tutoring systems, Lehman et al. (2012) showed in an experimental study that limiting students' control in a technology-enriched environment caused higher levels of students' negative emotions.

Students' achievement emotions are, to a great extent, related to domains (Goetz et al., 2007). While achievement emotions were analyzed in different content domains (like school subjects e.g., mathematic, Tulis, 2010; Götz et al., 2012; Bieg et al., 2017), research in teacher education is widely missing. For example, there are some studies on specific math anxiety of pre-service primary school teachers and students' emotions in teaching internship (Malinsky et al., 2006; Jackson, 2015; Yuan and Lee, 2016). Especially in the teacher-education-domain of “school education” there is a lack of research on students' emotions.

Furthermore, a domain represents the general frame or structure and may be defined as learning environment. It may be assumed that online learning environments are different from traditional, face-to-face instruction regarding students' emotions (control and value appraisals), motivation, and learning outcome. In contrast to on-campus courses, students explore online learning environments only individually with regard to their structure and features. In terms of control-value theory, this means that students' achievement emotions may relate not only to the content itself but rather to the digital learning environment. Butz et al. (2015) showed that there are not only significant differences between achievement emotions of students attending an on-campus course and students attending an online course, but that the control value beliefs of students attending the online course also differ with regard to their content-related and technology-related attribution.

Achievement Emotions and Technology Acceptance

In general but especially with respect to online learning environments, it may be assumed that students' acceptance and use of technology is a crucial condition of emotional experience and the quality of the learning process and outcome. For example, Daniels and Stupnisky (2012) argued that emotion research in online learning has made it “more important than ever to consider the source of the emotion in addition to the emotion itself,” asserting that students are likely to “experience emotions in response to the technology itself.” Accordingly, Regan et al. (2012) suggested that the factors affecting emotions in technology-enriched learning environments are different from those that influence emotions in traditional, on-campus environments. Therefore, domain-specificity, as well as technology acceptance and use are both important determinants for analyzing achievement emotions of university students in an online learning environment.

Numerous studies describe how technology is used in different domains. For example, Schmid et al. (2017) showed that teacher students in Germany are in comparison to students of other disciplines the most skeptical one is when it comes to the use of digital media. Moreover, teacher students are less motivated than other students to use digital media.

Research on technology acceptance tries to find factors that explain user attitudes, behavioral intention, and ultimate usage behavior. Davis (1985) postulated the expected benefits (value) and the expected user-friendliness (control) as important predictors of user acceptance in technology enriched learning environments. Technology acceptance is not only reflected regarding the frequency of using technology, but rather affective experience is closely linked to the concept of acceptance: “Acceptance includes a relatively permanent cognitive and affective perceptual component, coupled with a positive willingness to react to an e-learning system (attitude level), as well as a behavioral component that implies an actual use of the system (behavioral level)” (Olbrecht, 2010; translated from German).

The technology acceptance model (TAM) developed by Davis (1985) and Venkatesh and Davis (2000) theorizes that perceived usefulness influences attitudes and beliefs toward technology usage, and it is an important determinant of individuals' intentions to use the technology. Furthermore, Venkatesh (2000) argued that in addition to perceived usefulness the perceived ease of use is an important determinant for attitudes toward technology. Perceived control, intrinsic motivation (playfulness), and emotion (anxiety) have been tested as influencing users' perceptions about technology ease of use. The empirical results indicated that up to 60% of the variability of perceived ease of use was explained in this model (Venkatesh, 2000).

According to TAM, a student's intention to use an online learning system is determined by one's beliefs and attitude toward using the online system and the perceived usefulness of the system. Consequently, when the online learning system is perceived as easy to be used, the higher will be the student's perceived ability to use this online system successfully, and hence the student will experience more positive emotions and perform better in an online course (Venkatesh and Bala, 2008). Individual variables, such as self-efficacy, intrinsic motivation, cognitive absorption (Saadé and Bahli, 2005), and computer anxiety were all confirmed as determinants of the perceived ease of use (Gefen and Straub, 1997; Chang and Cheung, 2001; Gefen et al., 2003). External variables, such as characteristics of the learning environment affect the perceived usefulness directly or indirectly through the perceived ease of use (Compeau et al., 1999).

For example, Wong (2015) showed that teachers in Hong Kong have a positive attitude toward technology, with perceived usefulness having a greater impact on behavioral intention than perceived usability. In Germany, the TAM was used to evaluate the acceptance of the learning management system of the University of Oldenburg by students, lecturers, and administrators (Hamborg et al., 2014). It has to be considered that technology based learning environments may hinder the learning process if the technology is perceived by students being too complex and not useful to enhance their performance. Saadé and Kira (2006) showed in a study based on a structured equation modeling simulation that the influence of emotions (anxiety and pleasure) on perceived usefulness is indirectly moderated through the perceived ease of use.

Further studies focused on information systems and investigated the TAM constructs with respect to affect and anxiety (Agarwal and Karahanna, 2000; Venkatesh and Davis, 2000; Saadé and Kira, 2009). However, research is missing that applied TAM not only to anxiety and affect but also to different positive and negative achievement emotions, and to an online learning environment in teacher education.

Aim and Research Questions

As mentioned above, theoretically it may be assumed that academic performance may be affected by achievement emotions. According to the control value theory, perceived achievement emotions depend on the perception of control and values. We assume that the perception of control values can not only be related to aspects of content, but is also influenced by the learning environment (online vs. on-campus). In an online learning environment, technology acceptance is an integral part of the students' control value beliefs. However, research findings showed that student teachers in particular are skeptical about digital media. But technology acceptance is influenced on the one hand by attitudes and beliefs, on the other hand it affects achievement emotions, and can therefore foster or hinder the learning process. There is still a lack of studies that analyze achievement emotions of teacher students in the online- and on-campus teaching context, and in particular in the domain of “school education,” which is an important domain in teacher training. Therefore, this study examines (Pekrun, 2006) control-value theory of achievement emotions in the context of an online and an on-campus learning university course in the domain of school education. Furthermore, technology acceptance based on the technology acceptance model (Davis, 1989; Venkatesh and Davis, 2000) is focused. The study presented in this paper covers a retrospective comparison between students who attended an online or an on-campus course with respect to measures of technology related control-and value-appraisals, domain specific achievement task value and academic control, achievement emotions, and domain specific achievement.

The aim of this study was (1) to analyze to what extent there are differences in the experience of achievement emotions and in the perception of control and value based on individual characteristics (students' age, gender, and high school diploma). It was (2) tested whether differences can be identified with regard to the learning environment (in an online vs. an on-campus course) in terms of control- and value-appraisals, achievement emotions, and technology related beliefs. It is assumed that the online learning environment influences learners' perception of control and value differently (in comparison to an on-campus learning environment), and consequently affects learners' achievement emotions and domain specific learning outcome.

Methods

Sample

The study was carried out in the context of a university course preparing students for the state teacher examination in the domain school education at a German university. The course was administered as on-campus course until summer semester 2018 and then transferred to an online learning course in the following semester. The teacher training students had therefore no possibility to choose between the online course and the on-campus course. Rather all students participated voluntary because they were highly interested in a systematic preparation for their state teacher examination. Both courses focused on the same domain and topics (theory of education, instructional models and designing learning environments in school), and they were comparable regarding information input, performance records, and literature compendium. In addition, the online course included video-sequences from a lecture at the university, and free accessible video examples from school instruction. The online course consisted of theoretically based scripts, a variety of work sheets, and further literature. In addition, the students used a self-assessment tool. Two tutors supervised the online course and gave feedback regarding the results of self-assessment. The online platform applied in this course is well-established at the university. All slides, working material, literature, and general information were published on the online platform including a forum to pose and discuss questions.

Both courses were equal with respect to workload and educational objectives and contributed equivalently to students' preparation for the state teacher examination. The introduction and implementation of the online-course was consistent for all students. The lecturer who provided the on-campus course was involved in the development of the online-course. Therefore, it can be assumed that students perceived both environments as valuable for their learning process. Hence, endeavors have been made to foster the fidelity of implementation (O'Donnell, 2008). Data collection took place at the end of both courses based on a paper-pencil interrogation. Attendance in both courses, as well as participation in the study was voluntary at all times. All respondents were informed orally before the survey and in writing on each questionnaire about the objectives of the research project, anonymous use of collected data and the voluntariness of their participation. Informed consent was obtained from the participants, contact details for questions and objections were provided. The participants were throughout of full age and neither in need of protection (in contrast to children, sick or unstable persons), nor negative consequences for health or well-being were imperiled by the study. An ethics approval was not required as per applicable institutional and national guidelines and regulations and the informed consent of the participants was obtained by virtue of survey completion. The total sample consisted of N = 182 students with predominantly female students (82% female), enrolled in teacher education program. This is generally the case in teacher education. The mean age of the whole sample was M = 23.12 (SD = 2.30) years. As the course was a preparation course for state teacher examination, most of the students were enrolled in a higher semester (M = 6.81; SD = 2.51). In the on-campus course participated 88 students (83% female and 17% male; age from 20 to 32 years; M = 23.20; SD = 2.43). In the online course 94 students took part (82% female and 18% male between 20 and 33 years; M = 23.03; SD = 2.39). 128 students (63 students in on-campus and 65 students in online-course) completed an achievement test. The participation in the achievement test was also voluntary for all participants in both courses.

Measures

Achievement Emotions Questionnaire (AEQ)

Based on a 5-point scale (1 = not at all; 5 = very much so), students indicated the extent to which they experienced discrete emotions measured with Pekrun et al.'s (2005) Achievement Emotions Questionnaire (AEQ). The following eight emotions were measured: enjoyment, hope, pride, anger, fear, shame, hopelessness, and boredom. The emotions varied in terms of valence and activation: positive activating (enjoyment, hope, and pride), negative activating (anger, anxiety, and shame), positive deactivating (not measure) and negative deactivating (hopelessness, boredom). Participants rated their emotions separately in relation to (1) their overall experience in the on campus-, respectively, the online-course. In total, emotion scales of the AEQ comprised 80 items.

The perceived academic control scale was used to measure the content-related control (e.g., “The more effort I put into my courses, the better I do in them.”). The scale for content-related value (achievement task value) was used in orientation to Butz et al. (2015), adapted from Wigfield and Eccles' (1992) study of achievement task value (e.g., “It is important to me that I do well in the course in teacher education”).

For all achievement emotion scales good or excellent reliability using Cronbach's α coefficients (between α = 0.65 and α = 0.95) were received. However, reliability coefficients for the appraisal scales (perceived academic control, achievement task value) were weaker but still good or at least acceptable (between α = 0.63 and α = 0.73).

Technology Acceptance Model Questionnaire (TAM)

To measure technological control and technological value a part of the TAM questionnaire (Venkatesh and Bala, 2008) was applied. Venkatesh and Bala (2008) originally developed the TAM questionnaire for the use in the business sector. The TAM has been proven to be a powerful tool for examining the technology acceptance of pre- and in-service teachers (Scherer et al., 2019), therefore it was slightly adapted for the higher education context. All Items were measured on a 5 point Likert scale (1 = not at all to 5 = very much so). All items have been translated from English into German language. The reliability coefficients of technological control (e.g., “I have control over using the system.” α = 0.65) and technological value (e.g., “I find the system to be useful for the course in teacher education,” α = 0.70) were good or at least acceptable. Technological control and technological value are not the only predictors of teacher students' behavioral intentions (Scherer et al., 2019). Therefore the technology related beliefs scale (e.g., “Technology threatens people more than it benefits them,” α = 0.65) (Kaspar et al., 2002; Claßen, 2012) was applied in this study, as well.

Domain specific learning outcomes were measured by an achievement test (18 tasks, scoring with a maximum of 31 points) at the end of both courses. The achievement test developed for this study covered single choice questions and open questions with topics for the state teacher examination.

For group sizes, being nearly equal ANOVA is robust to violations of normality in terms of F-accuracy and power (Field, 2013). Shapiro-Wilk-test was significant for technology related beliefs (p = 0.01), perceived academic control (p = 0.04), achievement task value (p ≤ 0.01), achievement outcome (p ≤ 0.01), and all negative emotions (p ≤ 0.001). For the remaining scales a normal distribution was confirmed. Reliability coefficients and normality are described in Table 1.

TABLE 1

Table 1. Descriptive statistics, reliability (Cronbach's α), and normality.

Rationale for Analysis

All calculations were carried out using SPSS 25, effect sizes were computed with G^*Power 3.1. In order to compare means between online and on-campus courses, as well as gender differences, we assessed t-tests. Correlations between continuous variables were calculated by using Spearman's rank correlation coefficient. For each emotion, as well as achievement outcome, analysis of covariance (ANCOVA) was computed using the General Linear Model Procedure (GLM 2). All dependent variables were used simultaneously to test the emotional effects of the online and on-campus course. As covariates, we integrated control appraisals (perceived academic control, technological control), value appraisals (achievement task value, technological value), gender, mean grades of the high school diploma as preceding achievement, and technology related beliefs. As effect size measure, we report Cohen's d values of 0.20 as small effects, above 0.50 as medium, and values of 0.80 as huge effects, correlation coefficients r_s above 0.10 as small effects, 0.30 as moderate effects, and above 0.50 as strong effects, effect size Cohen's q for differences between Fisher-z-correlations of 0.10 as small effects, around 0.30 as moderate, and above 0.50 as strong effects, and partial Eta squared values η² of 0.01 as small effects, values above 0.059 as medium effects, and values of 0.138 or bigger as large effects (Cohen, 1988).

Results

In total, descriptive results showed that negative achievement emotions were spread on a low level amongst students while mean levels for positive achievement emotions, control and value appraisals were on a medium level (see Table 1). Altogether, students experience more positive than negative emotions in university courses in teacher education.

Gender and Group Differences

Despite huge differences in group size, gender differences regarding appraisals and emotions (see Fischer, 2000) were tested, independently of the learning environment. As for appraisals, perceived academic control [t₍₁₇₀₎ = 2.08, p = 0.04, d = 0.43], technological control [t₍₁₇₅₎ = 2.00, p = 0.05, d = 0.39], and achievement task value [t₍₁₆₉₎ = 2.53, p = 0.01, d = 0.55] showed significant gender differences for females scoring consistently higher than males. Regarding achievement emotions, only for hope [t₍₁₆₈₎ = 3.41, p ≤ 0.001, d = 0.71] significant gender differences were found, females showing more hope. There is no significant gender difference for achievement. It should be noted that only 15 males visited the on-campus course (in comparison to 73 females) and 17 males attended the online course (77 females). Because of the small group size for males (usually at least 20 participants per group are required, see Field, 2013; Huber et al., 2014) and the maladjustment of sample size between both groups, gender was not included as a covariate in further analyses of variance.

Some of the appraisal variables showed significant differences between the online and on-campus group. Achievement task value was rated higher in the on-campus group [t₍₁₆₉₎ = 2.53, p = 0.02, d = 0.55], while technological control was higher in the online group [t₍₁₆₈₎ = 3.41, p ≤ 0.001, d = 0.71]. As for remaining appraisal scales, as well as for age and mean grades of high school diploma, we did not find significant differences.

Regarding achievement emotions, the online group showed less enjoyment [t₍₁₆₇₎ = 3.73, p ≤ 0.001, d = 0.57] but more boredom [t₍₁₆₇₎ = 2.31, p = 0.02, d = 0.35], anxiety [t₍₁₆₈₎ = 2.10, p = 0.04, d = 0.32], and anger [t₍₁₆₆₎ = 3.77, p ≤ 0.001, d = 0.58] than the on-campus group. No significant differences between the two learning environments were found for hope, pride, shame, and hopelessness. Domain specific achievement was significantly higher for the on-campus course than for the online course [t₍₁₂₆₎ = 2.20, p = 0.03, d = 0.39].

Correlations

Due to non-normality of some scales, correlations were computed as Spearman's rank correlation coefficients. As for the total sample, most of the correlations between perceived academic control, achievement task value and emotions were significant, as theoretically expected (Pekrun et al., 2002; Pekrun, 2006; Pekrun and Perry, 2014). High correlations were found between achievement task value with enjoyment (r_s = 0.57, p ≤ 0.001) and hope (r_s = 0.53, p ≤ 0.001), respectively medium correlations with boredom (r_s = −0.41, p ≤ 0.001), pride (r_s = 0.40, p ≤ 0.001), and anger (r_s = −0.42, p ≤ 0.001). Academic control showed medium correlations with hope (r_s = 0.42, p ≤ 0.001), hopelessness (r_s = −0.38, p ≤ 0.001), shame (r_s = −0.33, p ≤ 0.001), anxiety (r_s = −0.41, p ≤ 0.001), and anger (r_s = −0.48, p ≤ 0.001). Correlations for technological control, technological value and technology related beliefs were inconsistent and weaker. While technological value showed medium correlations with enjoyment (r_s = 0.30, p ≤ 0.001), hope (r_s = 0.38, p ≤ 0.001), and pride (r_s = 0.40, p ≤ 0.001), as well as weak significant correlations with hopelessness (r_s = −0.23, p ≤ 0.01) and anger (r_s = −0.20, p = 0.01), only few significant but very weak correlations were found for technological control and technology related beliefs (see Table 2). Regarding students' age, we identified significant correlations only for boredom (r_s = −0.16, p = 0.04) and anxiety (r_s = −0.18, p = 0.02). Domain specific achievement was significantly positive related to hopelessness (r_s = 0.23, p ≤ 0.01) and anxiety (r_s = 0.20, p = 0.03) but neither with other emotions nor with any appraisals, while mean grades of high school diploma were not significantly correlated to emotions or appraisals at all, except for technological control (r_s = −0.26, p ≤ 0.01).

TABLE 2

Table 2. Correlations for total sample and sub-samples with effect size for differences.

Looking at correlations separately for the two groups, some differences between the online course and on-campus course are remarkable. While effect sizes of most of the correlation differences between the two groups were weak, significant differences were confirmed for technological value and positive emotions: For the online group, technological value seems to be more substantial for the arousal of enjoyment (r_s = 0.10, p = 0.39/r_s = 0.46, p ≤ 0.001, q = 0.40) and pride (r_s = 0.28, p = 0.02/r_s = 0.52, p ≤ 0.001, q = 0.29) as for the on-campus group. The online group also showed higher correlations of achievement task value and enjoyment (r_s = 0.06, p = 0.62/r_s = 0.26, p = 0.02, q = 0.29), as well as for perceived academic control, and anxiety (r_s = −0.25, p = 0.02/r_s = −0.52, p ≤ 0.001, q = 0.32) than the on-campus sample. Remarkable differences of correlations between the groups were also shown for domain specific achievement. We found medium differences for achievement task value (r_s = −0.21, p = 0.11/r_s = 0.19, p = 0.13, q = 0.41) and technology related beliefs (r_s = −0.32, p ≤ 0.01/r_s = 0.10, p = 0.40, q = 0.43), both showing negative correlations with achievement outcome for the on-campus sample and positive correlations for the online sample. Weak differences were found for both control appraisals with correlations for perceived academic control (r_s = 0.06, p = 0.62/r_s = −0.12, p = 0.35, q = 0.18) and technological control (r_s = 0.06, p = 0.65/r_s = −0.07, p = 0.57, q = 0.13) being negative for the online sample and slightly positive for the on-campus sample. In total, for the online group we found higher correlations for all perceived academic control and most of the technological value correlations, while for achievement task value, technological control and technology related beliefs correlation differences between the two groups were weaker and inconsistent (see Table 2).

Variance Analyses

For the appraisal scales showing many significant correlations with emotion scales, both control and both value scales, as well as technology related beliefs and age were included as covariates in further analyses of variance. Analyses of covariance (ANCOVA) were computed to test differences of students' achievement emotions between the online- and the on-campus group.

Testing the independence of treatment variable and covariate, the means for perceived academic control [F_{(1, 170)} = 0.88, p = 0.35], technological value [F_{(1, 168)} = 0.47, p = 0.49], technology related beliefs [F_{(1, 179)} = 0.07, p = 0.79], mean grades of high school diploma [F_{(1, 174)} = 0.18, p = 0.67], and gender [F_{(1, 180)} = 0.03, p = 0.86] are not significantly different. But due to the small group size for males and the maladjustment of sample size between both groups, gender was not be included as a covariate in ANCOVA. As already mentioned, t-tests suggest, that achievement task value [F_{(1, 169)} = 5.67, p = 0.02] and technological control [F_{(1, 175)} = 15.18, p ≤ 0.001] are not independent from the learning environment.

For enjoyment, no significant main effect of the learning environment was confirmed after controlling for the covariate effects, F_{(1, 155)} = 3.72, p = 0.06, partial η² = 0.02. Significant effects of covariates were found for achievement task value [F_{(1, 155)} = 45.21, p ≤ 0.001, partial η² = 0.23] and technological value [F_{(1, 155)} = 6.63, p = 0.01, partial η² = 0.04]. Effects of value appraisals—as shown by t-tests—mainly explained significant differences in students' enjoyment between both groups.

For hope, t-test showed no significant difference in emotion value. The main effect of the learning environment on hope after controlling for covariates was not significant, F_{(1, 155)} = 0.15, p = 0.70, partial η² = 0.00. The covariates perceived academic control [F_{(1, 155)} = 18.52, p ≤ 0.001, partial η² = 0.11], achievement task value [F_{(1, 155)} = 22.66, p ≤ 0.001, partial η² = 0.13], and technological value [F_{(1, 155)} = 13.62, p ≤ 0.001, partial η² = 0.08] correlated significantly with students' hope.

Also no significant differences between the groups were found for pride, but ANCOVAS showed a significant main effect of the learning environment after controlling the effects of the covariates, F_{(1, 153)} = 7.84, p = 0.01, partial η² = 0.05. Significant effects of covariates were shown for achievement task value [F_{(1, 153)} = 20.26, p ≤ 0.001, partial η² = 0.12] and technological value [F_{(1, 153)} = 13.66, p ≤ 0.001, partial η² = 0.08]. Hence, the effects of the value appraisals seemed to cover the effect of the learning environment on students' pride.

Regarding negative emotions, int-tests we did not find significant differences between the online- and on-campus group for shame and hopelessness. There was no significant effect of the learning environment for shame [F_{(1, 154)} = 1.59, p = 0.21, partial η² = 0.01] and hopelessness [F_{(1, 154)} = 0.77, p = 0.38, partial η² = 0.01]. The only significant covariate for both emotions was perceived academic control [shame: F_{(1, 154)} = 18.44, p ≤ 0.001, partial η² = 0.11, hopelessness: F_{(1, 154)} = 22.88, p ≤ 0.001, partial η² = 0.13].

The learning environment had no significant effect on students' anxiety after controlling the covariate effects, F_{(1, 155)} = 3.75, p = 0.06, partial η² = 0.02. As only significant covariate, perceived academic control [F_{(1, 155)} = 20.10, p ≤ 0.001, partial η² = 0.12] explained significant differences in students' anxiety between the online and on-campus group.

The strongest main effect of the learning environment after controlling for covariates in this study was confirmed for anger, F_{(1, 153)} = 10.66, p ≤ 0.01, partial η² = 0.07. Perceived academic control [F_{(1, 153)} = 26.12, p ≤ 0.001, partial η² = 0.15] and achievement task value [F_{(1, 153)} = 5.55, p = 0.02, partial η² = 0.04] were significant covariates for students' anger.

Finally, both covariates perceived academic control [F_{(1, 153)} = 4.18, p = 0.04, partial η² = 0.04], as well as achievement task value [F_{(1, 153)} = 7.07, p = 0.02, partial η² p ≤ 0.01] correlated significantly with students' boredom. But there was no significant main effect of the learning environment on boredom after controlling for the covariate effects, F_{(1, 153)} = 3.12, p = 0.08, partial η² = 0.02.

Therefore, the covariate effects can explain significant differences in students' boredom between the online- and on-campus samples.

Surprisingly, for domain specific achievement, neither the main effect, nor the covariates showed significant effects, although the t-test confirmed a significant difference between the groups.

A multivariate analysis of covariance (MANCOVA) was computed to test differences of students' achievement emotions and domain specific achievement between the online- and the on-campus group including all covariates and dependent variables at once. Significant main effects of the learning environment after controlling for the covariate effects were confirmed for anger [F_{(1, 112)} = 11.72, p ≤ 0.01, partial η² = 0.10], pride [F_{(1, 112)} = 6.92, p = 0.01, partial η² = 0.06] and boredom [F_{(1, 112)} = 4.55, p = 0.04, partial η² = 0.04], but not for domain specific achievement outcome [F_{(1, 112)} = 2.70, p = 0.10, partial η² = 0.02]. Significant effects of covariates on emotions were found for academic control (except for pride) and only on positive emotions for achievement task value, technological control, and technological value. For technology related beliefs' the only significant effect was on enjoyment [F_{(1, 112)} = 6.21, p = 0.01, partial η² = 0.05]. No significant effects by covariates were found for domain specific achievement.

Discussion

Based on the control-value theory of Pekrun (2006), the aim of this study was: (1) to analyze to what extent there are differences in the experience of teacher students regarding their achievement emotions and control and value appraisals with respect to individual characteristics, and (2) to compare teacher students' who attended an on-campus vs. an online-course regarding their achievement emotions, control- and value-appraisals, technology-related beliefs, and finally their domain specific achievement. The second research question refers to the assumptions of the technology acceptance model from Davis (1985). With regard to technology-based learning, our study focuses on a very topical issue. However, these research results cannot easily be applied to teacher training programs. The research project has concentrated in particular on the preparation of teacher students for the state examination in school education. First, the descriptive results showed that negative achievement emotions were on a low level for all students participating in this study while mean levels for positive achievement emotions, control and value appraisals were on a medium level. Altogether, students experience more positive than negative emotions in university courses in teacher education.

Regarding the first research question, the results interestingly showed no systematic gender or group differences regarding achievement emotions, and achievement. This is somewhat surprising since research shows that females report more intensively and more frequently on positive, as well as negative emotions than men, in general (Fujita et al., 1991; Barrett and Lally, 1999; Brebner, 2003). But in online learning environments male students made more socio-emotional contributions than women (Barrett and Lally, 1999). In the present study females scored higher for control and value appraisals and technological control. Following Bandura (2001) it may be argued that in the past students' educational development was largely determined by the schools and universities' learning environment to which they were assigned. However, the internet provides much more opportunities for students to control their own learning (Bandura, 2001). Controlling the own learning means self-regulated learning which is an important determinant in the context of online courses. Female students are more related to self-regulated learning than male students do (Joo et al., 2000). This result may be interpreted in line with previous studies (Yukselturk and Bulut, 2009; Cuadrado-García et al., 2010; Anderson and Haddad, 2019). However, it should be noted that these results might also be caused by the maladjustment of sample size regarding gender distribution.

Further results regarding achievement emotions and their appraisals showed that achievement task value was rated higher in the on-campus group while technological control was higher in the online group. Consequently, domain specific achievement was higher in the on-campus group. This supports the previous assumption that for some students it seems to be difficult to learn self-regulated.

Students in the online course reported higher levels of anger, anxiety and boredom than students in the on-campus course do. These results are in line with some previous studies. For example Regan et al. (2012) claimed that online environments have a distinct overall emotional tone that differs from traditional educational settings. Other studies show that technology-based learning environments lead to a bit more pleasure and less anxiety, although these results are not significant (Loderer et al., 2018). It is argued that the resources (especially the ability to learn self-regulated) students have may have an impact on their emotional experience. Students who learn self-determined need less learning time for a better performance, whereas anxious learners need more time and still achieve worse results, which could explain a higher level of anger (Marchand and Gutierrez, 2012; Schulmeister, 2018). In addition to the student-related determinants, the design of the e-learning environment can also intensify negative emotions. It should also not be underestimated that this course prepares for an important exam (the state teacher examination). In this context, besides direct support when questions arise, encouragement from a teacher can reduce anxiety. This personal face-to-face support is missing in the online course. In this context, it is important to note that, in contrast to the on-campus course, the online course was introduced the first time in the teacher training program. This may have caused a maladjustment regarding criteria for implementation quality due to the fact that the online learning environment was novel for both students and lecturers/tutors. Although the implementation of the online course was based on the intervention's program theory and strictly aligned to the established on-campus course, there may be a certain lack in fidelity of implementation (O'Donnell, 2008).

Regardless of the learning environment significant correlations for academic control and achievement task value with most of the achievement emotions support the control-value-theory (Pekrun, 2006; Pekrun and Perry, 2014). Findings for specific technological control, technological value and technology related beliefs were inconsistent and weak in effect size.

Interestingly, with regard to the second research question, some differences between the online and on-campus course are important to note. For the online group higher correlations were found for all perceived academic control and most of the technological value correlations, while for achievement task value, technological control and technology related beliefs correlation differences between both groups were less strong and inconsistent. Teacher students in online courses have emotions that are more negative and the experience less enjoyment. Contrary to our expectations, these differences are not mainly caused by the effect of the different learning environment, but more rather explained by the effects of control- and- value appraisals. Only for anger and pride, significant differences between both groups were explained by the different learning environment but the effect sizes was just medium following (Cohen, 1988) classification.

The differences in terms of feeling anger may have different causes. For example, the user-friendliness of the online course (technology-related control), the lack of personal contact with the lecturer, or the high demand for self-regulated learning, or trouble with interacting with other students could evoke anger. Less experienced pride compared to the on-campus course could be related to the fact that online learning courses offer less opportunities to compare oneself with others or, for example, to receive direct and personal positive feedback. Online assistance such as forums or tutors who give written feedback do not seem to be able to countervail the lack of personal contact and interaction.

Overall, the effect sizes are relatively small, suggesting that significant influence variables regarding achievement emotions were not considered in the two groups. As for domain specific achievement outcome, neither the main effect, nor the covariates showed significant effects. Previous studies on the relationship of emotions and achievement outcomes in e-learning environments are rare and ambiguous. Liew and Tan (2016) conclude that negative emotions reduce learning success. In contrast, they found no significant influence on positive emotions. In a study by Um et al. (2012), however, positive emotions led to improved performance in a transfer test but not in an understanding test.

It seems that other factors besides collected variables may play a more decisive role for achievement outcomes, e.g., intelligence, previous knowledge or achievement motivation. Due to the fact that achievement outcomes were not in the main focus of this study, those factors have not been examined.

Low effect sizes for achievement outcomes may be underestimated due to an attrition effect toward the achievement test. This voluntary test was completed by only 70% (69% in online course, 72% in on-campus course) of the students, so that the relatively small sample size of the study was even more reduced. As effect sizes being sensitive to the sample size, this may have caused lacking significance of the findings. Also the range of measured achievement and therefore the potential explanation of variance could be restricted through reduced sample size. In addition, the shrinking of participants in the achievement test may be a self-selection, given that e.g., only very (extrinsic) motivated and/or unconfident students took part. Opposite effects of these extreme groups may collide and lead to weak overall effects. As having no detailed information about not participating students and the reasons for their absence, potential consequences stay notional and vague.

In sum, it has been shown in this study that the learning environment might affect students' achievement task value and technological control. On the other hand, the results indicate that the learning environment (of online vs. on-campus course) seemed to have only weak effects on students' achievement emotions in this study, but these direct effects might be underestimated as they may be mediated through control- and- value appraisals. Therefore, further analyses are needed.

Regarding the existing research, this study applied Pekrun's (2006) control-value theory in the context of an empirical comparison of an online and an on-campus learning environment in a specific domain, namely teacher education. Furthermore, we measured technology acceptance. Taken together, this study's focus on an online learning environment, along with the application of Pekrun's (2006) control-value framework and the technology acceptance model (Venkatesh and Bala, 2008), is a new contribution to the research field of emotions in teacher education and online learning.

Limitations and Future Research

Regarding the cross sectional design of the study it is a clear limitation that the on-campus and the online courses were rated by students just one time and only with a retrospective focus. Measuring learning processes and emotions several times would have given the opportunity to test for longitudinal effects. Furthermore, the sample was relatively small and with a high percentage of female students—although this gives a representative picture of the distribution between female and male students in teacher education. With such a sample, gender effects have to be interpreted very carefully. Beyond, it was not possible to select students randomly to control for possible influencing factors on the individual level.

Students' technological control, technological value and technology related beliefs correlated only weak and ambiguously with achievement emotions, so that the contribution to the clarification of effects of the different learning environment was rather small. Therefore, the results should only be interpreted with caution. It may be assumed that the learning environment is only one of many other influencing factors, which have an impact on emotions, as well as on the learning process and outcomes. In future studies, further variables should be measured, such as self-regulation ability, self-efficacy, subject specific competencies, and personality. It would be also relevant to measure contextual factors such as support by a lecturer, quality of learning material, or accessibility to core technological features, such as to the online-chat.

Finally, it has to be pointed out that the online course analyzed in this study has just been established without a testing phase, while a very experienced lecturer has conducted the on-campus course in the same way for several semesters. Additionally it should be noted that research on achievement emotions mainly uses self-reports depending on subjectivity and social desirability. Further research in this field should apply research methods that allow a process oriented measurement, as well as observation of students to provide an added value with respect to different (cognitive, subjective, expressional, and behavioral) dimensions of emotions.

To conclude, technology based learning environments are a meaningful future educational setting, and many university students are already used to them. But there is still little known about the emotional and cognitive learning experiences in this environment. With the introduction of e-learning courses, students should also be supported in acquiring self-regulation strategies. While online learning environments may improve access to higher education, it should be considered that these learning environments affect the relationships between students' control and value appraisals, emotions, and achievement in a specific way. Especially for designing effective online learning environments, these relationships have to be taken into account.

From our point of view, further research should focus on specific aspects and tools of online learning environments and analyze them in more detail. The application of different qualitative (e.g., interviews, video observation) and quantitative methods (e.g., state-measurements, scripts of using online tools) may be fruitful. Finally, experimental studies (with variations of different structures and tools of online learning environments based on longitudinal designs would allow a process oriented and differential analysis of the relationships between students' control and value appraisals, achievement emotions, and performance.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Author Contributions

MS performed the measurements. MG-Z supervised the work. SM and MS analyzed the data. MG-Z, SM, and MS were involved in planning and drafted the manuscript. MG-Z took the lead in writing the manuscript, closely supported by MS and SM. All got involved in the interpretation and provided critical feedback and helped shape the research, analysis, and manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Agarwal, R., and Karahanna, E. (2000). Time flies when you're having fun: cognitive absorption and beliefs about information technology usage. Manag. Inform. Syst. Q. 24, 665–694. doi: 10.2307/3250951

CrossRef Full Text | Google Scholar

Anderson, D. M., and Haddad, C. J. (2019). Gender, voice and learning in online course environments. OLJ 9, 3–14. doi: 10.24059/olj.v9i1.1799

CrossRef Full Text | Google Scholar

Artino, A. R., and Jones, K. D. (2012). Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning. Internet High. Educ. 15, 170–175. doi: 10.1016/j.iheduc.2012.01.006

CrossRef Full Text | Google Scholar

Bandura, A. (2001). Social cognitive theory: an agentic perspective. Annu. Rev. Psychol. 52, 1–26. doi: 10.1146/annurev.psych.52.1.1

PubMed Abstract | CrossRef Full Text | Google Scholar

Barrett, E., and Lally, V. (1999). Gender differences in an on-line learning environment. J. Comp. Assist. Learn. 15, 48–60. doi: 10.1046/j.1365-2729.1999.151075.x

CrossRef Full Text | Google Scholar

Bieg, M., Goetz, T., Sticca, F., Brunner, E., Becker, E., Morger, V., et al. (2017). Teaching methods and their impact on students' emotions in mathematics: an experience-sampling approach. Int. J. Math. Educ. 49, 411–422. doi: 10.1007/s11858-017-0840-1

CrossRef Full Text | Google Scholar

Boekaerts, M., Pintrich, P. R., and Zeidner, M. (2000). Handbook of Self-Regulation. San Diego, CA: Academic Press. doi: 10.1016/B978-012109890-2/50030-5

CrossRef Full Text | Google Scholar

Brebner, J. (2003). Gender and emotions. Pers. Individ. Dif. 34, 387–394. doi: 10.1016/S0191-8869(02)00059-4

CrossRef Full Text | Google Scholar

Brown, I. T. J. (2002). Individual and technological factors affecting perceived ease of use of web-based learning technologies in a developing country. Electron. J. Inform. Syst. Dev. Countries 9, 1–15. doi: 10.1002/j.1681-4835.2002.tb00055.x

CrossRef Full Text | Google Scholar

Butz, N. T., Stupnisky, R. H., and Pekrun, R. (2015). Students' emotions for achievement and technology use in synchronous hybrid graduate programmes: a control-value approach. Re. Learn. Technol. 23:26097. doi: 10.3402/rlt.v23.26097

CrossRef Full Text | Google Scholar

Carver, C. S., and Scheier, M. (1990). “Principles of self-regulation: action and emotion,” in Handbook of Motivation and Cognition: Foundations of Social Behavior, eds E. T. Higgins and R. M. Sorrentino (New York, NY: The Guilford Press), 3–52.

Google Scholar

Chang, M. K., and Cheung, W. (2001). Determinants of the intention to use Internet/WWW at work: a confirmatory study. Inform. Manag. 39, 1–14. doi: 10.1016/S0378-7206(01)00075-1

CrossRef Full Text | Google Scholar

Claßen, K. (2012). Zur Psychologie von Technikakzeptanz im höheren Lebensalter: Die Rolle von Technikgenerationen. [On the Psychology of Technology Acceptance in Later Life: The Role of Technology Generations] (disseration). Ruprecht-Karls-University Heidelberg, Heidelberg. Available online at: http://archiv.ub.uni-heidelberg.de/volltextserver/14295/1/Dissertation%20Classen.pdf (accessed August 8, 2019).

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. Hillsdale, NJ: Lawrence Erlbaum.

Compeau, D., Higgins, C. A., and Huff, S. (1999). Social cognitive theory and individual reactions to computing technology: a longitudinal study. Manag.Inform. Syst. Q. 23, 145–158. doi: 10.2307/249749

CrossRef Full Text | Google Scholar

Cuadrado-García, M., Ruiz-Molina, M.-E., and Montoro-Pons, J. D. (2010). Are there gender differences in e-learning use and assessment? Evidence from an interuniversity online project in Europe. Proc. Soc. Behav. Sci. 2, 367–371. doi: 10.1016/j.sbspro.2010.03.027

CrossRef Full Text

Daniels, L. M., and Stupnisky, R. H. (2012). Not that different in theory: discussing the control-value theory of emotions in online learning environments. Internet High. Educ. 15, 222–226. doi: 10.1016/j.iheduc.2012.04.002

CrossRef Full Text | Google Scholar

Davis, F. D. (1985). A technology acceptance model for empirically testing new end-user information systems: theory and results (dissertation). Massachusetts Institute of Technology, Sloan School of Management, Cambridge, MA. Available online at: https://www.jstor.org/stable/249008 (accessed August 8, 2019).

Google Scholar

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340. doi: 10.2307/249008

CrossRef Full Text | Google Scholar

Deci, E. L., and Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. New York, NY: Plenum Press. doi: 10.1007/978-1-4899-2271-7

CrossRef Full Text

D'Errico, F., Paciello, M., and Cerniglia, L. (2016). When emotions enhance students' engagement in e-learning processes. J. ELearn. Knowl. Soc. 12, 9–23. doi: 10.20368/1971-8829/1144

CrossRef Full Text | Google Scholar

Ellis, H. C., and Ashbrook, P. W. (1988). “Resource allocation model of the effects of depressed mood states on memory,” in Affect, Cognition and Social Behavior: New Evidence and Integrative Attempts, eds K. Fiedler and J. Forgas (Toronto, ON: Hogrefe), 25–43.

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. London: SAGE Publications.

Google Scholar

Fischer, A. H. (2000). Gender and Emotion: Social Psychological Perspectives. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511628191

CrossRef Full Text | Google Scholar

Frenzel, A. C., Pekrun, R., and Goetz, T. (2007). Girls and mathematics —a “hopeless” issue? A control-value approach to gender differences in emotions towards mathematics. Eur. J. Psychol. Educ. 22, 497–514. doi: 10.1007/BF03173468

CrossRef Full Text | Google Scholar

Fujita, F., Diener, E., and Sandvik, E. (1991). Gender differences in negative affect and well-being: the case for emotional intensity. J. Pers. Soc. Psychol. 61, 427–434. doi: 10.1037/0022-3514.61.3.427

PubMed Abstract | CrossRef Full Text | Google Scholar

Gefen, D., Karahanna, E., and Straub, D. W. (2003). Inexperience and experience with online stores: the importance of TAM and trust. IEEE Trans. Eng. Manage 50, 307–321. doi: 10.1109/TEM.2003.817277

CrossRef Full Text | Google Scholar

Gefen, D., and Straub, D. W. (1997). Gender differences in the perception and use of e-mail: an extension to the technology acceptance model. Manag. Inform. Syst. Q. 21, 389–400. doi: 10.2307/249720

CrossRef Full Text | Google Scholar

Goetz, T., Frenzel, A. C., Pekrun, R., Hall, N. C., and Lüdtke, O. (2007). Between- and within-domain relations of students' academic emotions. J. Educ. Psychol. 99, 715–733. doi: 10.1037/0022-0663.99.4.715

CrossRef Full Text | Google Scholar

Götz, T., Keller, M. M., and Martiny, S. E. (2012). “Emotionales Erleben in den MINT-Fächern: Ursachen, Geschlechtsunterschiede und Interventionsmöglichkeiten [Emotional experience in the STEM subjects: causes, gender differences, and possibilities of intervention],” in Mädchen und Frauen in MINT: Bedingungen von Geschlechtsunterschieden und Interventionsmöglichkeiten [Girls and Women in STEM: Conditions of Gender Differences and Possibilities of Intervention], eds H. Stöger, A. Ziegler, and M. Heilemann (Berlin; Münster: LIT), 135–161.

Hamborg, K.-C., Brummerloh, M., Gieseking, M., and Wegner, J. (2014). Befunde zur Akzeptanz des Lernmanagement-Systems Stud.IP an der Universität Osnabrück. [Findings on the Acceptance of the Learning Management System Stud. IP at the University of Osnabrück]. Zentrum für Informationsmanagement und virtuelle Lehre. [Center for information management and virtual teaching]. Osnabrück. Available online at: https://repositorium.ub.uni-osnabrueck.de/bitstream/urn:nbn:de:gbv:700-2014062712580/2/workingpaper_01_2014_virtuos.pdf (accessed August 8, 2019).

Hawlitschek, A., and Fredrich, H. (2018). Die Einstellungen der Studierenden als Herausforderung für das Lehren und Lernen mit Digitalen Medien in der Wissenschaftlichen Weiterbildung [The Attitudes of Students as a Challenge for Teaching and Learning With Digital Media in Academic Further Education]. Kassel: Zeitschrift Hochschule und Weiterbildung (ZHWB)), 9–16.

Google Scholar

Huber, F., Meyer, F., and Lenzen, M. (2014). Grundlagen der Varianzanalyse: Konzeption - Durchführung - Auswertung. [Fundamentals of Variance Analysis: Conception - Implementation - Analysis], 14th Edn. Wiesbaden: Springer.

Iivari, J., and Ervasti, I. (1994). User information satisfaction: IS implementability and effectiveness. Inform. Manag. 27, 205–220. doi: 10.1016/0378-7206(94)90049-3

CrossRef Full Text | Google Scholar

Jackson, K. (2015). Math anxiety and attitudes in pre-service elementary teachers (thesis and dissertation). Rowan University, Glassboro, NJ. Available online at: https://rdw.rowan.edu/etd/409 (accessed August 13, 2018).

Joo, Y.-J., Bong, M., and Choi, H.-J. (2000). Self-efficacy for self-regulated learning, academic self-efficacy, and internet self-efficacy in web-based instruction. Educ. Technol. Res. Dev. 48, 5–17. doi: 10.1007/BF02313398

CrossRef Full Text | Google Scholar

Kaspar, R., Stefanie, B., and Mollenkopf, H. (2002). Technik im Alltag von Senioren: Arbeitsbericht zu Vertiefenden Auswertungen der BIS-Repräsentativerhebung. Teilprojekt Sozialwissenschaften der Interdisziplinären Forschergruppe sentha (Senioren und Technik). [Technology in the Everyday Life of Seniors: Work Report on in-Depth Analyses of the BIS Representative Survey. Subproject Social Sciences of the Interdisciplinary Research Group Sentha (Seniors and Technology)]. Heidelberg.

Keengwe, J., and Kang, J.-J. (2013). A review of empirical research on blended learning in teacher education programs. Educ. Inform. Technol. 18, 479–493. doi: 10.1007/s10639-011-9182-8

CrossRef Full Text | Google Scholar

Lehman, B., D'Mello, S., and Graesser, A. (2012). Confusion and complex learning during interactions with computer learning environments. Internet High. Educ. 15, 184–194. doi: 10.1016/j.iheduc.2012.01.002

CrossRef Full Text | Google Scholar

Liew, T. W., and Tan, S.-M. (2016). The effects of positive and negative mood on cognition and motivation in multimedia learning environment. Educ. Technol. Soc. 19, 104–115.

Google Scholar

Loderer, K., Pekrun, R., and Lester, J. C. (2018). Beyond cold technology: a systematic review and meta-analysis on emotions in technology-based learning environments. Learn. Instruct. doi: 10.1016/j.learninstruc.2018.08.002. [Epub ahead of print].

CrossRef Full Text | Google Scholar

Malinsky, M., Ross, A., Pannells, T., and McJunkin, M. (2006). Math anxiety in pre-service elementary school teachers. Education 127,274–279.

Google Scholar

Marchand, G. C., and Gutierrez, A. P. (2012). The role of emotion in the learning process. Comparisons between online and face-to-face learning settings. Internet High. Educ. 15, 150–160. doi: 10.1016/j.iheduc.2011.10.001

CrossRef Full Text | Google Scholar

O'Donnell, C. L. (2008) Defining, conceptualizing, and measuringfidelity of implementation and its relationshipto outcomes in K−12 curriculum intervention research. Rev. Educ. Res. 78, 33–84. doi: 10.3102/0034654307313793.

CrossRef Full Text | Google Scholar

Olbrecht, T. (2010). Akzeptanz von e-learning: eine Auseinandersetzung mit dem Technologieakzeptanzmodell zur analyse individueller und sozialer Einflussfaktoren. [Acceptance of e-learning: a discussion of the technology acceptance model for the analysis of individual and social influencing factors.] (dissertation). Friedrich-Schiller-University of Jena, Jena. Available online at: https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00021996/Olbrecht/Dissertation.pdf (accessed March 14, 2018).

Pekrun, R. (2006). The control-value theory of achievement emotions: assumptions, corollaries, and implications for educational research and practice. Educ. Psychol. Rev. 18, 315–341. doi: 10.1007/s10648-006-9029-9

CrossRef Full Text | Google Scholar

Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., and Perry, R. P. (2011). Measuring emotions in students' learning and performance: the achievement emotions questionnaire (AEQ). Contemp. Educ. Psychol. 36, 36–48. doi: 10.1016/j.cedpsych.2010.10.002

CrossRef Full Text | Google Scholar

Pekrun, R., Goetz, T., and Perry, P. R. (2005). Academic Emotions Questionnaire (AEQ): User's Manual. Munich: University of Munich, Department of Psychology; University of Manitoba. Retrieved February 21, 2017. Available online at: https://de.scribd.com/doc/217451779/2005-AEQ-Manual# (accessed July 17, 2019).

Pekrun, R., Goetz, T., Titz, W., and Perry, R. P. (2002). Academic emotions in students' self-regulated learning and achievement: a program of qualitative and quantitative research. Educ. Psychol. 37, 91–105. doi: 10.1207/S15326985EP3702_4

CrossRef Full Text | Google Scholar

Pekrun, R., and Perry, R. P. (2014). “Control-value theory of achievement emotions,” in Handbook of Emotions and Education, eds R. Pekrun and L. Linnenbrink-Garcia (London: Routledge), 120–141.

Google Scholar

Pekrun, R., and Stephens, E. J. (2010). Achievement emotions: a control-value approach. Soc. Personal. Psychol. Compass 4, 238–255. doi: 10.1111/j.1751-9004.2010.00259.x

CrossRef Full Text | Google Scholar

Regan, K., Evmenova, A., Baker, P., Jerome, M. K., Spencer, V., Lawson, H., et al. (2012). Experiences of instructors in online learning environments: identifying and regulating emotions. Internet High. Educ. 15, 204–212. doi: 10.1016/j.iheduc.2011.12.001

CrossRef Full Text | Google Scholar

Ryan, R. M., and Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78. doi: 10.1037/0003-066X.55.1.68

PubMed Abstract | CrossRef Full Text | Google Scholar

Saadé, R., and Bahli, B. (2005). The impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: an extension of the technology acceptance model. Inform. Manag. 42, 317–327. doi: 10.1016/j.im.2003.12.013

CrossRef Full Text | Google Scholar

Saadé, R., and Kira, D. (2006). The emotional state of technology acceptance. IISIT 3, 529–539. doi: 10.28945/913

CrossRef Full Text

Saadé, R. G., and Kira, D. (2009). Computer anxiety in e-learning: The effect of computer self-efficacy. J. Inform. Technol. Educ. 8, 177–191. doi: 10.28945/166

CrossRef Full Text | Google Scholar

Scherer, R., Siddiq, F., and Tondeur, J. (2019). The technology acceptance model (TAM): a meta-analytic structural equation modeling approach to explaining teachers' adoption of digital technology in education. Comput. Educ. 128, 13–35. doi: 10.1016/j.compedu.2018.09.009

CrossRef Full Text | Google Scholar

Schmid, U., Goertz, L., Radomski, S., Thom, S., and Behrens, J. (2017). Monitor Digitale Bildung: Die Hochschulen im digitalen Zeitalter [Digital Education Monitor: Higher Education in the Digital Age]. Gütersloh: Bertelsmann Stiftung.

Schulmeister, R. (2018). “Präsenz und selbststudium im e-learning: indizien für eine besondere Rolle der Präsenz [Presence and selfinstruction in e-learning: Indications of a special role of presence],” in Digitale Lehrformen für ein Studierendenzentriertes und Kompetenzorientiertes Studium. Eine Tagung des Projekts nexus in Zusammenarbeit mit dem Center für Digitale Systeme (CeDiS) der Freien Universität Berlin [Digital Teaching Formats for Student-Centred and Competence-Oriented Studies. A Conference of the Nexus Project in Cooperation With the Center for Digital Systems (CeDiS) of Freie Universität Berlin]. Conference Transcript (1st ed.), ed German Rectors' Conference (HRK), and Free University of Berlin (Münster: Waxmann), 6–26.

Tulis, M. (2010). Individualisierung im Fach Mathematik: Effekte auf Leistung und Emotionen [Individualization in Mathematics: Effects on Achievement and Emotions]. Berlin: Logos.

Um, E., Plass, J. L., Hayward, E. O., and Homer, B. D. (2012). Emotional design in multimedia learning. J. Educ. Psychol. 194, 485–498. doi: 10.1037/a0026609

CrossRef Full Text | Google Scholar

Venkatesh, V. (2000). Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. Inform. Syst. Res. 11, 342–365. doi: 10.1287/isre.11.4.342.11872

CrossRef Full Text | Google Scholar

Venkatesh, V., and Bala, H. (2008). Technology acceptance model 3 and a research agenda on interventions. Decis. Sci. 39, 273–315. doi: 10.1111/j.1540-5915.2008.00192.x

CrossRef Full Text | Google Scholar

Venkatesh, V., and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag. Sci. 46, 186–204. doi: 10.1287/mnsc.46.2.186.11926

CrossRef Full Text | Google Scholar

Wigfield, A., and Eccles, J. S. (1992). The development of achievement task values: a theoretical analysis. Dev. Rev. 12, 265–310. doi: 10.1016/0273-2297(92)90011-P

CrossRef Full Text | Google Scholar

Wong, G. K. W. (2015). Understanding technology acceptance in pre-service teachers of primary mathematics in Hong Kong. Aust. J. Educ. Technol. 31, 713–735. doi: 10.14742/ajet.1890

CrossRef Full Text | Google Scholar

Yuan, R., and Lee, I. (2016). ‘I need to be strong and competent': a narrative inquiry of a student-teacher's emotions and identities in teaching practicum. Teach. Teach. 22, 819–841. doi: 10.1080/13540602.2016.1185819

CrossRef Full Text | Google Scholar

Yukselturk, E., and Bulut, S. (2009). Gender differences in self-regulated online learning environment. Educ. Technol. Soc. 12, 12–22.

Google Scholar

Zembylas, M. (2008). Adult learners' emotions in online learning. Distance Educ. 29, 71–87. doi: 10.1080/01587910802004852

CrossRef Full Text | Google Scholar