Affective Profiles in a Massive Open Online Course and Their Relationship with Engagement

The importance of affects in learning has been firmly established in face-to-face learning environments (Pekrun, 2006), and now researchers are examining the roles of affects in online learning environments. A special issue of Internet in Higher Education showed strong similarities between the experience and effects of emotions/affects in online learning environments and traditional classrooms (Daniels and Stupnisky, 2012). However, little is known about the role of affective experience in perhaps the newest and arguably least traditional online learning environment: massive open online courses (MOOCs). Not only do MOOCs represent a novel learning environment but they also consist of a highly diverse set of learners united from all parts of the world by a shared interest in content and an Internet connection (Saadatdoost et al., 2015). The purpose of this research was threefold. First, we identified profiles of common achievement affects, namely, anxiety, relief, boredom, and guilt over the duration of one MOOC. Second, we investigated the extent to which learners’ interpretation of the MOOC as providing support for their competence and autonomy predicted affective profiles. And third, we examined the differences in cognitive, behavioral, and social engagement associated with each profile.

Massive Open Online Courses

In their original form, MOOCs were designed to generate knowledge through connectivity. These “connectivist MOOCs” were based on a pedagogy of sharing information through digital networks (Kop and Hill, 2008; Siemens, 2012), and providing an open platform for students to discuss content and create learning through connecting. Indeed, the first MOOC called “Connectivism and Connective Knowledge” was run through the University of Manitoba, Canada and involved 25 learners with official student status and 2,300 learners unaffiliated with the university (Daniel, 2012).

Although the connectivity associated with the original MOOCs may be appealing for some, it also made course design and delivery more difficult. Thus, the second generation of MOOCs emerged: xMOOCs. Rather than emphasizing connectivism, xMOOCs follow a learning structure more synonymous with traditional university (Clarke, 2013). For example, Udacity’s xMOOC on artificial intelligence provides learners with a syllabus, instructional videos, and an estimated timeline for course completion (Dennis, 2012; Udacity, 2011–2017). Although xMOOCs register unprecedented numbers of learners, for example, 160,000 learners enrolled in the inaugural artificial intelligence course, these learners are not necessarily intentionally connected to create knowledge, and instead all receive the same instruction. This format has allowed many world-class universities to offer xMOOCs to students for credit and to individuals interested in learning but not enrolled at the university (Dennis, 2012). According to 2014 statistics, there are at least 400 universities offering 2,400 MOOCs (Shah, 2014), and interest appears to be expanding (Liyanagunawardena et al., 2013).

Although MOOCs appear to be popular, their usage data reveal interesting patterns regarding enrollment and completion (Ferguson and Clow, 2015). For instance, MOOC students demonstrate varying levels of participation and interaction with the online material. They have been classified into four patterns of engagement, which include sampling, those who watched some videos but did not finish any assignments or exams, auditing, those who watched most videos but chose not to finish assignments or exams, completing, those who finished most of the assignments, quizzes, and disengaging, those who finished the assignments at the beginning only (Kizilcec et al., 2013). Engagement is also lower for people who enroll after a scheduled start date than those who enroll before the date (Cassidy et al., 2013). Kizilcec et al. (2013) concluded that learners classified as completing rated their overall experience as significantly better than remaining groups. However, this is often quite a small group relative to enrollment. In fact, it is quite common for completion rates to be between 4 and 10% (Liyanagunawardena et al., 2013; Stein, 2013) even though the students report enrolling out of interest, for a challenge, to experience MOOCs, or to receive a certificate of completion (Hew and Cheung, 2014). Most of this information has been derived from usage reports and does not bring a psychological perspective to bear on students’ experiences of an MOOC.

Empirical Research on MOOCs

A small body of research on motivation and affective experience specifically in MOOCs is accumulating. For example, Durksen et al. (2016) found that one MOOC environment provided high degrees of autonomy and competence, but not relatedness. The researchers speculated that autonomy is easier to foster given the high degree of freedom MOOCs grant students, while relatedness may be more difficult given their independent nature. In another study on the same MOOC, students reported lower levels of social engagement than cognitive or behavioral engagement (Daniels et al., 2016). Nawrot and Doucet (2014) suggest that two barriers to engagement are inability to manage time and maintain motivation.

Affective experiences are also receiving some attention. For example, Daniels et al. (2016) found that students reported more pleasant than unpleasant affects during an MOOC with 61% of participants reporting high levels of enjoyment and 71% indicating they did not feel bored (Daniels et al., 2016). Affective experiences have also been linked to the frequency with which students access course content. For example, in a hybrid version of an MOOC within a small private online class combined with in-class instruction students who consistently accessed the online resources experienced less test anxiety and had a better academic performance when compared with peers who engaged with the material less (Noteborn and García, 2016). Students in a qualitative study of their post-course evaluation of an MOOC (Knox, 2014) called E-Learning and Digital Cultures expressed feeling overwhelmed because the content of the MOOC was too vast, leading to anxiety. They described the course as providing overload, noise, a sense of loss, and used metaphors of water (i.e., the ocean) to describe its massiveness. One researcher has suggested that MOOCs put a high cognitive load on novice students, giving early learners too much information too quickly, creating negative affects such as anxiety and also boredom (Brennan, 2013). Applying a theoretical perspective to this emerging body of literature can help give it some cohesion.

Affective Experiences and Learning

Postsecondary student’s affective experiences have received lots of attention from educational researchers (e.g., Daniels et al., 2009; D’Mello et al., 2010; Pekrun et al., 2011; Tempelaar et al., 2012; Villavicencio and Bernardo, 2013; Tze et al., 2016). Much of the evidence on students’ affects is based on Pekrun’s (Pekrun, 2006) control-value theory of emotions. According to this theoretical framework, an emotion/affect can be categorized as either activating or a deactivating. Anxiety and boredom are both negative emotions/affects. While boredom is a deactivating one, anxiety is a more complex affect which is both activating and deactivating. Specifically, anxiety is associated with both reduced intrinsic motivation and an increase of extrinsic motivation to avoid failures (Pekrun, 2006). The complex nature of anxiety can also be explained by the Yerkes–Dodson law (Reevy et al., 2010), which delineates the relationship between affective arousal and performance. Low anxiety may be failing in generating sufficient arousal for engagement and optimal performance. By contrast, high anxiety is associated with too much arousal—being overwhelmed, which in turn is related to reduced performance. In addition to categorizing emotions/affects, the control-value theory of emotions explains how the learning environment and cognitive appraisal contribute to the experience of a given affective experience and in turn what consequences result from the affect (Pekrun, 2006; Pekrun et al., 2011).

Research on both the antecedents and consequences of affects in educational settings has expanded recently (e.g., Leony et al., 2015). In terms of antecedents, cognitive appraisals are considered the most proximal. For example, Pekrun et al. (2010) found that boredom was predicted by low levels of control and value appraisals. The theory also acknowledges that the learning environment serves as a distal antecedent affecting emotions/affects through cognitive appraisals (Pekrun and Perry, 2014). For example, Pekrun and Stephens (2009) commented that achievement goals were related to experience of different affects and indeed several studies document a relationship between goal orientation and affects. Daniels et al. (2009) found that mastery goals were negatively associated with boredom and anxiety. Pekrun et al. (2006, 2009) replicated these results and extended them to show that mastery was positively associated with hope and pride and negatively associated with anger, hopelessness, and shame whereas performance approach was positively associated with only pride. A recent meta-analysis (Huang, 2011) concluded that the associations between mastery and positive affects and performance-avoidance and negative affects were large based on Cohen’s guidelines. It has also been argued that perceived control in Pekrun’s (Pekrun, 2006) framework is consistent with Deci and Ryan’s (Deci and Ryan, 2012) basic psychology needs of autonomy or competence. Tong et al. (2009) examined how competence was related to affects. The authors found that competence was negatively associated with anger, sadness and fear and was positively related to joy. In the study of Tze et al. (2014), perceived autonomy support was negatively related to experience of boredom. These studies together suggest the importance to take both distal and proximal antecedents into consideration to explain how a particular affect is triggered in an MOOC.

An increasing number of studies have investigated consequences of experiencing a particular affect in achievement settings. Anxiety, particularly test anxiety, has been the traditional focus when evaluating affects in educational contexts (e.g., Pekrun et al., 2002; Chapell et al., 2005); whereas other affects (e.g., boredom) have gained momentum in the past decade. Researchers are often interested in the impact of a particular affect on students’ engagement and achievement. Villavicencio and Bernardo (2013) found that anger, anxiety, shame, and hopelessness negatively correlated with university student’s final grade. Similarly, in Tze et al. (2016) boredom had a modest negative effect size with academic attainment. Pekrun and his colleagues found that postsecondary students’ positive affects (e.g., hope) were related to self-regulation and course performance while a reverse pattern was observed for negative affects (e.g., anxiety, anger, and boredom). While positive emotions (e.g., enjoyment) positively predicted achievement (e.g., Pekrun et al., 2017), relief is an exception, which is not positively associated with adaptive learning strategies (e.g., organization; Pekrun et al., 2002) nor with intrinsic motivation or effort (Pekrun et al., 2011).

Although our current understanding of different discrete achievement affects with regard to antecedents and consequences has been increasing recently, most studies were conducted in traditional face-to-face classroom settings. In light of current paradigm swing to technological-based learning, particularly MOOCs, it is important to examine how affects emerge, change, and relate to engagement in this novel learning environment. Specifically, the investigation of negative affects (e.g., boredom, anxiety, and guilt) and the exceptional positive affect—relief—would shed light on the affective responses and the associated engagement patterns among MOOC learners.

Methodology

Procedure

This study was reviewed and approved by the University of Alberta Research Ethics Board 2 which adheres to the Declaration of Helsinki. Informed consent was implied by the overt action of completing the questionnaire after reading the information letter. These data were collected from the University of Alberta’s second offering of an xMOOC called Dino 101. This xMOOC was about dinosaur paleobiology and was offered during the winter 2014 semester. Dino 101 consisted of 12 weekly lessons, and students had the option of following a schedule or creating their own timeline. It utilized various interactive formats including discussion boards, video lectures, and wikis. Over 23,000 individuals world-wide enrolled in Dino 101 and completed the course for one of the following reasons: (1) for credit at the University of Alberta, (2) for a certificate in a “signature track” for Coursera, which students earned by completing 12 quizzes, a midterm, and a final exam, or (3) for no form of credit. The majority of participants who completed the survey (97%) were not enrolled for credit. We collected the data via online surveys from 1,035 individuals enrolled in the MOOC from January to April 2014. We sent the students an electronic link to a questionnaire (via Survey Monkey©) in an email from the professor at four time points. The students received the email with the survey link after they completed lesson 2, called “Death and Fossilization,” lesson 6, called “Attack and Defense,” lesson 8, called “Evolution,” and lesson 12, called “Dinosaur Extinction.” All students enrolled in the MOOC received the email and link regardless of their progress in the course. Participation was voluntary. The surveys required approximately 10 min to complete and included questions about the students’ motivational and affective experiences in Dino 101.

Participants

We retained data from a total of 320 MOOC learners who completed all four administrations of the questionnaire (56.5% female and 43.4% male). More than one-third of these participants were between 25 and 34 years old, and about a quarter were between 18 and 24 years old. With regard to levels of education, 42.8% received a Bachelor’s degree and 26.9% obtained an advanced degree. The specific age distribution and educational status are provided in Table 1.

TABLE 1

Table 1. Percentages of each age group and education level.

Measures

Basic Psychological Needs

Two basic psychological needs—autonomy and competency—were measured at Time 1 using the scale developed by Betoret and Artiga (2011). The original scale contained four items measuring each basic psychological need, and the autonomy and competency subscales showed acceptable reliabilities (i.e., αs = 0.76 and 0.65, respectively). In consideration of practical constraints, we selected three of the four items and adapted the subscale to suit an MOOC context. The measure began with a common statement (i.e., “to what extent do you agree with the following item”), followed by the needs of autonomy and competency statements, such as “I felt capable while learning in Dino 101” (competency) and “I was able to freely choose the task I will do during Dino 101” (autonomy). Participants were asked to respond using a 7-point scale (1 = strongly disagree to 7 = strongly agree). This adapted scale showed good reliability: α = 0.81 (autonomy) and α = 0.82 (competency).

Affect

A single item was used to assess MOOC learners’ boredom, relief, guilt, and anxiety across all four time points. Participants responded on a 7-point scale (1 = strongly disagree to 7 = strongly agree). The items were from the Academic Emotions Questionnaire developed by Pekrun et al. (2005). Gogol et al. (2014) found that a single-item measure still has sound psychometric properties and is an alternative when full-scale measures are too cumbersome to be administered.

Engagement

Behavioral engagement and cognitive engagement were measured using the school engagement scale developed by Fredericks et al. (2005). We adapted two items for each engagement domain and created one item for each domain, whereby three items were used to assess behavioral and social engagement. An example used to measure cognitive engagement was “I read extra materials to learn more about things we did in DINO 101”, and an example for behavioral engagement was “I followed the rules of DINO 101”. Social engagement was assessed using Klassen et al. (2013) measure. Three items were selected from the original scale and were adapted to Dino 101. A sample item to measure social engagement was “I connected well with my peers in DINO101.” These adapted engagement scales showed good to excellent reliabilities: α = 0.73 (behavioral engagement), α = 0.76 (cognitive engagement), and α = 0.90 (social engagement).

Plan of Analyses

First, latent profile analysis (LPA) was used to classify individuals experience similar pattern of each affect over the MOOC course, based on their responses to a continuous measure in each affect category: anxiety, boredom, guilt, and relief. The number of classes, or profiles, was examined. Marsh et al. (2009) discussed an exploratory approach of LPA. These authors suggested that different numbers of profiles should be examined using LPA, and the best one is selected based not on a single index score but rather on a thorough consideration with regards to theoretical support, prior empirical evidences and interpretability of findings, in addition to likelihood-based tests and information-based indices. The Bayesian Information Criterion (BIC) is a good indicator to identify numbers of profiles (Nylund et al., 2007). The Lo–Mendell–Rubin (LMR) test, when used for a modest sample size, provides acceptable significant test results on identifying a correct number of profiles and to reject null hypothesis (i.e., when compared with k − 1 class solution; e.g., Nylund et al., 2007; Tofighi and Enders, 2007). In addition, high entropy value was recommended to be included for a selection of the best solution (Zhao and Karypic, 2004; Jung and Wickrama, 2008). A smaller BIC value, a significant LMR test result, and a closer to one entropy value (Lo et al., 2001; Zhao and Karypic, 2004), coupled with theoretical support and interpretability, were used to select the best fitting solution in this study. Second, after the best solution was identified, levels of education, competency, and autonomy were used to predict class membership in each affect category, using multinomial logistic regression. Third, the extent to which belonging to a given class in boredom, relief, guilt, and anxiety differed on levels of behavioral, cognitive, and social engagement were examined using multivariate analysis of variance (MANOVA).

Results

Preliminary Analyses

Table 2 shows descriptive statistics of all measures used in the study. Table 3 has the correlations between all variables. Levels of education were negatively related to competency, r = −0.16, p = 0.004, whereas autonomy and competency were positive related, r = 0.77, p < 0.001. Negative affects (i.e., anxiety, boredom, and guilt) significantly and positively correlated with one another on all instances (rs range from 0.13 to 0.75, ps range from 0.025 to <0.001). Interestingly, relief was positively related to anxiety across all times (rs range from 0.19 to 0.27, ps < 0.001). All three dimensions of learners’ engagement were significantly correlated (rs range from 0.14 to 0.58, ps range from 0.01 to <0.001).

TABLE 2

Table 2. Descriptive statistics of all measures.

TABLE 3

Table 3. Correlations of all measures.

LPA Model Selection and Class Membership

Anxiety

Table 4 shows the LPA results. For anxiety, the three-class solution was selected because of a lower BIC value of 3,998.02 and close to 1.0 entropy value (i.e., 0.996), coupled with a significant LRT result when compared a two-class solution (p < 0.001) and a non-significant result in a four-class solution (p = 0.11). Majority of learners (76.9%) fell into Class 1: having low-grade anxiety. The remaining two classes showed similar trends but differed in intensity. There were 14.3% of learners in Class 2: modest level of anxiety fluctuating over the course. More specifically, this group of learners became more anxious after the course started and yet was able to manage this feeling at about the item-mean level from the third quarter onward. Class 3 described a group of anxious learners, who showed a steep jump of anxious feeling once the course began and remained substantially anxious. Figures 1–4 show the profiles for each affect examined.

TABLE 4

Table 4. Latent profile analysis results.

FIGURE 1

Figure 1. Latent profiles of anxiety.

FIGURE 2

Figure 2. Latent profiles of boredom.

FIGURE 3

Figure 3. Latent profiles of guilt.

FIGURE 4

Figure 4. Latent profiles of relief.

Boredom

For boredom, although the BIC values suggested more complex solutions, a two-class solution was chosen in view of interpretability of results, LRT comparisons (i.e., significantly better than a one-class solution, p < 0.001, but no significant difference with a three-class solution, p = 0.34), and a high entropy value for a two-class solution (i.e., 0.975). Furthermore, when covariates—education, competency, and autonomy—were included, these selected models remained the best-fitting models while preserving the nature of class memberships. Many learners (82.5%) belonged to Class 1, which showed a slight reduction of low-grade boredom over time. However, the remaining 17.5% learners had a concerning boredom profile. Their level of boredom increased drastically over time and only returned to the item-mean level at the end of the course.

Guilt

For guilt, although the BIC values suggested more complex solutions, a two-class solution was chosen in view of interpretability of results, LRT comparisons, and a high entropy value of 0.999 for a two-class solution. The two-class solution also remained best fitting when considering education, competency, and autonomy as covariates. Class 1 captured 90.6% learners with regard to their feelings of guilt. This particular profile featured a low level of guilt being reduced over time. By contrast, Class 2 (9.4%) had an ever-increasing sense of guilt, and such feeling reached a plateau toward the end of the course.

Relief

When selecting the best fitting model for relief, a consideration of having no qualitative change in the nature of class membership takes precedent (e.g., Marsh et al., 2009). Although the model-fit indicators suggested a more complex four-class solution, the nature of class membership was not interpretable when covariates were included. In light of this, a more parsimonious model (two-class solution), which showed an adequate fit and an interpretable result with covariates, was selected (BIC = 4,643.22, entropy = 0.857). Most students (69.7%) belonged to Class 1, which showed a moderate level of relief over time. By contrast, Class 2 featured a group of learners who reported a high level of relief at the beginning of the course and this feeling increased over time.

Prediction of Class Membership by Education, Autonomy, and Competence

Anxiety

One level up in education and a unit increase in perceived competency are related to a 0.321 increase and a 0.269 increase, respectively, in relative log odds of being in Class 1 (consistently low-grade anxiety) than in Class 3 (a steep jump and an intense anxious feeling). By contrast, none of the predictors showed a significant finding between Class 2 and Class 3.

Boredom

Autonomy played a role in predicting class membership of boredom. A unit increase in autonomy resulted in a 0.117 increase in relative log odds of being in Class 1 (a steadily reduction of low-grade boredom) than in Class 2 (an inverted U shape of boredom).

Guilt

Competency and autonomy influenced class membership for guilt. One unit increase in competency and autonomy measures were associated with a 0.173 increase and a 0.133 increase in relative log odds of being in Class 1 (low guilt) than in Class 2 (an increase sense of feeling guilty).

Relief

Interestingly, one level up in education was associated with a 0.348 increase in relative log odds of being in Class 1, which showed a moderate level of relief. By contrast, one unit increase in autonomy was related to a 0.16 decrease in relative log odds of being in Class 1.

Differences in Engagement among Class Memberships

Given that behavioral, cognitive, and social engagement were significantly related to one another (p < 0.05), MANOVA was used to identify how class memberships differed in this outcome measure. Table 5 shows the results of MANOVA.

TABLE 5

Table 5. Differences in engagement among emotion profiles.

Anxiety

The three LPA anxiety classes were examined regarding their differences in engagement using MANOVA, and the result was significant, Wilks’ lambda F(6) = 9.584, p < 0.001. Post hoc analysis, with adjustment made using Scheffe test, was conducted. Scheffe test is the most conservative procedure that minimizes Type 1 error when performing multiple comparisons (e.g., Grice and Iwasaki, 2007; Kao and Green, 2008). Results revealed Class 1 had a significantly higher level of behavioral engagement (M = 19.63, SD = 2.01) than Class 2 (M = 17.00, SD = 3.74, p < 0.001), and Class 3 (M = 17.82, SD = 4.63, p = 0.003). In addition, learners belonging to Class 1 also showed a higher level of cognitive engagement (M = 18.25, SD = 3.15) than Class 2 (M = 16.74, SD = 3.80, p = 0.021). However, Class 3, which was characterized by high anxiety, reported a higher level of social engagement (M = 14. 61, SD = 4.57) than Class 1 (M = 12.20, SD = 4.44, p = 0.024). Despite reporting high level of anxiety, Class 3 did not differ from Class 2 in terms of behavioral, cognitive, and social engagement.

Boredom

The two LPA boredom classes significantly differed on the levels of engagement, Wilks’ lambda F(3) = 24.097, p < 0.001. In particular Class 1, showing a steadily reduction of low-grade boredom, reported higher levels of behavioral (M = 19.62, SD = 2.11) and cognitive engagement (M = 18.46, SD = 2.83) than Class 2 (behavioral engagement: M = 16.61, SD = 4.08, p < 0.001; cognitive engagement: M = 15.50, SD = 4.65, p < 0.001), which was characterized by an inverted U shape pattern of boredom.

Guilt

The extent to which the two LPA guilt classes differed on engagement was evaluated using MANOVA. The result was significant, Wilks’ lambda F(3) = 40.31, p < 0.001. Class 1, which featured a steady reduction of low-grade guilt, reported being more engaged behaviorally (M = 19.55, SD = 2.05) and cognitively (M = 18.31 = SE = 3.05) than Class 2 (behavioral engagement: M = 14.67, SD = 4.71, p < 0.001; cognitive engagement: M = 14.43; SD = 4.58, p < 0.001), which had an increasing pattern of feeling guilty over time.

Relief

The MANOVA result of engagement was also significant among the two LPA relief classes, Wilks’ lambda F(3) = 20.928, p < 0.001. Specifically, Class 2 (a high level of relief sustained) reported more engagement across all three dimensions (behavioral engagement: M = 19.70, SD = 1.93; cognitive engagement: M = 19.15, SD = 2.73; and social engagement: M = 15.05, SD = 4.73) than Class 1 (behavioral engagement: M = 18.83, SD = 3.08, p = 0.01; cognitive engagement: M = 17.42, SD = 3.54, p < 0.001; and social engagement: M = 11.31, SD = 3.77, p < 0.001), which showed a moderate level relief over the course.

Discussion

The overarching goal of this study was to investigate affective profiles, their antecedents, and impact on engagement in an MOOC. By using LPA, we were able to identify different profiles within each affect (i.e., anxiety, boredom, guilt, and relief) over time, and each profile was associated with a unique set of antecedents and engagement. Furthermore, in Saadatdoost et al. (2015) review of MOOCs literature, the authors called for a further investigation of engagement among MOOC learners. Therefore, our study also provided insights regarding how affective profiles were associated with varying degree of behavioral, cognitive, and social engagement in an MOOC.

Affective Profiles during an MOOC

Most MOOC learners in our study fell into Class 1 relief, which was characterized by a steady moderate level of this affect. Although relief is a positive affect, it reduces immediate motivation on a given academic task but preserves the motivating force in later learning (Pekrun et al., 2002; cited in Rowe et al., 2015). Class 1 relief profile could be related to the structure of MOOCs, in which learners completed learning at their own pace, whereas typical university courses have fixed schedules of instruction and for submission of assignments.

It is not surprising to see that most MOOC learners showed a fairly low level of negative affects: the vast majority of learners in an MOOC sought out the course of their own volition and thus are likely interested in the content and have already exerted some effort (McAuley et al., 2010). Although MOOCs bring individuals who are interested in learning (Liyanagunawardena et al., 2013), our findings revealed that some MOOC learners still exhibited worrisome negative affective profiles. Specifically, learners in the Class 2 boredom profile were characterized by increasing boredom until the end of the course, learners in the Class 2 guilt profile showed increasing moderate levels of guilt for the duration of the course, and the learners in Class 3 experienced high levels of anxiety. The profiles for boredom and guilt may represent learners whose initial interest and effort were unsustainable. The content may not have been as exciting as they had hoped upon registration or the workload may have been more than their effort could sustain resulting in a generalized feeling of guilt. Despite these affective profiles, the learners in these groups nonetheless, completed not only the course but also our questionnaires showing that they were able to work through these negative feelings. As for anxiety, Rowe et al. (2015) found that postsecondary instructors considered that “a certain level of anxiety or intensity was necessary for learning” (p. 8). In our findings, the majority of learners (76.9%) belonged to a low anxiety profile, while only less than 16% of learners showed a Class 2 pattern (i.e., moderate level of anxiety). The flexibility in the MOOCs might have helped reduce typical learners’ anxiety in achievement settings. This may explain a large percentage in Class 1 low anxiety profile. Moreover, because most learners are not enrolled in the MOOC for university credit the “stakes” are logically lower. In other words, MOOCs may be a learning environment that simply does not invoke anxiety for the majority of learners.

Predictors of Class Memberships

Massive open online courses are commonly conceptualized as courses being offered for free and open to everyone to register (e.g., Saadatdoost et al., 2015). This means that no formal requirement or prerequisites have to be met before enrolling in an MOOC (e.g., Gaebel, 2013). People in an MOOC are literally from anywhere on the planet, with any type of cultural and educational background. In addition, because of the structure of MOOCs learners might perceive some level of competence and be able to pursue the content in an autonomous fashion (McAuley et al., 2010). It is possible that different affective profiles were related to prior level of education, perceived competency and autonomy offered in the MOOC.

The higher the levels of education obtained, the more likely that MOOC learners in our study would belong to Class 1 boredom profile (low level), Class 1 relief profile (low level), and Class 1 anxiety profile (moderate level). This could possibly be due to study skills learned from completing formal education; these skills might include, but are not limited to, sustaining attention in an MOOCs despite challenging content and being flexible to use different types of learning strategies (e.g., in-depth inquiry vs problem solving). However, levels of education did not differentiate the class memberships for guilt.

When an individual perceives himself or herself as competent in the MOOC, it is more likely this person would be in the low guilt (Class 1) and low anxiety (Class 1) profiles compared with the profile with an increase sense of feeling guilt and the highly anxious profile, respectively. Given that competence serves as a foundation to perceived control (Pekrun and Perry, 2014), a high level of this basic psychological need could buffer the chance of getting into a negative activating affective profile.

With regard to autonomy, the higher the level it was, the more likely that an MOOC learner would belong to a low boredom profile, and a low guilt profile, and less likely he/she would be in a moderate relief profile. The relationship between autonomy support and boredom is consistent with previous empirical evidence (e.g., Tze et al., 2014). Similarly, when learners’ autonomous learning is supported, the more likely they will internalize the learning and put effort in a course (e.g., Niemiec and Ryan, 2009). Weiner (1985), cited in Pekrun et al. (2007), conceptualized that the feeling of guilt was aroused because of attributing one’s lack of effort in a failure situation. With more effort put on studying, the source of guilt could be reduced. When autonomy is high, the more likely an MOOC learner would be in a moderate relief profile, maintaining his/her motivation for subsequent learning.

Differences in Engagement

Massive open online courses offer a way for almost everyone to learn; however, learners enrolled in an MOOC may not engage similarly. The extent to which their levels of behavioral, cognitive and social engagement differ depends on their affective profiles.

Although relief is a deactivating affect, it preserves learners’ motivation to learn in subsequent stages, which were evident in their levels of engagement at the end of the MOOC. The low boredom profile and the low guilt profile showed more engagement in learning the course materials and in adhering to rules in the MOOC (e.g., completing quizzes on a regular basis) than the contrasting profiles (i.e., an inverted U shaped boredom profile and an increase guilt profile). These findings suggest the importance of reducing these negative affects among MOOC learners. With regard to anxiety, which is a complex affect, our results showed an interesting pattern on engagement. The low anxiety profile was behaviorally and cognitive more engaged than the moderate, yet fluctuating, anxiety profile. However, the highly anxious profile was socially more engaged than the low anxiety profile. As was discussed in Pekrun and Perry (2014), on one hand, anxiety could reduce intrinsic motivation to learn; on the other hand, it might foster external regulation. Similarly, based on self-determination theory (Ryan and Deci, 2000), Appleton et al. (2008) discussed that anxiety falls into the introjection process category, which involves behaving in manners according to external rules, expectations or reward (Deci et al., 1991). In addition, Yerkes–Dodson law (Reevy et al., 2010) also explains the different engagement pattern among low, moderate, and high anxiety profiles. With a very high and fluctuating anxiety profile, MOOC learners might feel overwhelmed by learning tasks, and hence, instead of cognitively engaging with the learning, they seek more social support from their peers to meet the course demands. With low anxiety profile, these MOOC learners might perceive that the learning was management, thereby they were cognitively and behaviorally more engaged.

Limitations and Directions for Future Research

There are some limitations that are worth noting and call for future investigation. First, we did not measure how often MOOC learners accessed the course materials. Although we had collected their usage data (e.g., the frequency of being online and participating into forum discussion), these data were too high inference for us to confidently understand how they made use of the materials. Future research should consider psychological- and theoretical-guided research with usage, such as time spending on review course materials and the depth of contents discussed in a forum, and how these objective student engagement measures relates to MOOC learners’ affects. In addition, our findings reflect responses from MOOC completers who not only completed all four surveys but also possibly tended to enjoy the course. The exclusion of MOOC learners who did not complete all four surveys potentially posted a threat to the validity and generalizability, and yet the focus of MOOC completers enabled us to look into the affective profiles over the entire course of MOOC, given that affective experiences might change in response to learning process. Hence, results should be interpreted from this perspective, and future research should consider exploring affect and engagement profiles of learners who do not complete an MOOC. Finally, despite the use of a longitudinal design, our findings reflect experiences in a single MOOC. Hence, future research should investigate the generalizability of our findings to other MOOCs. Given that the theoretical assertion of control value is much supported, we have confidence that the results could somehow be generalized.

Massive open online courses represent an exciting and new way to reach hundreds of thousands of learners who are genuinely interested in the course content for reasons much less performance based than in other achievement environments. Our results show that some Dino 101 learners still suffered from higher than desirable levels of boredom, anxiety, and guilt, and lower levels than desirable of relief. The level of competence and autonomy that MOOC learners perceived was related to these affective profiles. Thus, there is room for MOOC developers to refine their courses to maximize opportunities for competence and autonomy with the hope of lessening the experience of negative affects. The positive affective profile is also important because our results demonstrated that a positive affect is associated with increased engagement. The exception to this were that learners with high anxiety were more socially engaged, indicating MOOCs with blogs, messaging, or other ways of interacting may benefit anxious learners by fostering support. Overall, this study indicates that MOOCs may be appealing because they speak to people’s interest and provide an accessible avenue for individuals to pursue knowledge.

Ethics Statement

This study was carried out in accordance with the recommendations of the University of Alberta Research Ethics Board 2, which adheres to the Declaration of Helsinki. Informed consent was implied by the overt action of completing the questionnaire after reading the information letter. The protocol Pro00043399 was approved by the University of Alberta Research Ethics Board 2.

Author Contributions

VT conceptualized the study and wrote the manuscript. LD conceptualized the methodology and implementation. EB and LL involved in data collection and manuscript writing.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This research was supported by an award to the second author from the Digital Learning Pilots: Research and Development at the University of Alberta.

References

Appleton, J. J., Christenson, S. L., and Furlong, M. J. (2008). Student engagement with school: critical conceptual and methodological issues of the construct. Psychol. Sch. 45, 369–386. doi: 10.1002/pits.20303

CrossRef Full Text | Google Scholar

Betoret, F. D., and Artiga, A. G. (2011). The relationship among student basic need satisfaction, approaches to learning, reporting of avoidance strategies and achievement. Electron. J. Res. Educ. Psychol. 9, 463–496.

Google Scholar

Brennan, K. (2013). In connectivism, no one can hear you scream: a guide to understanding the MOOC novice. Hybrid Pedagogy. Available at: http://www.hybridpedagogy.com/journal/in-connectivism-no-one-can-hear-you-scream-a-guide-to-understanding-the-mooc-novice/

Google Scholar

Cassidy, D., Breakwell, N., and Baily, J. (2013). Keeping them clicking: promoting student engagement in MOOC design. Irel. J. Teach. Learn. Higher Educ. 6, 2–15.

Google Scholar

Chapell, M. S., Blanding, Z. B., Silverstein, M. E., Takahashi, M., Newman, R., Gubi, A., et al. (2005). Test anxiety and academic performance in undergraduate and graduate students. J. Educ. Psychol. 97, 268–274. doi:10.1037/0022-0663.97.2.268

CrossRef Full Text | Google Scholar

Clarke, T. (2013). The advance of the MOOCs (massive open online courses): The impending globalisation of business education? Education and Training 55, 403–413.

Google Scholar

Daniel, J. (2012). Making sense of MOOCs: musings in a maze of myth, paradox, and possibility. J. Interact. Media Educ. 2012, 18. doi:10.5334/2012-18

CrossRef Full Text | Google Scholar

Daniels, L. M., Adams, C., and McCaffrey, A. (2016). “Emotional and social engagement in a massive open online course: an examination of Dino 101,” in Emotions, Technology, and Learning, eds S. Y. Tettegah and M. P. McCreery (New York, NY: Elsevier), 25–41.

Google Scholar

Daniels, L. M., and Stupnisky, R. H. (2012). Not that different in theory: a discussion of the control-value theory of emotions in online and face-to-face learning environments. Internet Higher Educ. 15, 222–226. doi:10.1016/j.iheduc.2012.04.002

CrossRef Full Text | Google Scholar

Daniels, L. M., Stupnisky, R. H., Pekru, R., Haynes, T. L., Perry, R. P., and Newall, N. E. (2009). A longitudinal analysis of achievement goals: from affective antecedents to emotional effects and achievement outcomes. J. Educ. Psychol. 4, 948–963. doi:10.1037/a0016096

CrossRef Full Text | Google Scholar

Deci, E., and Ryan, R. (2012). “Self-determination theory,” in Handbook of Theories of Social Psychology, Vol. 1, eds P. Van Lange, A. Kruglanski, and E. Higgins (London: SAGE), 416–438.

Google Scholar

Dennis, M. (2012). The impact of MOOCs on higher education. Coll Univ 88, 24–30.

Google Scholar

Deci, E. L., Vallerand, R. J., Pelletier, L. G., and Ryan, R. M. (1991). Motivation and education: the self-determination perspective. Educ. Psychol. 26, 325–346. doi:10.1080/00461520.1991.9653137

CrossRef Full Text | Google Scholar

D’Mello, S. K., Lehman, B., and Person, N. (2010). Monitoring affect states during effortful problem solving activities. Int. J. Artif. Intell. Educ. 20, 361–389. doi:10.3233/JAI-2010-012

CrossRef Full Text | Google Scholar

Durksen, T. L., Chu, M.-W., Ahmad, Z. F., Radil, A. I., and Daniels, L. M. (2016). Motivation in a MOOC: a probabilistic analysis of online learners’ basic psychological needs. Soc. Psychol. Educ. 19, 241–260. doi:10.1007/s11218-015-9331-9

CrossRef Full Text | Google Scholar

Ferguson, R., and Clow, D. (2015). Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs). Paper Presented at the Fifth International Learning Analytics and Knowledge Conference, Poughkeepsie, NY. Available at: http://oro.open.ac.uk/42345/1/LAK15%20Paper.pdf

Google Scholar

Fredericks, J. A., Blumenfeld, P., Friedel, J., and Paris, A. (2005). “School engagement,” in What do Children Need to Flourish? Conceptualizing and Measuring Indicators of Positive Development, eds K. A. Moore and L. Lippman (New York, NY: Springer Science and Business Media), 305–321.

Google Scholar

Gaebel, M. (2013). MOOCs Massive Open Online Course. Available at: www.eua.be/Libraries/publication/EUA_Occasional_papers_MOOCs

Google Scholar

Gogol, K., Brunner, M., Goetz, T., Martin, R., Ugen, S., Fischbach, A., et al. (2014). ’My questionnaire is too long!’ The assessments of motivational-affective constructs with three-item and single-item measures. Contemp. Educ. Psychol. 39, 188–205. doi:10.1016/j.cedpsych.2014.04.002

CrossRef Full Text | Google Scholar

Grice, J. W., and Iwasaki, M. (2007). A truly multivariate approach to MANOVA. Appl. Multivar. Res. 12, 199–226. doi:10.22329/amr.v12i3.660

CrossRef Full Text | Google Scholar

Hew, K. F., and Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): motivation and challenges. Educ. Res. Rev. 12, 45–58. doi:10.1016/j.edurev.2014.05.001

CrossRef Full Text | Google Scholar

Huang, C. (2011). Achievement goals and achievement emotions: a meta-analysis. Educ. Psychol. Rev. 23, 359–288. doi:10.1007/s10648-011-9155-x

CrossRef Full Text | Google Scholar

Jung, T., and Wickrama, K. A. S. (2008). An introduction to latent class growth analysis and growth mixture modeling. Soc. Pers. Psychol. Compass 2, 302–317. doi:10.1111/j.1751-9004.2007.00054.x

CrossRef Full Text | Google Scholar

Kao, L. S., and Green, C. E. (2008). Analysis of variance: is there a difference in means and what does it mean? J. Surg. Res. 144, 158–170. doi:10.1016/j.jss.2007.02.053

PubMed Abstract | CrossRef Full Text | Google Scholar

Kizilcec, R. F., Piech, C., and Schneider, E. (2013). Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. Paper Presented at the Third International Conference on Learning Analytics and Knowledge, Leuven, Belgium. Available at: http://dl.acm.org/citation.cfm?id=2460330

Google Scholar

Klassen, R. M., Yerdelen, S., and Durksen, T. L. (2013). Measuring teacher engagement: development of the engaged teachers scale (TES). Front. Learn. Res. 1:33–52. doi:10.14786/flr.v1i2.44

CrossRef Full Text | Google Scholar

Knox, J. (2014). Digital culture clash: “massive” education in the e-learning and digital cultures MOOC. Distance Educ. 35, 164–177. doi:10.1080/01587919.2014.917704

CrossRef Full Text | Google Scholar

Kop, R., and Hill, A. (2008). Connectivism: Learning theory of the future or vestige of the past? Int. Rev. Res. Open Dist. Learn. 9. Available at: http://www.irrodl.org/index.php/irrodl/article/view/523/1103

Google Scholar

Leony, D., Muñoz-Merino, P. J., Ruipérez-Valiente, J. A., Pardo, A., and Kloos, C. D. (2015). Detection and evaluation of emotions in massive open online courses. J. Univ. Comput. Sci. 219, 638–655. doi:10.3217/jucs-021-05-0638

CrossRef Full Text | Google Scholar

Liyanagunawardena, T. R., Adams, A. A., and Willams, S. A. (2013). MOOCs: a systematic study of the published literature 2008–2012. Int. Rev. Res. Open Distance Learn. 14, 202–227. doi:10.19173/irrodl.v14i3.1455

CrossRef Full Text | Google Scholar

Lo, Y., Mendell, N. R., and Rubin, D. B. (2001). Testing the number of components in a normal mixture. Biometrika 88, 767–778. doi:10.1093/biomet/88.3.767

CrossRef Full Text | Google Scholar

Marsh, H. W., Lüdtke, O., Trautwein, U., and Morin, A. J. S. (2009). Classical latent profile analysis of academic self-concept dimensions: synergy of person- and variable-centered approaches to theoretical models of self-concept. Struct. Equation Model. Multidiscip. J. 16, 191–225. doi:10.1080/10705510902751010

CrossRef Full Text | Google Scholar

McAuley, A., Stewart, B., Siemens, G., and Cormier, D. (2010). The MOOC Model for Digital Practice. Available at: https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/MOOC_Final.pdf

Google Scholar

Nawrot, I., and Doucet, A. (2014). Building engagement for MOOC students – introducing support for time management on online learning platforms. Paper Presented at the 23rd International World Wide Web Conference, Seoul, South Korea. Available at: https://www.cs.helsinki.fi/u/doucet/papers/WebET2014.pdf

Google Scholar

Niemiec, C. P., and Ryan, R. M. (2009). Autonomy, competence, and relatedness in the classroom: applying self-determination theory to educational practices. Theor. Res. Educ. 7, 133–144. doi:10.1177/1477878509104318

CrossRef Full Text | Google Scholar

Noteborn, G. C. M., and García, G. E. (2016). “Turning MOOCs around: increasing undergraduate academic performance by reducing test-anxiety in a flipped classroom setting,” in Emotions, Technology, and Learning, eds S. Y. Tettegah and M. P. McCreery’s (New York, NY: Elsevier), 3–24.

Google Scholar

Nylund, K. L., Asparouhov, T., and Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Struct. Educ. Model. Multidiscip. J. 14, 535–569. doi:10.1080/10705510701575396

CrossRef Full Text | Google Scholar

Pekrun, R. (2006). The control-value theory of achievement emotions: assumptions, corollaries, and implications for educational research and practice. Educ. Psychol. Rev. 18, 315–341. doi:10.1007/s10648-006-9029-9

CrossRef Full Text | Google Scholar

Pekrun, R., Elliot, A. J., and Maier, M. A. (2006). Achievement goals and discrete achievement emotions: a theoretical model and prospective test. J. Educ. Psychol. 98, 583–597. doi:10.1037/0022-0663.98.3.583

CrossRef Full Text | Google Scholar

Pekrun, R., Elliot, A. J., and Maier, M. A. (2009). Achievement goals and achievement emotions: testing a model of their joint relations with academic performance. J. Educ. Psychol. 101, 115–135. doi:10.1037/a0013383

CrossRef Full Text | Google Scholar

Pekrun, R., Goetz, T., and Perry, R. P. (2005). Academic Emotions Questionnaire (AEQ). User’s Manual. Department of Psychology, University of Munich.

Google Scholar

Pekrun, R., Frenzel, A. C., Geotz, T., and Perry, R. P. (2007). “The control-value theory of achievement emotions: an integrative approach to emotions in education,” in Emotion in Education, eds P. A. Schutz and R. Pekrun (Amsterdam: Academic Press), 13–36.

Google Scholar

Pekrun, R., Goetz, T., Daniels, L. M., Stupnisky, R. H., and Perry, R. P. (2010). Boredom in achievement settings: Exploring control-value antecedents and performance outcomes of a neglected emotion. J. Educ. Psychol. 102, 531–549. doi:10.1037/a0019243

CrossRef Full Text | Google Scholar

Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., and Perry, R. P. (2011). Measuring emotions in students’ learning and performance: the achievement emotions questionnaire (AEQ). Contemp. Educ. Psychol. 36, 36–48. doi:10.1016/j.cedpsych.2010.10.002

CrossRef Full Text | Google Scholar

Pekrun, R., Goetz, T., Titz, W., and Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educ. Psychol. 37, 91–105. doi:10.1207/S15326985EP3702_4

CrossRef Full Text | Google Scholar

Pekrun, R., Lichtenfeld, S., Marsh, H. W., Murayama, K., and Goetz, T. (2017). Achievement emotions and academic performance: longitudinal models of reciprocal effects. Child Dev. 88, 1653–1670. doi:10.1111/cdev.12704

PubMed Abstract | CrossRef Full Text | Google Scholar

Pekrun, R., and Perry, R. P. (2014). “Control-value theory of achievement emotions,” in International Handbook of Emotions in Education, eds R. Pekrun and L. Linnenbrink-Garcia (New York: Routledge), 120–141.

Google Scholar

Pekrun, R., and Stephens, E. J. (2009). Goals, emotions, and emotion regulation: perspectives of the control-value theory. Hum. Dev. 52, 357–365. doi:10.1159/000242349

CrossRef Full Text | Google Scholar

Reevy, G. M., Ozer, Y. M., and Ito, Y. (2010). Encyclopedia of Emotion, Vol. 2. Santa Barbara, CA: Greenwood.

Google Scholar

Rowe, A. D., Fitness, J., and Wood, L. N. (2015). University student and lecturer perceptions of positive emotions in learning. Int. J. Qual. Stud. Educ. 28, 1–20. doi:10.1080/09518398.2013.847506

CrossRef Full Text | Google Scholar

Ryan, R. M., and Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78. doi:10.1037/0003-066X.55.1.68

PubMed Abstract | CrossRef Full Text | Google Scholar

Saadatdoost, R., Sim, A. T. H., Jafarkarimi, H., and Hee, J. M. (2015). Exploring MOOC from education and information systems perspectives: a short literature review. Educ. Rev. 67, 505–518. doi:10.1080/00131911.2015.1058748

CrossRef Full Text | Google Scholar

Shah, D. (2014). MOOCs in 2014: Breaking Down the Numbers. Available at: https://www.edsurge.com/news/2014-12-26-moocs-in-2014-breaking-down-the-numbers

Google Scholar

Siemens, G. (2012). What is the Theory That Underpins our MOOCs? Blog elearnspace.

Google Scholar

Stein, K. (2013). Penn GSE Study Shows MOOCs Have Relatively Few Active Users, With Only a Few Persisting to Course End. Available at: http://www.gse.upenn.edu/news/press-releases/penn-gse-study-shows-moocs-have-relatively-few-active-users-only-few-persisting-

Google Scholar

Tempelaar, D. T., Niculescu, A., Rienties, B., Gijselaers, W. H., and Giesbers, B. (2012). How achievement emotions impact students’ decisions for online learning, and what precedes those emotions. Internet Higher Educ. 15, 161–169. doi:10.1016/j.iheduc.2011.10.003

CrossRef Full Text | Google Scholar

Tofighi, D., and Enders, C. K. (2007). “Identifying the correct number of classes in growth mixture models,” in Mixture Models in Latent Variable Research, ed. G. R. Hancock (Greenwich, CT: Information Age), 317–341.

Google Scholar

Tong, E. M. W., Bishop, G. D., Enkelmann, H. C., Diong, S. M., Why, Y. P., Khader, M., et al. (2009). Emotion and appraisal profiles of the needs for competence and relatedness. Basic Appl. Soc. Psych. 31, 218–225. doi:10.1080/01973530903058326

CrossRef Full Text | Google Scholar

Tze, V. M. C., Daniels, L. M., and Klassen, R. M. (2016). Evaluating the relationship between boredom and achievement outcomes: a meta-analysis. Educ. Psychol. Rev. 28, 119–144. doi:10.1007/s10648-015-9301-y

CrossRef Full Text | Google Scholar

Tze, V. M. C., Klassen, R. M., and Daniels, L. M. (2014). Patterns of boredom and its relationship with perceived autonomy support and engagement. Contemp. Educ. Psychol. 39, 175–187. doi:10.1016/j.cedpsych.2014.05.001

CrossRef Full Text | Google Scholar

Udacity. (2011–2017). Artificial Intelligence Nanodegree. Available at: https://www.udacity.com/course/artificial-intelligence-nanodegree--nd889

Google Scholar

Villavicencio, F. T., and Bernardo, A. B. I. (2013). Negative emotions moderate the relationship between self-efficacy and achievement of Filipino students. Psychol. Stud. 58, 225–232. doi:10.1007/s12646-013-0193-y

CrossRef Full Text | Google Scholar

Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychol. Rev. 92, 548–573. doi:10.1037/0033-295X.92.4.548

CrossRef Full Text | Google Scholar

Zhao, Y., and Karypic, G. (2004). Empirical and theoretical comparisons of selected criterion functions for document clustering. Mach. Learn. 55, 311–331. doi:10.1023/B:MACH.0000027785.44527.d6

CrossRef Full Text | Google Scholar