A Framework for Applying the Learning Sciences to MOOC Design

Evidence-based practices grounded in the learning sciences provide an opportunity for improved learning experiences in traditional in-person, as well as in hybrid and online environments. We advocate specifically that large-scale, online learning experiences, such as Massive Open Online Courses (MOOCs) benefit from applications of the learning sciences. To that end, we present a description of how and why we use specific learning science practices in a biochemistry MOOC with the intention to contribute to the discussion about the quality of online learning experiences and lower the barrier for other practitioners seeking a framework for implementing evidence-based course design. We believe that the application of the learning sciences makes online learning experiences more rigorous and effective, and practitioners should optimize the use of these strategies through clever tests in specific contexts.


INTRODUCTION
The quality of learning experiences across the 13,500 Massive Open Online Courses (MOOCs) offered since 2012 (Shah, 2019) is heterogeneous. There are calls to assess the design of MOOCs, establish criteria for development, and standardize quality assurance measures (Alario-Hoyos et al., 2014a,b;Kopp and Lackner, 2014;Yousef et al., 2014;Doherty et al., 2015;Lee et al., 2016). Best practices for teaching and learning rooted in the learning sciences advance scientific teaching methods through the adoption of active learning (Handelsman et al., 2004;Freeman et al., 2014) and promote efficient and effective ways to study (Miyatsu et al., 2018). Digital learning designers translate many of these best practices to online learning environments. Our aim is to document this translation from the perspective of MOOC developers by presenting a framework for applying evidence-based strategies in a biochemistry MOOC and advocating for testing the effectiveness of instructional practices in specific online contexts.
As practitioners concerned with maximizing learning gains in an understudied, yet widely used non-traditional learning environment (Veletsianos and Shepherdson, 2016), we are agnostic to adhering to strategies based in one conceptual framework of educational research over another. Digital learning designers should have all possible tools at their disposal for creating engaging and effective learning experiences. To this end, we take a systematic, eclectic, instructional design approach (Yanchar and Gabbitas, 2011;Honebein and Sink, 2012) to learning engineering, where we draw upon practices rooted in behaviorism, cognitivism, social learning, and constructivism to address the challenges of online learning at scale.
Here we document practices supported by the learning sciences that we readily implement in an online learning environment to contribute to conversations in the community of practice about evidence-based approaches to MOOC design. We hope these conversations lower the barrier for other developers of online and hybrid experiences to apply these methods. We found the following practices both robust from a theoretical standpoint, and practical in terms of application in our course design. We outline the evidence-based practices applied to course assets organized by function (pre-instruction, multimedia content, formative assessment, supporting resources, and summative assessment). However, it is worth noting that many of the practices overlap in their usage. This article is an exposition on why and how to use the learning sciences in a specific context as a model for others while highlighting a few strategies, and not an exhaustive review of these practices, nor a formal assessment of their efficacy.
FIGURE 1 | A mapping of evidence-based practices stemming from the learning sciences (gray outer nodes) to course assets (colorful inner nodes). The course assets are color-coded by function: green = pre-instruction, yellow = multimedia, purple = supporting resources, blue = formative assessments, and orange = summative assessment. To explore this figure as an interactive visualization, please visit https://web.mit.edu/mitxbio/learning_sciences.html.

COURSE OVERVIEW
The biochemistry MOOC has a hierarchical structure composed of mainly eight different topic units (like chapters in a book), each broken down into one or two learning sequences (analogous to sections of each chapter). The learning sequences contain a series of course assets (Figure 1, colorful inner nodes) informed by evidence-based principles (Figure 1, gray outer nodes). These course assets are designed to prime the learner for the material (pre-instructional strategies; Figure 1, green nodes), convey foundational information (multimedia content; Figure 1, yellow node), build understanding (formative assessments; Figure 1, blue nodes), and support learning (supporting resources; Figure 1, purple nodes). In addition to topic units and other supplemental material, the course ends with a comprehensive exam (summative assessment; Figure 1, orange node).

Pre-Instruction
To prime our MOOC learners for the introduction of new material in each learning sequence, we employ two pre-instructional techniques, learning objectives and advance organizers (Figure 1, green nodes). Positioned at the beginning of the learning sequences, we outline one or two big picture goals for each sequence as well as several measurable objectives the learner should meet by engaging with the sequence material. By outlining the goals and objectives that we used to develop and align the assessments during course development, we clearly communicate our expectations for the learner, and promote transparency in the course (Hartley and Davies, 1976). We know from a survey of our hybrid learning experiences that students find these goals and objectives useful for studying, even when not explicitly directed to use them in this way. We also use a type of advance organizer (Weisberg, 1970;Hartley and Davies, 1976;Meng and Patty, 1991;Bui and McDaniel, 2015) in the form of image handouts. By presenting key images ahead of instruction, the learners have an opportunity to preview the material, scaffold their note-taking, and make deeper connections during the video segments. These handouts contain a series of images that appear in the sequences, each coupled with an openended, yet focused question (Wylie and Chi, 2014) designed to make connections between what each image represents and the learning sequence topic (Supplementary Figure 1). We do not formally assess the coupled questions, but rather prompt selfexplanation. Learners answer the questions in their own words, revealing their own thought process and understanding, which promotes metacognitive processes (Chi et al., 1994;Aleven and Koedinger, 2002;Ainsworth and Th Loizou, 2003).

Multimedia Content
To connect learners' prior knowledge to more advanced concepts in biochemistry, we offer a series of video and images, which we designed to maximize engagement and minimize extraneous cognitive load (Sweller, 1994(Sweller, , 2005Sweller et al., 2019) (Figure 1, yellow node). We filmed a live classroom, where the professor writes on a chalkboard to discuss the material unscripted. Students may prefer the professor writing on a board over presenting slides because the method is more interactive, the pace of the presentation is more accessible, and the information is streamlined (Armour et al., 2016). There is also evidence that unscripted lectures may be more engaging to learners than scripted deliveries (Thornton et al., 2017). This method of multimedia content delivery meets the principles of multimedia learning that promote generative processing (personalization, voice, and embodiment) as well as some that maintain essential processing (modality and segmenting) (Clark and Mayer, 2016;Mayer, 2017). Often embedded within the videos are brief animations of the dynamic, process-oriented concepts and complex structures overlaid on the active discussion in class (Supplementary Video 1). These animations are consistent in style and color to familiarize learners with the representation of specific concepts (pre-training) (Mayer et al., 2002). We also took care to adhere to the remaining multimedia learning principles (coherence, signaling, redundancy, and contiguity) while maintaining scientific accuracy, to promote a deeper understanding of the core concepts represented and maintain a manageable cognitive load for learners (Sweller, 2005;Clark and Mayer, 2016;Mayer, 2017).

Formative Assessments
We include over 600 formative assessment questions to guide learners through the process of constructing a foundational understanding of biochemistry and transferring that understanding to novel contexts (Figure 1, blue nodes). All the assessments are graded automatically, including multiple choice responses, checkbox, and numerical and text inputs. To gauge understanding of the new material introduced in each video segment, we intersperse test yourself questions (TYs) between videos. We include concrete examples in TYs to help learners generalize abstract ideas and transfer knowledge to different contexts (Paivio et al., 1994;Weinstein et al., 2018).
These concrete examples can also serve a dual purpose to personally connect the learner to the material. Our examples often have relevance to current research, medical applications, or everyday life, which can help promote intrinsic motivation to learn (Ambrose et al., 2010). One such example is a TY in the pH and buffers unit, where we ask learners to calculate the pH of a homemade buttermilk substitute given the volume of lemon juice and milk, the amount of citric acid in lemon juice, and the dissociation constant of citric acid. This question goes beyond a simple calculation and engages the learner in how biochemistry applies to daily living.
Engaging learners is a challenge for most formative assessments, so we constructed problem sets (PSs) at the end of every unit, to generate interest, as well as prompt learners to think deeply about what they learned and apply their knowledge to new contexts. We take inspiration from narrative-centered learning in gaming (Mott et al., 1999) to leverage the great appeal of storytelling in education. The PS questions are connected by an ongoing story about taking biochemistry and engaging in research as an undergraduate, where we inject the learners as characters in the narrative. The assessments that compose the PSs reference objectives covered throughout the associated learning sequence(s), and as such represent both spaced retrieval and interleaving practices (Karpicke and Roediger, 2007;Roediger and Butler, 2011;Birnbaum et al., 2013). Retrieval practices such as these, and the resulting testing effect, are useful techniques for studying and retaining information on longer timescales (Roediger and Karpicke, 2006;Rowland, 2014). The PS questions also contain scaffolding to break complex questions into component parts, offering opportunities for the learners to adapt their approaches based on the immediate feedback given (Reiser, 2004). An example of scaffolding in the course is where we task the learners with a series of questions about a mathematical description of cooperative binding. First, we ask learners to identify the correct mathematical transformations of their raw data to set up the calculation. Then the learners need to select the correct mathematical expression to fit the data, and perform calculations to generate a model from the transformed data and mathematical expression selected earlier. Finally, learners must interpret the output of the model by explaining the biochemical logic of their results. Learners have less confusion, indicated by fewer discussion forum posts about challenging assessments, when we break down assessments into stepwise processes.
To give feedback to the learners on whether they answered a formative assessment correctly in real time, and provide an opportunity to reassess and adapt their strategy to answer again, every formative assessment in the MOOC has immediate grading for correctness. Moreover, since previous work demonstrates the benefit of formative feedback (Bangert-Drowns and Morgan, 1991; Moreno, 2004;Nicol and Macfarlane-Dick, 2006;Hattie and Timperley, 2007;Shute, 2008), we offer detailed and specific feedback that clarifies attributes of the question target, the larger topic at hand, or the relationships between correct and incorrect responses (Shute, 2008). In addition to learning more about the correct answer, or why other options are incorrect, we present the process to arrive at the correct solution through a worked example for some question types. Worked examples guide the learner step-by-step to the solution from an expert's point of view, with the intention of facilitating the transfer of knowledge and skills to similar questions (Atkinson et al., 2000).

Supporting Resources
We include a number of resources for our learners that are meant to supplement their learning, although they are not required or count toward advancing progress (Figure 1, purple nodes). There is a separate section of the MOOC for study resources, which includes text, graphics, videos, and links that detail specific concepts, skills, and techniques that the learners should have as prerequisite knowledge. These study resources are linked explicitly in the sections of the course in which their reference could be useful. For example, in the enzyme catalysis learning sequence, we link relevant TYs to the organic chemistry study resources for a refresher on functional groups and reaction terminology. By embedding references to material that can function as a review, we are attempting to activate learner prior knowledge, which is a foundational step in facilitating lasting learning (Ambrose et al., 2010).
We also include optional molecular viewer activities. We use a molecular visualization software that structural biochemists employ in their research to help learners view and manipulate protein structural models from deposited data. Thus, these molecular viewer activities are an example of authentic learning, where learners apply their newly learned biochemistry skills in a context that is shared with professional scientists (Herrington et al., 2004;Herrington and Kervin, 2007;Lombardi, 2007;Oliver et al., 2007). The connection of educational materials to reallife applications is related to expert-level thinking (Semsar et al., 2011), and may help adult learners see themselves as belonging in scientific fields (Rossiter, 2007). Although these activities exploring protein structure are consequential for learning, the assessments associated with them do not influence the learners' grades. Low-stakes tasks such as these facilitate trying something intellectually risky and relieve some pressure to perform on assessments, which may also help intrinsic motivation and selfesteem (Nicol and Macfarlane-Dick, 2006). In this context, the optional molecular viewer assignments reduce the pressure to perform while engaging with a potentially unfamiliar tool.
At the bottom of each page in the course (excluding summative assessments), we include the opportunity to engage in the discussion forum. Discussion forum posts offer a way for learners to personally engage with each other and the instructors over the course material. Learners are encouraged to introduce themselves and their motivations for taking the MOOC, which we hope connects the material to personal values they hold. By articulating the value that the MOOC offers for them, learners may feel more motivated to sustain participation (Canning et al., 2018). The discussion forum is also a place where learners can have informal conversations about the material, ask for help from staff or peers, and answer each others' questions and comments. Engaging with other learners and staff, and helping each other build an understanding of course material can contribute to academic achievement and satisfaction (Jung et al., 2002;Kellogg et al., 2014).

Summative Assessment
The final, summative assessment of the biochemistry MOOC is a competency exam (CE) (Figure 1, orange node). We designed the CE to test the majority of learning objectives introduced in the course by engaging in scientific thinking, synthesizing concepts, and transferring knowledge to new contexts. We use the hierarchy of cognitive processing outlined by Bloom's taxonomy (Krathwohl, 2002;Krathwohl and Anderson, 2009) to guide the creation of all assessments. According to the taxonomy, many CE questions require application, analysis, and evaluation, similar to PS questions. Both CE and PS questions are more cognitively demanding to answer than TYs, which require more recall.

DISCUSSION
We used a systematic, eclectic, instructional design approach to drive our design decisions in a biochemistry MOOC. Largely this approach follows from our unique perspectives as biology PhDs with pedagogical experience. Our roles in course development involve learning engineering (Wilcox et al., 2016), which lays at the intersections of applying expertise in the subject matter, educational technology, instructional and graphic design, educational research, data analytics, teaching, and project management. Although familiar with the learning science literature, we are free from the constraints of specializing in a specific area of educational research set by tenure track or grants. This freedom allows us to maintain a practitioners' point of view, draw inspiration from the different frameworks of how learning works, and collectively deploy an amalgamation of strategies that enhance the design of our learning experiences.
As scientists, we value evidence-based practices, and the empirical approach the learning sciences offers. As developers of digital learning experiences, we understand that there are orchestration constraints in MOOC development that present challenges to implementing best practices for teaching and learning outlined by the literature. There is a great need to test educational design decisions in context and assess the relevant variables in successful implementation (Moir, 2018). These evaluations of design should be both formal, as randomized control trials, which is the standard for testing the effects of intervention in education (US Department of Education; Institute of Education Sciences; National Center for Education Evaluation Regional Assistance, 2003), and also more informal through the iterative revisions necessary to keep MOOCs current and rigorous. Testing in specific contexts is needed furthermore because there are differences in practice across institutions and/or courses in what best promotes engagement and learning. For example, the relationship between video length and learner engagement is a popular topic because of the heavy reliance on multimedia in many MOOCs. In one study, researchers recommend that videos should be <6 min to maximize engagement (Guo et al., 2014), however, our own research indicates that video length is not a significant determinant in engagement in another one of our biology MOOCs (Thornton et al., 2017). This exemplifies the need to test different strategies, collect learner data, and make evidence-based design decisions informed by implementation research that are relevant to each course.
Application of the learning sciences to hybrid course and MOOC design provides a strong foundation of evidence-based practices that one can optimize for different online learning experiences. Making design decisions grounded in scientific evidence is a crucial first step. Equally important is the dissemination of these decision-making processes and subsequent evaluation of their implementation. By documenting the process of incorporating and testing applications of the learning sciences, we collectively can contribute to enriching the community of practice for digital learning designers while providing a more robust experience for learners. We see this perspective as a step forward to incite conversations and actions around evidence-based design efforts in online educational environments.

AUTHOR CONTRIBUTIONS
DG and MW contributed to the conception and scope of the perspective. DG wrote the initial draft of the manuscript, and both authors contributed to manuscript revision, read and approved the submitted version.

SUPPLEMENTARY MATERIAL
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc. 2020.500481/full#supplementary-material Supplementary Video 1 | A clip of a video segment detailing the steps in calcium ion transportation, where we cut back and forth between the professor explaining each step to a live classroom and an animation of this dynamic process added in post-production.