Exploring Aspects of Open Educational Resources Through OER-Enabled Pedagogy

Introduction

For almost two decades, Open Educational Resources (OER) have been playing a role in the educational scene while promoting the expansion of quality education (Mulder, 2013). OER are learning materials, including textbooks, that are openly licensed and that permit no-cost access, use, adaptation, and redistribution with no or limited restrictions (Hewlett Foundation, 2020). Using OER can harness the 5Rs permissions to enable flexible and creative applications of these resources in instruction (Wiley, 2014). The 5Rs permissions, which are the right to retain, reuse, revise, remix, and redistribute the resources, are activities that can be applied to an OER as allowed by specific copyright licenses (Wiley and Hilton, 2018). The 5Rs support open pedagogical approaches to instruction sometimes referred to as OER-enabled pedagogy. Wiley and Hilton (2018) define OER-enabled pedagogy as “the set of teaching and learning practices that are only possible or practical in the context of the 5R permissions which are characteristic of OER” (p. 135).

This study compares constructs important in current OER research between students in an undergraduate nutrition course where one semester uses an OER and another semester uses the OER and applies OER-enabled pedagogy within instruction. It explored various aspects of the student experience within this context.

Literature Review

Considerable research has been conducted examining the factors affecting the student experience using OER materials as well as student progress toward graduation (Hilton, 2016; Hilton et al., 2016a). Many questions about the impact of OER adoption have been highlighted by the Open Education Group (Hilton et al., 2016b) and have been used to create a framework specific for OER investigation. This framework, identified as the COUP Framework, addresses principal aspects of education that can be impacted by the use of OER and focuses on the issues of “cost,” “outcomes,” “use,” and “perceptions” (Bliss et al., 2013b; Hilton et al., 2016a).

A much-researched area of OER adoption focuses on the cost of textbooks (Hilton et al., 2014; Lashley et al., 2017). One of the salient factors driving the OER movement is that textbook prices have soared over the years. The 2016 Student Public Interest Research Groups (PIRGs) report (Senack and Donoghue, 2016) indicated that there has been an increase of 73% in textbook costs in the previous decade alone, which is about four times the rate of general inflation. The 2018 Student PIRGS report indicated that, after exploring numerous options for reducing textbook prices, the author recommended that OER present the best solution to reduce textbook costs (Vitez, 2018).

Another factor of interest in OER research is that of student output, such as grades (Pawlyshyn et al., 2013; Winitzky-Stephens and Pickavance, 2017). Research in this area has examined how students, who are using traditional texts, have compared with students using open textbooks in terms of student performance measures such as overall course grades and final examination scores (Bowen et al., 2012; Hilton et al., 2016a). A number of studies have found that there was no significant difference in grades between students in classes that used a traditional textbook and those in classes using OER (Allen et al., 2015; Choi and Carpenter, 2017; Croteau, 2017; Hendricks et al., 2017; Fialkowski et al., 2020). Research also shows that, when faced with the high cost of textbooks, a student may choose to avoid buying a book because of budgetary restrictions, even when knowing it might impact course grades (Prasad and Usagawa, 2014). Course completion rates may be affected by textbook costs (Hilton et al., 2016a). Students sometimes drop or withdraw from courses after realizing the need to purchase textbooks (Gale, 2016). However, a study by Fischer et al. (2015) study found that when comparing completion rates between groups using a traditional textbook and OER treatment groups, there was no significant completion rate difference in most of the courses under review. It has also been hypothesized that the use of no-cost open textbooks might lighten a student's financial situation enough to allow them an increased credit load, which would in turn allow faster progress to graduation (Fischer et al., 2015).

Research has also examined student use of OER materials in terms of the time spent with the materials as well as the amount of the material that is actually used during a course (Lindshield and Adhikari, 2013; Hendricks et al., 2017). Hendricks et al. (2017) found that the patterns of use of the textbook between a class using a traditional textbook and one using an OER were very similar. In a report by the California Open Educational Resources Council (2016), similar results were found in a survey of 351 students in higher education. Students indicated that they used the OER textbooks they had been assigned at about the same rates as they used traditional textbooks in other classes. However, a recent study of an undergraduate nutrition class that compared a traditional textbook with an OER textbook indicated that students rated their use of the OER textbook significantly more (Fialkowski et al., 2020).

Finally, much research has investigated student perceptions of the overall quality of OER when OER are utilized (Hilton et al., 2013; Everard and Pierre, 2014; Cooney, 2017). In a study published in 2017, Cooney examined the impact of an OER introduced in three sections of a New York City College of Technology Health Psychology course regarding student perceptions of the OER. Students reported that course readings were equal to or better than those found in the traditional textbooks (2017). Jhangiani and Jhangiani (2017) reported on a study of post-secondary students in British Columbia. This study found that 96% of students indicated that the open textbooks were at or above average in quality compared to traditional textbooks.

Though not included in the COUP framework, the construct of engagement within a course could also affect student success (Webber et al., 2013). Student engagement has been defined as an investment, commitment, participation or an effortful involvement in learning (Henrie et al., 2015). The intent of engagement is to create an environment that enhances student learning and therefore performance (Trowler and Trowler, 2010). Few OER studies have included the construct of engagement as part of the research focus. The recent study of an undergraduate nutrition class included “engagement” as part of its methodological framework (Fialkowski et al., 2020), while the 2017 study at New York City College of Technology, Cooney (2017) relayed that students reported an increase in satisfaction with their learning experience and in their engagement with course lessons when using an OER rather than a traditional textbook in their Health Psychology course. Though “engagement” was not identified as a focus of study, several researchers have mentioned that students reported an increase in satisfaction with learning and engagement in courses when using OER (Rowell, 2015; Cooney, 2017). Only one other study specifically focused on levels of engagement when students used OER, though the domains within the construct were not clearly defined, and there seemed to be overlap in definition with “perceptions” and “use” (Ikahihifo et al., 2017).

It is interesting to investigate potential links between the construct of engagement and OER-enabled pedagogy (OP). Jhangiani and Biswas-Diener (2017) indicated that open educational practices encompass a range of practices. These include the creation, adaptation, and adoption of OER, but also encompass OER-enabled pedagogy, open course development, open science, and open access as well. Some researchers are beginning to explore how shifts in pedagogy, including student involvement in resource development and contribution to OER, might impact student learning outcomes (DeRosa and Robinson, 2017). Numerous studies have addressed the benefits realized by students when OER are adopted in their courses (Bliss et al., 2013a; Hilton et al., 2013, 2014; Fischer et al., 2015); however, discussion has been limited around the impact of OER-enabled pedagogy on student learning outcomes when OER and OP, afforded by the 5Rs of OER, are harnessed (DeRosa and Robinson, 2017).

This current study has expanded the original COUP methodology developed by the Open Education Group (Hilton et al., 2016b) to include the construct of “engagement.” Under this expanded umbrella, it examined more traditional aspects of OER (cost, output, use, and perception) and added the construct of “engagement” to the framework. In addition, it explored the impact that OP might provide on the COUPE constructs in an undergraduate class. It examined differences between undergraduate students using an OER textbook and students using the same OER textbook as well as engaged in OER-enabled pedagogy.

Context

As of 2018, almost 18,000 students were enrolled at the University of Hawai'i at Mānoa (UHM), with just under 13,000 registered undergraduates. This study reports on differences found between two semesters of an introductory nutrition class offered at UHM. This course is a large, service course that meets general education requirements and is taken mostly by students who are undergraduates and not nutrition majors. The class is organized around lectures, aided by slide presentations. Due to the size of the class, discussion is often limited. Students are able to access the complete course materials through the online course management system. Approximately 800 students register for the course annually. It typically offers two face-to-face sections as well as one online section each semester. The face-to face sections for the Spring 2018 and the Fall 2018 semesters were selected for this study, one section of the course for the Spring semester and two sections for the Fall semester. The same instructor taught both semesters.

The focus of the research was on the differences in terms of outcomes (student final grade, withdrawal and drop rates), use (use of the OER text), and perception of the material, as well as engagement within the class. Because the construct of cost was not an issue in this context—with the textbook offered at no cost for both semesters—it was not a dependent variable. However, cost plays an important role in students' perceptions in OER research (Ikahihifo et al., 2017); therefore, data on cost were collected for discussion. The course was selected because the faculty responsible for the course had recently participated in a campus wide OER initiative grant, which awarded monetary support for the development of an OER. The OER textbook for this course was developed over several semesters and was then published using the Pressbooks platform, available online or in a downloadable format (Fialkowski et al., 2018). One instructor agreed to participate in this study by applying an OER-enabled pedagogical approach to certain aspects of instruction. Prior to this study, in the Fall 2017 semester, a commercial textbook had been used in this course. The OER textbook for the course was introduced during the Spring 2018 semester (control group; identified as OER in this study) and then used in the subsequent Fall 2018 semester (treatment group; identified as OER+OP in this study) while adding an OER-enabled pedagogical approach.

The OER-enabled pedagogical approach took the form of a major assignment. One of the instructors had, for some time, been interested in addressing a health issue not covered in traditional nutrition textbooks. This was a focus on the concept of “health at every size,” which focuses on understanding size diversity and that each person's body weight is influenced by a genetic inheritance in bone structure, body size, shape, and weight difference and that health can be maintained regardless of differences in size (Bacon et al., 2005). For the treatment sections of the course, the instructor incorporated a research unit as part of the overall course assignments. The assignment focused on conducting research on some aspect of this topic so that scholarly articles would help to provide a foundation for the future development of this unit. Student findings would be incorporated into the OER textbook. Students were introduced to the concept of nutrition research by a librarian liaising with the instructor. After the introduction to nutrition research, students were then assigned to research a topic of their interest within the theme of “health at every size,” exploring at least five resources. They were to summarize one resource that would later be incorporated into the resource section of the OER textbook. This project represented one-quarter of assignment grades for the course.

Materials and Methods

A sequential explanatory mixed method design was used for this study. The initial quantitative phase used a quasi-experimental approach with data gathered from the institution and from an online student survey. Phase two used an interpretive approach collecting data via interviews to provide a more in-depth understanding of the research concepts. Two research questions guided this study:

Research Question #1. What are the differences in outcomes, use, perception, and engagement between undergraduate students who use an OER textbook and students who use an OER textbook and engage in OER-enabled pedagogy?

Research Question#2. What insights can be learned from students who use OER and OER-enabled pedagogy in an undergraduate course regarding cost, outcomes, use, perception, and engagement?

The first research question was addressed by the initial quantitative phase of the study, while the second research question was addressed through data collected in the qualitative phase of the study. Both research questions were also guided by the COUP framework (Bliss et al., 2013b; Hilton et al., 2016a) with the addition of “engagement” as a construct.

Data Collection

Following an exempt approval from the Institutional Review Board, data to address the questions were collected from three sources, with written informed consent from participants.

Institutional Data

Institutional data were examined to determine the total number of students enrolled in each of the course sections for both semesters (see Table 1). This source also provided data on drop and withdrawal rates as well as final grades for all sections.

TABLE 1

Table 1. Enrollment, course completion, response rate for the survey and number of student interview participants in an introductory nutrition course by semester.

Survey

Data from students were collected through an online survey administered anonymously during the last quarter of each semester. The survey invitation was sent through the course management system to all enrolled students who could then choose to click on the link to participate. The online survey explored student perceptions, use, and engagement with the OER textbook, containing 24 closed and open-ended questions that provided both quantitative and qualitative data. Instructions were provided at the end of the survey for students to receive bonus points for participation.

Interview

In this phase of the study, qualitative data were gathered from student interviews conducted at the end of each semester. The interview phase built directly upon the results from the initial, quantitative survey phase in order to explore the COUPE constructs more deeply. Students were randomly selected from each section and sent an email invitation to be interviewed. The face-to-face interview consisted of 20 open-ended questions, including questions about the assignments. Students were provided gift cards for their participation.

Data Analysis

An analysis comparing student withdrawal and drop rates and the final grade (a grade of C or better) is referred to by OER researchers as the Course Throughput Rate (CTR) and has been applied to various OER research studies (Fischer et al., 2015; Hilton et al., 2016a,b). The CTR was analyzed using a two-sample z-test of differences in proportions.

Three questions on the online survey asked about the use of the OER: the frequency of use throughout the semester; the amount of material read over the length of the semester; and how the OER was actually used (such as a supplement to the course, etc.). In addition, the survey explored student perceptions about the OER textbook. Specifically, there were three questions that focused on this construct: one pertained to the match between the textbook content and the learning objectives for the course; a second asked about the overall quality of the textbook; and the final question asked for a rating of the perceived ease of textbook use. Within the construct of “engagement” for this research, four domains were investigated: interest (in the class); challenge (challenged thinking in a positive way); understanding (of the course concepts); and participation.

An independent samples t-test was used to analyze responses to individual questions in each construct and then to analyze responses for the composite constructs relating to “use,” “perception,” and “engagement.” Descriptive statistics were used in the analysis of some of the multiple-choice and Likert-style questions on the survey. Specific questions about course assignments, which included the OP assignment in the Fall semester, were included in the student interviews. Open-ended questions on the survey collected qualitative data on student perceptions of positive and negative features of the OER. This data, as well as the responses from the interviews, were analyzed by the author (BT) through thematic coding. Passages in the text were identified and coded by idea, which was a way of categorizing or indexing the text thematically to establish a framework of concepts. Braun and Clarke (2008) relay that this type of thematic analysis is a very flexible and useful research tool that can be applied in order to provide rich, detailed, and complex data.

Qualitative data for the OER semester and the OER+OP semester were analyzed separately to compare themes and to ascertain if themes apart from the COUPE emerged. Data were organized initially by the five main constructs representing the COUPE framework. Themes and sub-themes emerged from this framework.

Results

Respondent Demographics

The OER semester included 127 students enrolled in one section of the course, of which 113 completed an online survey, and nine participated in interviews. There were 291 students initially enrolled in two sections for the OER+OP semester, which were taught by the same instructor. A total of 204 participants completed the survey for the OER+OP semester, with 11 students participating in follow-up interviews (see Table 1). Ninety-seven percent of the students completed the survey for the OER semester, while 82% completed it for the OER+OP semester.

Student demographic profiles for each semester were very similar (see Table 2). The Spring semester, using OER only with no OP activity, will be referred to as “OER,” and the Fall semester using OER and the OER-enabled pedagogy will be referred to as “OER+OP.” For each semester, certain sections of the course were identified for the study; therefore, data do not represent the complete population enrolled in all sections of the course.

TABLE 2

Table 2. Student participant characteristics in an introductory nutrition course over two semesters.

Of the student survey respondents for both semesters, freshmen made up the majority with almost 70% of the students taking 13 or more credit hours. Most of the students were female with ~96% ranging from ages 19 to 25 (see Table 2). Although the data are not presented in Table 2, survey respondents represented a wide array of ethnic backgrounds. UHM is one of the most ethnically diverse universities in the United States (Chronicle of Higher Education, 2018), and the students enrolled were reflective of that diversity.

Quantitative Findings

Quantitative data reported on the cost, outcomes, usage, perceptions, and engagement as outlined in the following sections.

Cost

Though cost was not a dependent variable in this research, as an OER was used for both semesters, it is worth noting that with the advent of the Spring 2018 semester, when the OER replaced a traditional textbook, there was a significant potential cost savings to students. The commercial textbook that had been traditionally used was sold at the campus bookstore for $130 during the Fall 2017 semester. The availability of an OER for this course, therefore, represented a potential annual savings for students of ~$104,000.

Outcomes

Student outcome was measured using the CTR comparing the semester using the OER and the semester when the OER was used with the addition of OP. This measure combines the effect of the drop rate, with the withdrawal rate, and the final grade to indicate the percentage of students who were present on the first day of class with those who completed the course with a final grade of C or better (Hilton et al., 2016b). The course throughput rates are indicated in Table 3. The z-score was calculated using the proportions and the enrollment numbers from the first day of class. The value of z for this calculation is 1.3118. The value of p is 0.1902, with the results indicating no significant difference at p < 0.05.

TABLE 3

Table 3. Course throughput rate for OER and OER+OP semesters with sample proportions.

Use

Responses on the survey indicated similar patterns between the two groups as can be seen in Table 4. One slight difference was while a small number of the OER+OP students reported daily use of the OER, while less than one percent of the OER-only group reported daily use.

TABLE 4

Table 4. Survey results for questions measuring use of OER for both OER and OER+OP semesters.

Patterns for the approximate amount of the textbook that was read were much more similar. Approximately 10% of students over both semesters responded that they read “none” or “almost none” of the textbook. Most students (OER=75.2% and OER+OP=72%) reported reading a little bit or about half of the textbook.

A t-test of independent samples was run for each of the questions comprising the “use” construct (see Table 5). Results indicated no significant difference. Data indicated that the majority of students in both classes used the OER as a supplemental resource. There also was no significant difference in the overall construct of “use” between the OER class (M = 2.92, SD = 0.80) and the OER+OP class (M = 2.92, SD = 0.81); t₍₃₁₅₎ = −0.1, p = 0.82.

TABLE 5

Table 5. Independent samples T-test comparing use of OER for OER and OER+OP semesters.

When asked on the survey how the students actually used the textbook, over 90% of students (OER = 91.2% and OER+OP = 92.2%) responded that they mostly used the textbook to supplement course information or to reference visual materials but not as a main source of information. Included in this statistic was the number of students who reported not using the textbook much or at all (OER = 18.6%, and OER+OP = 24%).

Perceptions

In regard to the question asking about the match between the content in the textbook and the learning objectives for the course, the OER data indicated “agreement” and “strong agreement” at 68.1%, while the OER+OP data indicated approximately the same at 66.2% (see Table 6).

TABLE 6

Table 6. Survey results for questions measuring student perceptions of the OER for OER and OER+OP semesters.

Another question asked about the ease or difficulty in using the textbook. A majority of students in both semesters felt the OER textbook ranged from “Somewhat Easy” to “Easy” or “Very Easy” to use (85.9 and 83.9% respectively).

In regard to the perception about the quality of the textbook, there was a close distribution of responses between “Average,” “Above Average,” and “Excellent,” with a slight emphasis on “Above Average” by both classes (OER=38.1% and OER+OP=39.2%). There was no significant difference was found when running the independent samples t-test comparing perceptions between the classes for the individual questions (see Table 7). This was confirmed when running a t-test for independent samples on the combined questions for the perception construct between the OER class (M = 5.18, SD = 0.80) and the OER+OP class (M = 5.07, SD = 0.84); t₍₃₁₅₎ = −1.08, p = 0.28.

TABLE 7

Table 7. Independent samples T-test comparing perceptions of OER for OER and OER+OP semesters.

Engagement

A summary of integrated responses to the questions seeking information on “engagement” is shown in Table 8. In regard to maintaining interest in the course, more than half of the OER class indicated that they “Agreed” or “Strongly agreed” that the OER helped to maintain their interest (60.2%), though only half of the students in the OER+OP class indicated the same (50.5%). Almost 62% of the OER class “Agreed” or “Strongly agreed” that the OER helped to challenge their thinking in a positive way, whereas 54.5% indicated the same in the OER+OP class. Data indicated that students felt that the OER helped to support their understanding of concepts in the classes. For the OER class, 69.9% of students “Agreed” or “Strongly agreed” with this, while 63.2% of students in the OER+OP class did so. Regarding encouraging participation, the survey data indicated that 57.5% of students in the OER class “Somewhat” to “Strongly agreed” that the OER encouraged class participation. Response in the OER+OP class, however, was lower as 52% of students indicated that the OER encouraged participation.

TABLE 8

Table 8. Survey results for questions measuring student engagement using OER for OER and OER+OP semesters.

The independent samples t-tests run on the individual questions for “engagement” revealed no significant difference except for “understanding” (see Table 9). Students in the OER-only class rated the ability of the textbook to help them understand concepts in the course statistically more positively than students in the OER+OP class. Once again, when a t-test for independent samples was run for the combined questions constituting the construct of “engagement” as shown in Table 9, there was no significant difference between the OER group (M = 5.40, SD = 1.01) and the OER+OP group (M = 5.16, SD = 1.06); t₍₃₁₅₎ = −1.73, p = 0.084, although the OER+OP mean was lower.

TABLE 9

Table 9. Independent samples T-test comparing engagement with OER for OER and OER+OP semesters.

Qualitative Findings

Themes that emerged from the qualitative data are reviewed in the following sections.

Cost

Through the analysis of the qualitative data for each semester, three main themes evolved related to cost: the use of the OER had a financial impact, an emotional impact, and an educational impact on the course.

Financial impact

Interview data revealed that a majority of students were very appreciative of the free textbook as it allowed them more flexibility with their limited budgets. Most of the students interviewed needed to work during their schooling, and the no-cost textbook allowed them to use the savings in other areas such as food or transportation. One student relayed that she was able to take an extra class because of the savings. For both semesters, the qualitative survey data indicated that “cost” was the second-most mentioned aspect of “liking the textbook,” second to having online access. One student from the OER+OP interview shared that “having this [OER] available free and online for me is very helpful because I usually don't plan on paying for any textbooks.”

Emotional impact

A number of students shared that by not having to try and purchase an expensive text, it relieved them of the extra stress in trying to stretch their budget. One student said, “Since I didn't have to wait and see if I really needed to buy the textbook, I had it right away as soon as the class began. This took a lot of worry off of my shoulders.”

Educational impact on course

What emerged from the interview data was that cost can determine whether a student will purchase a textbook. When students were asked to speculate on whether they would have purchased the text for the class at the cost of $130., almost half of the students interviewed in the OER semester said they would not have purchased the textbook, while over two-thirds in the OER+OP semester said they would not have purchased it. In contrast, both groups indicated that had they not had access to the textbook due to cost, it would have limited their information, thus potentially impacting course success. A student from the OER+OP semester said, “I think that a lot of college students would [not purchase the book] and they would just try to focus on the lectures and the slide shows that the professor offers. But I think that just shows how money can really limit a person's learning.”

Outcomes

The major theme that became evident from the student interview data for this construct was regarding performance. Performance related to both course grades and course assignments.

What emerged from the data indicated students from both groups used the OER textbook to support their learning. Some examples mentioned by students were as follows: reinforcing concepts, providing more in-depth information, assisting in exam preparation and review, and aiding in homework, which would impact student grades and how well they completed the assignments. Most students in the OER class responded that they felt using the textbook was beneficial to their grade, while fewer of the OER+OP students felt the same. One student in the OER group explained, “I think it really impacted my grade. I did end up using the textbook a lot, as compared to my other classes, just because there was so much more in-depth information that I needed to know for sure for exams or quizzes. It definitely helped, I think.” Both the OER and the OER+OP interview data indicated that the impact of assignments on student performance in class was not great. It became clear through the student interviews that many students in both groups thought the course exams generally impacted their overall performance more than the course assignments.

Use

For this construct, the data revealed two major themes: features of the OER that promoted use; and actual use of the resource.

Features

Analysis of the qualitative data from the open-ended survey questions and the interviews revealed parallel features of the OER that encouraged use. Those features were broken down into three groupings of overall organization: the design of the textbook, the way the text was structured, and navigation (see Table 10). Students also reported in interviews over both semesters that the features of the textbook that encouraged use was that the textbook was online and portable and that it was well-organized. Interview data collected from the OER+OP class were very similar in pattern to the OER class, with no specific features emerging as more prevalent for the OER+OP class. Students were asked in the interview about the ease or difficulty in using the OER. One or two students in both groups reported that, initially, they were challenged by the format options for accessing the text and that it took some effort to learn to navigate it properly. A student from the OER semester interview exclaimed that “unlike my other courses where if I forget my book, I have to go back for it or do without, this online OER is available all the time.”

TABLE 10

Table 10. Features of the OER textbook that encouraged use as reported in OER and OER+OP student interviews.

Actual use

Most students in the interviews for both semesters reported using the OER mainly as a supplemental resource. It emerged from interview data that students relied on the presentation slides, lecture notes, and links to supplemental information as their main sources of information for the class. Students in both semesters shared that it wasn't necessary to use the OER extensively because the information provided by the professor was very complete. During the interviews, students from both groups shared the almost unanimous response that the textbook wasn't essential to complete the assignments. One student in the OER+OP group admitted that “I didn't really read the entire [textbook]. I usually see a question, I look at what's the main topic, and then I go to the textbook … and just use ‘Control F’ to find what I need. That's pretty much how I use the textbook.”

Perceptions

In addition to the interview data, two open-ended questions on the survey asked students to provide feedback on the OER textbook. Very similar themes emerged through analysis of the data from the open-ended questions as from the interviews. When comparing the OER and the OER+OP thematic data between the groups, there was a marked similarity in patterns of information.

Content correlates to course objectives

Interview data indicated that there existed a parallel between the content of the course and the course objectives. The data indicated that students from both semesters thought that there was a match between the content of the OER textbook and the course learning objectives. A number of students in both semesters shared that they appreciated how closely the OER paralleled the course because it took the guesswork out of finding information. One student explained, “I think [the textbook] was spot on to what we were learning in class. I remember reading along in the textbook and having my PowerPoints open, and like, Oh, it's kind of the same format, … which made it easy when studying for exams and tests and things.”

Quality

Regarding the quality of the OER, two sub-themes emerged from the data: the OER was easy to comprehend and it added depth to the course. A student from the OER+OP semester shared this information on the survey: “I liked how straight forward the context was, it goes straight to the point. I never once was confused with what was being said. All the information given in the textbook was useful, not every textbook is like that.” In an interview in the OER semester, one student sounded surprised when she spoke about the quality of information “… it was really amazing. You know, you can actually learn a lot—just from reading it!”

Engagement

Qualitative data on “engagement” were also collected through the interviews and focused on interest, challenge, understanding, and participation. Several themes were identified.

Interesting topics

A theme that emerged for both semesters was that interest was maintained because of the topics covered in the OER textbook and in the class assignments. One student in the OER semester relayed that “the text helped me to build a better foundation for [the class].” Several students shared in the interviews that the topics, rather than just the textbook, promoted interest. One student in the OER+OP semester explained, “it was really more the topics that [the professor] brought up that were engaging for me …and everything was just intellectually challenging because it is a different mindset.” There was more positive data reported by the OER class on how the class assignments affected interest; however, one specific mention of how the OP assignment helped to maintain interest in the OER+OP class was from one student's interview feedback. This student shared that the assignment was of interest because “… skills learned from the assignment will help me with research in future classes, and I appreciate learning these skills.” One student in the OP class shared that “researching different topics and opening up articles for a diverse group of topics … wasn't challenging. It was just more interesting.”

Real-world application

Some of the interview data indicated how the OER textbook impacted course understanding. Both classes reported that the OER textbook helped with understanding by providing definitions to class concepts and to materials from research articles and outside resources. One student stated, “I'm going to keep [the OER textbook] downloaded on my laptop because I'm into the topic. I'm into nutrition, so I'm going to look back at it and be able to reference it.”

In focusing on how the class assignments helped with understanding the course concepts, both groups affirmed that the assignments pertained to their life. Data indicated that students felt the content was relevant to their life because of the topics. This was especially true in the OER group. In an interview, one student in the OER group relayed that “I was able to change my diet or suggest to friends what they could do to be healthier.” A number of students in the OER+OP group also discussed the real-world application of the assignments. One student in this interview group shared that the OP assignment “will really keep students engaged as opposed to what you would normally think of as an assignment for a nutrition class.” Another student in the OER+OP group shared, “The assignments … helped you understand the concepts, and they also made you go a little bit deeper and helped you to relate the materials from class to the real world and to situations.”

Localization

The authors of the OER had spent time contextualizing the material to include local and regional examples, which was appreciated by students. One student from the OER+OP semester shared that the OER helped to maintain his interest because of the information about Hawai'i and the Pacific and said, “I grew up in here and I didn't even know these things about Hawai'i.” One student responded in the open-ended survey data that “the details … made learning more interesting and engaging. The incorporation of the Hawaiian culture is also unique to our university, and I appreciate it.”

Sharing forward

The interview data revealed that only a small number of students in the OER+OP semester completely understood that the OP assignment would impact the future textbook. However, one student in the OER+OP class was impressed that they were working on an assignment where the information would later appear as part of the textbook: “We're helping other students in the future with their research.” Another student shared, “I thought it was interesting … how you guys are going to use [the assignment] for the … textbooks because it shows that what we find interesting, maybe other students will find that interesting too. And then it'll really keep the students engaged as opposed to what you would normally think a nutrition class is just saying, ‘This is healthy, this is bad, this is good.’” The OP assignment was viewed as a means to understand how research is conducted and then shared. “The research opened my eyes to how we're getting the information that we are, and where it's being sourced. That's pretty cool.” Finally, one student thought “it was really cool that we got to contribute in some way, and it was interesting for us as students. Not just something that the teacher assigned. We sort of had some control over it.”

Comparison of Quantitative and Qualitative Results

The quantitative data collected regarding “cost” indicated that students choosing to purchase a new textbook for the course would have had to spend $130 at the campus bookstore. Themes that emerged from the qualitative data indicated that this cost would have had a financial and emotional impact on students as well as impacting how they did in the course. Qualitative data gathered from student interviews indicated that, for a number of students, the cost would have prohibited them from purchasing the text. Interview data also revealed that the cost would have had an impact on the success of completing the course.

The qualitative data supported the quantitative data in terms of student outcomes. The course throughput rate comparing both classes revealed no significant difference. This was substantiated by the theme that emerged in analyzing data from the interviews. The data portrayed a mixed message when analyzing if the OER textbook impacted overall performance: students in the OER semester felt more strongly that it did so, while fewer students in the OER+OP semester felt the textbook was impactful. A majority in both classes in the survey reported that the assignments had little impact on their overall performance in the class.

Both quantitative and qualitative data indicated that the OER was not fully used. The interview data supported the finding that students mostly used lecture notes and slides as the main source of information. Both the survey and interview data indicated that the assignments (even in the OER+OP group) did not necessarily encourage the use of the OER. Only a very small number of students in both interview groups relayed that they read the OER closely. In fact, the majority of students in both groups shared that the OER was used mostly as a supplemental text. Students in both groups indicated that there were certain features that did promote use of the OER: the online format, the well-organized structure, and certain functions like the navigation and search features.

Referring to the data for “perceptions” and analyzing patterns for the match between OER content to course objectives, student interview data suggested similar patterns found in the quantitative survey data. Both indicated that the content found in the OER matched closely with the course objectives. The same patterns were evident regarding the quality of information for both quantitative survey data and interview data. Once again, the interview data indicated a greater feeling that the OER contained quality information.

Finally, this research explored student “engagement” as influenced by the OER text and by the OP assignment through examining levels of interest, challenge, understanding, and participation as evidenced by the quantitative data and through themes that emerged from the qualitative data. The quantitative data generally paralleled the qualitative data when examining interest level, where both classes felt the OER somewhat helped to maintain interest. Once again, thematic data suggested that interesting and relevant topics were important influencing factors. The latter was especially true for the OER-only semester. However, students in the OER+OP semester, who understood the goal of the OP assignment, reflected very positively on their role in contributing to the OER. Data were not consistent when comparing quantitative data with qualitative data in exploring how students were positively challenged by the OER textbook and by the assignments. While the quantitative data indicated a little over average agreement that the material in the textbook positively challenged them for both semesters, the data that emerged from the interviews indicated a much weaker support for this concept. This could possibly be due to the different contexts in which students answered the questions. There might have been more clarification about the concept of “challenge” in the interview scenario, which lent a different meaning to the concept than was understood by students taking the survey. The quantitative data indicated that 89.4% of students in the OER-only class “Somewhat” to “Strongly agreed” that the OER supported understanding, while 82.8% in the OER+OP class indicated the same. Themes that emerged in the qualitative data substantiated this finding for the use of the OER textbook. On the other hand, though still positive, the interview data for the OER+OP class revealed that students didn't really feel that the assignments promoted understanding. Finally, as the concept of “participation” was explored, the quantitative data and the qualitative data were very similar for both the OER and the OER+OP classes when exploring how the OER textbook encouraged class participation. For both data sources, approximately half of the students felt that the textbook encouraged participation. Data from the interviews on how the assignments affected participation indicated that generally the assignments weren't impactful in either group.

Discussion

This study explored various aspects of student experiences when utilizing OER and when applying OER-enabled pedagogy within an undergraduate course over two university semesters. It focused on the impact of cost, student outcomes, resource use, various perceptions, and engagement within this context. The conceptual framework utilized in this study extended the original COUP framework (Hilton et al., 2016b), which has focused on “cost,” “outcomes,” “use,” and “perceptions” that has been used as the basis of a great deal of research on OER to include the construct of student “engagement.”

This study was unique because most OER research compares similarities and differences between traditional textbooks and OER use. However, this study employed the use of an OER for both semesters while seeking to determine if OER+OP significantly impacted students. Quantitative data comparing the OER group and the OER+OP group indicated that there was no significant difference regarding student outcomes, use, perceptions, and engagement. Future research should be conducted exploring OER-enabled pedagogy in different contexts. For example, would OER+OP be more significant in smaller classes, in upper division classes, or is it as beneficial in online classes compared to those conducted face-to-face. A small number of educators are beginning to examine OER-enabled pedagogy in their courses (DeRosa and Robinson, 2017; Jhangiani and Biswas-Diener, 2017), but only one empirical study has been conducted examining OP at the university level (Hilton et al., 2019). Therefore, this study can be considered an undertaking to begin exploring OP and factors promoting student learning in higher education within the context of OER.

COUPE

Findings in this research were similar to those in other OER research in terms of cost savings, course load, and impact on budget (Hilton et al., 2014; Senack, 2014; Fischer et al., 2015); however, this research differed in that qualitative data were gathered that indicated a degree of relief from the emotional impact when an OER was utilized. Considering that over both semesters in this study, almost one-third of the participants reported having a total household income of $49,999 or less (OER = 31% and OER+OP = 32.4%) and that for both semesters 24% of students were utilizing economic grants, the cost savings could have quite an impact on students' budgets. Student outcomes found in this study were consistent with some OER research as well (Hilton et al., 2013; Choi and Carpenter, 2017; Fialkowski et al., 2020) indicating no significant difference in outcomes when OER is utilized. However, other OER research does support a difference in outcomes (Feldstein et al., 2012; Winitzky-Stephens and Pickavance, 2017). Just as in the Bliss et al. (2013b) study comparing traditional and OER textbooks, no significant difference was found in use patterns in this study between groups. Findings from the qualitative data were important, however, because one theme that emerged discussed features of the OER that promoted use. The finding that “use” of the OER was primarily as a supplementary document was possibly influenced by complete and thorough lecture notes, slides, and linked information rather than by poor design or lack of information within the text. Understanding which features affect the design, text, and function of a resource have implications for future OER development. This is true also for themes revealed in the qualitative data regarding perceptions: quality and content are important to students as well. Much of the OER research focusing on perception has dealt primarily with quantitative data (Feldstein et al., 2012; Gil et al., 2013); therefore, this research adds a deeper understanding to that aspect of OER research.

Engagement

This research is unique in a second way: it extended the original COUP framework to include the construct of “engagement,” work that began when utilizing this survey instrument in recent OER research (Fialkowski et al., 2020). Though a great deal of educational research has focused on the concept of student engagement (Fredericks, 2011; Fallen et al., 2013; Gunuc, 2014; Fredericks and McColskey, 2018), research on engagement when using OER and OER+OP has been limited. Unlike the Rowell (2015) study and the Ikahihifo et al. (2017) research, the current research used a multidimensional view of engagement, which focused on the areas of behavioral, emotional, and cognitive engagement (Fredericks, 2011). Once again, themes were identified from the current study's qualitative data, themes that could potentially be considered when further developing OER and OER-enabled instruction. For example, it became clear that when course topics were contextualized and made relevant to a student's life, students expressed a high degree of interest in the course. This has implications for instructional design. Another theme that carries implications for instructional design emerged from interviews with the OER+OP group. Those students, who understood the nature of the OP assignment, were excited about the prospect of sharing their work forward through the completed assignment. This finding supports the exploratory work being done by researchers in terms of open practices and learning and should encourage further investigation (DeRosa and Robinson, 2017).

Limitations

Several limitations of this study exist. Data were self-reported and therefore may not be standardized. All assignments could have been completed through accessing the learning management system. This may have accounted for some of the students, when interviewed in the OER+OP semester, confessing to not understanding that the results from the research assignment would potentially be added to the OER textbook, thus including student-contributed work as part of the resource. This, therefore, gave rise to the speculation that a portion of the students (who might not have always attended the face-to-face class) may not have understood about their potential contribution to the OER. In addition, during the interviews, students also indicated that the classes were mostly lecture style, with limited discussion taking place, in part, due to the large class numbers. This style of delivery limits student engagement as discussion and participation were not an integral component of the class context. Finally, since this study took place within one course at one university, the findings cannot be extrapolated to wider populations.

Conclusion and Future Research

Over the last decade, there has been a considerable amount of research conducted that has examined various aspects of OER adoption in terms of the cost, outcomes, use, and perception of OER based on the COUP conceptual framework proposed by the Open Education Group (Hilton et al., 2016b). However, scarce research has taken place that has focused on the aspect of student engagement in courses using OER, in particular when OER-enabled pedagogy has been incorporated. This study was unique because it began to explore these concepts within an undergraduate class.

Though no significant quantitative difference was found between the OER and the OER+OP groups in this study, it should be acknowledged that there was no negative impact to student learning outcomes, and these students were able to save money through the use of the OER. Gratitude at the savings was acknowledged repeatedly by comments of appreciation through both the open-ended survey questions and through the interview data. The difference found in the qualitative data supported the idea that student engagement might be different in a course that offers an OER-enabled pedagogical approach by possibly empowering students to contribute to the field.

As costs for higher education continue to rise (College Board, 2018), future research can continue to explore the role that OER and open educational practices might play in that context. In addition, it can explore how OER-enabled pedagogy might potentially impact student learning outcomes. Future research should also more fully explore how student engagement might be influenced when using OER and OER-enabled pedagogy. These findings will hopefully prompt educators to explore these facets of learning and teaching with OER.

Data Availability Statement

The datasets presented in this study can be found in online repositories. The name of the repository is ScholarSpace and accession number(s) can be found below: http://hdl.handle.net/10125/44908.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

BT created instruments, collected and analyzed data, and drafted the research article. MF initiated the original OER project and guided the creation of the OER as well as provided feedback on the article. JD introduced the OER into instruction, provided feedback on the resource and the article, and implemented an OER-enabled pedagogical assignment featured in the article.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This study represents a segment of the dissertation research conducted by BT. The entire dissertation will be available in ScholarSpace (https://scholarspace.manoa.hawaidu/), the University of Hawai'i at Mānoa's open access institutional repository. I would particularly like to acknowledge and thank my dissertation advisor, Dr. Christine Sorensen-Irvine, for her excellent support and guidance (BT). The authors wish to thank the individuals who participated in this research.

References

Allen, G., Guzman-Alvarex, A., Molinaro, M., and Larsen, D. (2015). Assessing the Impact and Efficacy of the Open-Access ChemWiki Textbook Project. Educause Learning Initiative Brief. Retrieved from: https://library.educause.edu/resources/2015/1/assessing-the-impact-and-efficacy-of-the-openaccess-chemwiki-textbook-project (accessed August 4, 2017).

Google Scholar

Bacon, L., Stern, J. S., Van Loan, M. D., and Keim, N. L. (2005). Size acceptance and intuitive eating improve health for obese, female chronic dieters. J. Am. Dietetic Assoc. 105, 929–936. doi: 10.1016/j.jada.2005.03.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Bliss, T., Hilton, J. III., Wiley, D., and Thanos, K. (2013a). The cost and quality of online open textbooks: perceptions of community college faculty and students. First Monday 18:3972. doi: 10.5210/fm.v18i1.3972

CrossRef Full Text | Google Scholar

Bliss, T., Robinson, T. J., Hilton, J. III., and Wiley, D. (2013b). An OER COUP: college teacher and student perceptions of open educational resources. J. Interactive Med. Educ. 4, 1–25. doi: 10.5334/2013-04

CrossRef Full Text | Google Scholar

Bowen, W. G., Chingos, M. M., Lack, K. A., and Nygren, T. I. (2012). Interactive Learning Online at Public Universities: Evidence From Randomized Trials. Ithaka S + R. Avaliable online at: https://sr.ithaka.org/wp-content/uploads/2015/08/sr-ithaka-interactive-learning-online-at-public-universities.pdf

Google Scholar

Braun, V., and Clarke, V. (2008). Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101. doi: 10.1191/1478088706qp063oa

CrossRef Full Text | Google Scholar

California Open Educational Resources Council (2016). OER Adoption Study: Using Open Educational Resources in the College Classroom. Retrieved from: http://tinyurl.com/WPOERAdoption040116 (accessed September 8, 2018).

Google Scholar

Choi, Y. M., and Carpenter, C. (2017). Evaluating the impact of open educational resources: a case study. Libraries Acad. 17, 685–693. doi: 10.1353/pla.2017.0041

CrossRef Full Text | Google Scholar

Chronicle of Higher Education (2018). The Almanac of Higher Education, 2018-2019. Retrieved from: https://www.chronicle.com/specialreport/The-Almanac-of-Higher/214 (accessed July 28, 2017).

College Board (2018). Trends in College Pricing, 2018. (Trends in Higher Education Series). Retrieved from: https://trends.collegeboard.org/sites/default/files/2018-trends-in-college-pricing.pdf (accessed February 18, 2019).

Google Scholar

Cooney, C. (2017). What impacts do OER have on students? Students share their experiences with a health psychology OER at New York City College of Technology. Int. Rev. Res. Open Distrib. Learn. 18, 155–178. doi: 10.19173/irrodl.v18i4.3111

CrossRef Full Text | Google Scholar

Croteau, E. (2017). Measures of student success with textbook transformations: the affordable learning georgia initiative. Open Praxis 9, 93–108. doi: 10.5944/openpraxis.9.1.505

CrossRef Full Text | Google Scholar

DeRosa, R., and Robinson, S. (2017). “From OER to open pedagogy: harnessing the power of open,” in Open: The Philosophy and Practices That Are Revolutionizing Education and Science, eds. R. S. Jhangiani and R. Biswas-Diener (London, UK: Ubiquity Press), 115–124.

Google Scholar

Everard, A., and Pierre, K. S. (2014). A case for student adoption of open textbooks. J. Acad. Bus. Educ. 15, 66–76.

Google Scholar

Fallen, E., Walsh, S., and Prendergast, T. (2013). An activity-based approach to learning and teaching of research methods: measuring student engagement and learning. Irish J. Acad. Pract. 2:2. doi: 10.21427/D7Q72W

CrossRef Full Text | Google Scholar

Feldstein, A., Martin, M., Hudson, A., Warren, K., Hilton, J. III., and Wiley, D. (2012). Open textbooks and increased student access and outcomes. Eur. J. Open Distance E-Learn. 2:9. Retrieved from: http://www.eurodl.org/materials/contrib/2012/Feldsteint_et_al.pdf

Google Scholar

Fialkowski, M. K., Calabrese, A., Tillinghast, B. H., Titchenal, A., Meinke, W., Banna, J., et al. (2020). Open educational resource textbook impact on students in an introductory nutrition course. J. Nutr. Educ. Behav. 52, 359–368. doi: 10.1016/j.jneb.2019.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Fialkowski, M. K., Titchenall, A., Calabrese, S., Gibby, C., and Meinke, W. (2018). Human nutrition. Retrieved from: https://dspace.lib.hawaii.edu/handle/10790/3487 (accessed January 5, 2019).

Google Scholar

Fischer, L., Hilton, J. III., Robinson, T. J., and Wiley, D. A. (2015). A multi-institutional study of the impact of open textbook adoption on the learning outcomes of postsecondary students. J. Comp. Higher Educ. 27, 159–172. doi: 10.1007/s12528-015-9101-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Fredericks, J. (2011). Engagement in school and out-of-school contexts: a multidimensional view of engagement. Theory Pract. 50, 327–335. doi: 10.1080/00405841.2011.607401

CrossRef Full Text | Google Scholar

Fredericks, J. A., and McColskey, W. (2018). “The measurement of student engagement: a cognitive analysis of various methods and student self-report instruments,” in Handbook of Research on Student Engagement, ed S. L. Christenson (New York, NY: Springer-Verlag), 763–782.

Gale, M. M. (2016). Impacts of open educational resources on enrollment rates, withdrawal rates, and academic performance in the virginia community college system (Master's Project, Old Dominion University). Retrieved from: http://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1425&context=ots_masters_projects (accessed October 18, 2018).

Google Scholar

Gil, P., Candelas, F., Jara, C., Garcia, G., and Torres, F. (2013). Web-based OERs in computer networks. Int. J. Eng. Educ. 29, 1537–1550. Retrieved from: http://www.ijee.ie/latestissues/Vol29-6/21_ijee2802ns.pdf

Google Scholar

Gunuc, S. (2014). The relationships between student engagement and their academic achievement. Int. J. New Trends Educ. Implicat. 5, 216–231. doi: 10.2224/sbp.7054

CrossRef Full Text | Google Scholar

Hendricks, C., Reinsberg, S. A., and Rieger, G. (2017). The adoption of an open textbook in a large physics course: an analysis of cost, outcomes, use, and perceptions. Int. Rev. Res. Open Distrib. Learn. 18, 78–99. doi: 10.19173/irrodl.v18i4.3006

CrossRef Full Text | Google Scholar

Henrie, C. R., Halverson, L. R., and Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: a review. Comp. Educ. 90, 36–53. doi: 10.1016/j.compedu.2015.09.005

CrossRef Full Text | Google Scholar

Hewlett Foundation. (2020). Open Education Strategy. Retrieved from: https://hewlett.org/wp-content/uploads/2016/08/Hewlett-OpenEd-Strategy-2020.pdf (accessed January 4, 2020).

Google Scholar

Hilton, J. III. (2016). Open educational resources and college textbook choices: a review of research on efficacy and perceptions. Educ. Tech. Res. Dev. 64, 573–590. doi: 10.1007/s11423-016-9434-9

CrossRef Full Text | Google Scholar

Hilton, J. III., Fischer, L., Wiley, D., and Williams, L. (2016a). Maintaining momentum toward graduation: OER and the course throughput rate. Int. Rev. Res. Open Distrib. Learn. 17, 37–50. doi: 10.19173/irrodl.v17i6.2686

CrossRef Full Text | Google Scholar

Hilton, J. III., Wiley, D., Fischer, L., and Nyland, R. (2016b). Guidebook to Research on Open Educational Resources Adoption. Open Education Group. Retrieved from: http://openedgroup.org/wp-content/uploads/2016/08/OER-Research-Guidebook.pdf (accessed July 8, 2018).

Google Scholar

Hilton, J. L. III., Gaudet, D., Clark, P., Robinson, J., and Wiley, D. (2013). The adoption of open educational resources by one community college math department. Int. Rev. Res. Open Distrib. Learn. 14. doi: 10.19173/irrodl.v14i4.1523

CrossRef Full Text | Google Scholar

Hilton, J. L. III., Robinson, J., Wiley, D., and Ackerman, D. (2014). Cost-savings achieved in two semesters through the adoption of open educational resources. Int. Rev. Res. Open Distrib. Learn. 15, 67–84. doi: 10.19173/irrodl.v15i2.1700

CrossRef Full Text | Google Scholar

Hilton, J. L. III., Wiley, D., Chaffee, R., Darrow, J., Guilmett, J., Harper, S., et al. (2019). Student perceptions of open pedagogy: an exploratory study. Open Praxis 11, 275–288. doi: 10.5944/openpraxis.11.3.973

CrossRef Full Text | Google Scholar

Ikahihifo, T., Spring, K., Rosecrans, J., and Watson, J. (2017). Assessing the savings from open educational resources on student academic goals. Int. Rev. Res. Open Distrib. Learn. 18. doi: 10.19173/irrodl.v18i7.2754

CrossRef Full Text | Google Scholar

Jhangiani, R., and Jhangiani, S. (2017). Investigating the perceptions, use, and impact of open textbooks: a survey of post-secondary students in British Columbia. Int. Rev. Res. Open Distrib. Learn. 18, 172–192. doi: 10.19173/irrodl.v18i4.3012

CrossRef Full Text | Google Scholar

Jhangiani, R. S., and Biswas-Diener, R. (2017). Open: The Philosophy and Practices that are Revolutionizing Education and Science (London: Ubiquity Press).

Google Scholar

Lashley, J., Cummings-Sauls, R., Bennett, A., and Lindshield, B. (2017). Cultivating textbook alternatives from the ground up: one public university's sustainable model for open and alternative educational resource proliferation. Int. Rev. Res. Open Distrib. Learn 18, 212–230. doi: 10.19173/irrodl.v18i4.3010

CrossRef Full Text | Google Scholar

Lindshield, B. L., and Adhikari, K. (2013). Online and campus college students like using an open educational resource instead of a traditional textbook. J. Online Learn. Teach. 9, 26–38.

Google Scholar

Mulder, F. (2013). The logic of national policies and strategies for open educational resources. Int. Rev. Res. Open Distrib. Learn. 14, 96–105. doi: 10.19173/irrodl.v14i2.1536

CrossRef Full Text | Google Scholar

Pawlyshyn, N., Braddlee, D., Casper, L., and Miller, H. (2013). Adopting OER: A Case Study of Cross-Institutional Collaboration and Innovation. Retrieved from: http://er.educause.edu/articles/2013/11/adopting-oer-a-case-study-of-crossinstitutional-collaboration-and-innovation (accessed June 20, 2017).

Prasad, D., and Usagawa, T. (2014) Scoping the possibilities: Student preferences towards open textbooks adoption for e-learning. Creat. Educ. 5, 2027–2040. doi: 10.4236/ce.2014.524227

CrossRef Full Text | Google Scholar

Rowell, J. L. (2015). Student perceptions: teaching and learning with open educational resources (Ph.D. dissertation; Johnson City, TN: East Tennessee State University, U.S.). Retrieved from: https://dc.etsu.edu/cgi/viewcontent.cgi?article=3925&context=etd

Google Scholar

Senack, E. (2014). Fixing the Broken Textbook Market: How Students Respond to High Textbook Costs and Demand Alternatives. Student PIRGs 2014. Retrieved from: https://uspirg.org/sites/pirg/files/reports/NATIONAL%20Fixing%20Broken%20Textbooks%20Report1.pdf (accessed May 1, 2018).

Google Scholar

Senack, E., and Donoghue, R. (2016). Covering the Cost: Why We Can No Longer Afford to Ignore High Textbook Prices. Student PIRGs 2016. Retrieved from: http://www.studentpirgs.org/reports/sp/covering-cost (accessed October 3, 2018).

Google Scholar

Trowler, V., and Trowler, P. (2010). Student Engagement Evidence Summary. The Higher Education Academy. Retrieved from: https://www.heacademy.ac.uk/system/files/studentengagementevidencesummary_1.pdf (accessed March 18, 2019).

Google Scholar

Vitez, K. (2018). Open 101: An Action Plan for Affordable Textbooks (Student PIRGs 2018). Retrieved from: www.studentpirgs.org/textbooks (accessed April 9, 2019).

Webber, K. L., Krylow, R. B., and Zhang, Q. (2013). Does involvement really matter? indicators of college student success and satisfaction. J. College Stud. Dev. 54, 591–611. doi: 10.1353/csd.2013.0090

CrossRef Full Text | Google Scholar

Wiley, D. (2014). The Access Compromise and the 5th R. Iterating Toward Openness. Web log post. Retrieved from: https://opencontent.org/blog/archives/3221 (accessed July 20, 2018).

Wiley, D., and Hilton, J. III. (2018). Defining OER-enabled pedagogy. Int. Rev. Res. Open Distrib. Learn. 19:3601. doi: 10.19173/irrodl.v19i4.3601

CrossRef Full Text | Google Scholar

Winitzky-Stephens, J., and Pickavance, J. (2017). Open educational resources and student course outcomes: a multilevel analysis. Int. Rev. Res. Open Distrib. Learn. 18, 35–49. doi: 10.19173/irrodl.v18i4.3118

CrossRef Full Text | Google Scholar