Designing effective discussion forum in MOOCs: insights from learner perspectives

1 Introduction

With the rapid development of information technology, E-learning has become an indispensable method for acquiring new knowledge and developing skills. Additionally, over the past couple of years, online education has grown to be an integral component of our educational landscape, largely due to the global COVID-19 pandemic (Baber, 2021; Muthuprasad et al., 2021; Rahiem, 2021). Furthermore, recent trends in online education, encompassing knowledge acquisition and the development of practical and communication skills, have led to a surge in the number of individuals engaging in distance learning (Ma et al., 2021). The appeal of online education has also been amplified by the demand for flexible learning schedules, an adaptable learning environment, and access to remote education opportunities (Castro and Tumibay, 2021).

As conventional teaching methods struggle to meet the escalating demands of educational institutions such as universities and colleges, Massive Open Online Courses (MOOCs) have emerged as a prominent online education platform, offering a wide range of course content to diverse learner profiles. Several MOOC platforms, including Coursera (coursera.org), edX (edx.org), FutureLearn (futurelearn.com), Udemy (udemy.com), Udacity (udacity.com), and SWAYAM (swayam.gov.in), have gained substantial attention due to their flexibility in terms of participation, motivation, language options, and accessibility, thereby opening doors to lifelong learning opportunities in a cost-effective and efficient manner (Ezen-Can et al., 2015; Reich and Ruipérez-Valiente, 2019). In terms of time management, MOOCs provide prerecorded video lectures that cater to the convenience of both learners and instructors (Chauhan and Goel, 2015). Furthermore, within MOOC courses, interaction among learners and instructors plays a pivotal role in enhancing the learning experience and knowledge sharing (Frau-Meigs et al., 2021). The discussion forum, traditionally viewed as a means of communication, also assumes the role of a learning activity unveiling novel educational opportunities within a social context. Despite the rapid growth of online learning over the past 2 years, it still lacks effective facilities for meaningful learning discussions between learners and instructors, particularly in asynchronous courses. Additionally, MOOC learners face challenges in interacting with instructors and peers due to the design of the platform provided, which have been identified as a primary cause of learner dropouts (Gamage et al., 2020). The COVID-19 pandemic has disrupted conventional classroom methods due to lockdowns, leading to a surge in online learning adoption. However, managing interaction in virtual learning platforms, whether real-time or asynchronous, becomes cumbersome when dealing with a large number of participants, especially within discussion forums. Accessing and interacting with these forums can be particularly challenging, affecting both learners’ access to information and instructors’ ability to respond promptly. Therefore, there is a compelling need to explore and design effective discussion tools within MOOCs to determine whether these platforms promote interactions that go beyond the scope of conventional forums.

Discussion forums within MOOCs are great platforms for participants who share common interests. Additionally, active participation by learners in discussion forums significantly fosters the concept of “intercreativity,” which was introduced by Tim Berners-Lee back in 1996. This concept encourages collective problem-solving activities and collaborative content creation. However, the existing discussion forum is a massive bundle of unstructured knowledge that requires effective utilization to improve the interaction and intercreativity of learners for better learning outcomes (Frau-Meigs et al., 2021). Despite the recognized importance of communication and interaction among participants in educational settings, several constraints such as the navigation of information in existing discussion forum persist, hindering the development of a truly interactive approach. Therefore, it is crucial to persistently focus on improving digital tools to achieve even higher levels of satisfaction and enhance the transfer of learning along with contribution to the long-term educational society.

Understanding the factors influencing interaction and learner satisfaction can play a vital role in reducing dropout rates and enhancing perceived learning in asynchronous online learning environments. Three key factors contributing to learner satisfaction and perceived learning are the clarity of design, interaction with instructors, and active engagement among peer learners (Swan, 2001). These factors are interconnected with four types of interactivities within a community of inquiry:

(1) Interaction with content (cognitive presence), where the learner interacts with content to gain content knowledge through media, such as discussion forums. This type of interaction includes a concept of clear feedback and clear navigation to support the effective design of online learning, requiring future research to explore these concepts. According to research cognitive presence made the largest contribution to student satisfaction. However, the combination of cognitive presence and teaching presence contributes to student satisfaction in an asynchronous discussion forum (Singh et al., 2022).

(2) Interaction with instructors (teaching presence), where the instructor delivers content knowledge, clarifies the concept in the discussion forum and increases learner motivation so that learners can ask questions confidently in the future (Terras and Ramsay, 2015). This kind of interaction shows a correlation between perceived interactions with instructors and the average number of responses per learner that instructors made in the discussion forum. However, instructors face some challenges, such as faceless classrooms, adapting learner-centered teaching, and managing time and techniques for establishing the learning community. The demand of current research in online teaching is for attention to instructor activity and interaction, as well as dealing with content interaction, specifically attention toward structure, design, and interaction among peer learners (learning community).

(3) Interaction with peer learners (social presence), where learners can learn from each other and enhance their knowledge, is an important factor in the success of online courses. Researchers have identified and discussed how individual differences in the learner profile present psychological challenges for MOOC-based learners: individual differences in skills, preferences, and cognitive profile; engagement, motivation, learning, and performance; and the ability to monitor and appropriately respond to the demands of both the external and internal contexts of learning. To address such psychological challenges, we support the application of a learner perspective on MOOC-based learning and demonstrate how such an approach, by its very nature, supports the detailed consideration of an individual learner’s psychological attributes, skills, and preferences, thereby highlighting the importance of considering the psychological constructs that explain learner behavior. The identification of the learners’ skills requires maximizing the educational benefits of MOOCs, together with the conceptualization of the psychological underpinnings of skills as barriers and enablers to technology-enhanced learning, permitting the characterization of factors that support effective MOOC-based learning, thereby offering interesting insights into maximizing the benefits of MOOC-based learning (Terras and Ramsay, 2015).

(4) Interaction with the interface, involving the usability of the medium to interact with the content, instructor, and other peer learners related to the usability of the discussion forum. This usability is defined by perceived usefulness and ease of use. Overall, research has shown that well-structured online course design and easy-to-use platforms afford increased access to instructors and feature more equitable and successful discussions, leading to a higher level of satisfaction and the ability to work together to support online learning (Swan, 2001).

Incorporating Moore’s transaction distance theory, it’s clear that interaction influences learners’ psychological perceptions of distance (Moore, 1989). Learner engagement plays a crucial role in evaluating the learning process in MOOCs, motivating them to participate in discussion forums. MOOC learners often engage differently from those in traditional online courses. They learn by discussing, applying, and talking about the material they are learning Engaged learners are more likely to participate in discussions and show improved communication relevance and autonomy in MOOC reviews (Wu, 2021). However, the sheer number of learners in MOOCs means instructors are less likely to provide timely support or feedback to all participants, leading to variations in learner engagement (Chiu and Hew, 2018).

Previous studies have primarily focused on factors influencing interactions between learners and instructors (Neha and Kim, 2020, 2021a,b). However, this study shifts its focus toward effective discussion forum design and its impact on learners’ perceptions. Learning engagement is closely tied to learners’ contributions to the discussion forum, views, and course assessment (Ramesh et al., 2013). Well-structured forums with timely instructor responses are pivotal for the learning process (Jung et al., 2002) and ineffectively designed discussion forums lead to decreased learner engagement. Previous research stated that navigating existing forums is a cumbersome task (Ntourmas et al., 2019). Therefore, this study aimed to bridge the gap between the experiences of learners and educational designers of MOOCs. Despite the interest in issues faced by learners, few studies have focused on the instructor’s experience and the learner’s performance in MOOC. Moreover, previous studies have failed to address the organization of the discussion forum and usability issues of the forum from the learner’s perspective. Accordingly, a navigation experiment was conducted on 28 participants for 2 weeks in September 2022. Each participant was rewarded after spending 1 h in total on the experiment, including the explanation, purpose, instructions for experimenting, testing of three designs, and feedback. In this experiment, three types of discussion forum platforms with different designs were presented to the participants. The experiment was designed based on the following three factors: (1) classifications of discussion forum posts and participants; (2) patterns of discussion forums; and (3) learners’ evaluation of their experience. The findings of this study will thus contribute to the redesigning of an effective asynchronous discussion forum to improve the quality of the learning process, adding substantially to our understanding of an interactive user interface in terms of design, usability, and time efficiency, which can increase the interest of learners and instructors. The aim of the study in the context of an asynchronous discussion forum was (a) support the learning process, (b) to motivate learners, and (c) to achieve learning goals by solving usability and accessibility issues of discussion forum design. This study is thus expected to contribute to our understanding of learners’ perspectives in asynchronous discussion forums by highlighting the importance of discussion forum design that can be implemented by MOOC designers for effective and efficient interaction in the asynchronous discussion forum. The study aids researchers in conducting empirical studies, surveys, or user testing to gain deeper insights into how to enhance the design and usability of asynchronous discussion forums in MOOCs. The following three research questions were framed in this study:

RQ1: How can discussion forum posts in MOOCs be effectively classified or organized to improve learner engagement and interaction efficiency?

RQ2: What are the specific design patterns for discussion forums in MOOCs that enhance the overall learning experience, and how do these patterns impact learner engagement and usability?

RQ3: How does the implementation of user-centered design elements and usability improvements in MOOC discussion forums impact learners’ navigation efficiency and overall satisfaction with the learning experience?

The study employs the Technology Acceptance Model (TAM) to validate the discussion forum design model (Cheung and Vogel, 2013). TAM, initially introduced in 1989, explores technology adoption based on two variables: Perceived Usefulness and Perceived Ease of Use (Davis, 1989). In this study, Perceived Usefulness pertains to users’ belief in the redesigned discussion forum’s ability to improve the learning process, while Perceived Ease of Use relates to the ease of using the redesigned forum. A conceptual model has shown a significant relationship between Perceived Ease of Use and Perceived Interaction, affecting learners’ intention to use discussion forums (Abdullah and Ward, 2016). The interaction between learners and instructors contributes to positive behavioral intentions toward using discussion forums and MOOCs, creating a high-quality learning environment. The study proposes five hypotheses based on TAM:

H1: Perceived Usefulness of discussion forums will positively influence the attitudes of learners toward online discussions.

H2: Perceived Ease of Use will positively influence attitudes toward online discussion forums.

H3: Perceived Ease of Use will positively influence Perceived Usefulness.

H4: Attitudes toward online discussion forums will positively influence intention to use.

H5: Intention to use online discussion forums will positively influence asynchronous learning system usage.

2 Materials and methods

Designing an interactive education tool and evaluating learners’ behaviors’ while using the educational tool is a challenging task, especially if there is a high number of participating students. Therefore, this study was divided into six stages.

2.1 Targeted platform

We collected and extracted all the discussion forum data from an Indian MOOC platform, SWAYAM, approved by the University Grant Commission for classes with a large number of learners¹ and developed by the Ministry of Education, National Program on Technology Enhanced Learning, and Indian Institute of Technology Madras with the help of Google Inc. and Persistent Systems Ltd. along with nine national coordinators. SWAYAM means “itself” in Hindi and it loosely translates to “self-learning”; however, facilitators are also available to deal with the discussion forums. This platform provides all the courses free of charge except the examination, which comes along with certification and valid, transferable university course credits. It delivers all courses taught from ninth grade to post-graduate levels in India by well-known instructors from prestigious institutes (Ambadkar and Ambadkar, 2020). Most of the courses are in English. However, India is a country with diverse cultures and languages. Therefore, currently, the platform also provides 27 post-graduate courses in eight different Indian languages (Hindi, Gujarati, Marathi, Bangla, Telugu, Kannada, Malayalam, and Tamil). In our case study, we only selected courses available in English. Currently, approximately 28 million learners are enrolled in the SWAYAM platform, and 2,226,700 learners are registered for the examination. However, successful certified learners only made up about 50%, that is, 1,180,257 (https://swayam.gov.in/nc_details/ accessed on 6 November 2022). The ratio of enrolled learners to passed learners is a significant challenge. Therefore, it is important to design an interactive educational tool for a massive number of learners and courses.

2.2 Extracting discussion forum posts

The discussion forum of SWAYAM consists of unstructured comments. Comments may include questions, phrases, sentences, paragraphs, and replicated ones. Usually, a course in SWAYAM lasts for 12 weeks, and a learner can put up comments in an online discussion forum of a course they are registered in, which may be related to the content, quizzes, assignments, or inquiries. Data were extracted in text form using Python. The data consisted of real discussion forum posts with post ID, post title, learner ID, post content, post URL, and post duration. All the course materials included pre-recorded video lectures, transcripts, assignments, quizzes, and Google group discussion forums. The study adopted user-centred design (UCD) and learner-centred design (LCD) methods (Dhar and Yammiyavar, 2012) to address the needs of users in the context of the learning system, with the involvement of both learners and users (Hasani et al., 2022). Table 1 lists the adopted UCD and LCD approaches. Considering the UCD and LCD approaches, the study followed five steps:

TABLE 1

Table 1. UCD and LCD approach (based on Dhar and Yammiyavar, 2012).

Step 1: Specifying the context of use by classifying posts and literature study;

Step 2: Identifying user and learner requirements;

Step 3: Designing the interactive design;

Step 4: Prototyping;

Step 5: Design evaluation.

2.3 Labeling the posts from a massive discussion forum

Specifically, we exported data from the academic writing course from SWAYAM because our recruitment of participants was from research backgrounds. We adopted a Latent Dirichlet Allocation (LDA) technique to understand discussion forum posts (Cummins and Nambudiri, 2022). The LDA model is an unsupervised technique widely used for analyzing content with the categorization of similar posts in discussion forums (Huang et al., 2021; Peng et al., 2021). Each extracted topic is a compact semantic cluster and works with the low similarity between different labels or topics (Huang and Wang, 2021). Therefore, the topics were labeled manually based on the highest probability (prob) of a word occurring in the topic itself. We used the tf-idf method to calculate the weight of each keyword. Among the 819 posts, we classified themes as Content related (CR) and Non-Content Related (NCR). However, the eight highest frequency words from five relevant topics, namely manuscript (52), journal (52), h-index (27), plagiarism (22), and publish (14) under the category of CR, were considered, as shown in Table 2. However, as all keywords from the CR category can be interrelated, it depends on the instructor to decide how many categories are necessary according to the frequency of posts and can be maintained by MOOC design facilitators. Similarly, Table 3 describes the three NCR-relevant topics having the 10 highest frequency words that were considered to represent the NCR topics, namely assignment (152), examination (194), and distress (32). In the NCR categorization, the three relevant topics with the 10 highest frequency words were considered. This study thus tracked the interest of learners in a particular topic.

TABLE 2

Table 2. Word probabilities within topics created by the LDA model for CR.

TABLE 3

Table 3. Word probabilities within topics created by the LDA model for NCR.

2.4 Recruitment of participants

We selected three unique combinations as criteria to recruit participants from our institute-JAIST, as shown in Table 4. The first combination was adopted to examine the perspectives of education and design. As our laboratory (KIM LAB) has an excellent student with a background in educational research and design, we selected all English-speaking students. The second combination was of Indian students at our university and the MOOC platform to observe their familiarity with the system. However, the ratio was 50/50. Although some Indian participants were already familiar with the SWAYAM platform. However, they had no previous experience with the SWAYAM discussion forum. The third combination, making up a majority of the recruited participants, was from diverse countries with diverse experiences and ideas from international participants.

TABLE 4

Table 4. Criteria for recruitment of participants.

Table 5 summarizes the participants’ demographics and backgrounds. A total of 28 participants from China, India, Japan, Myanmar, Vietnam, Bangladesh, and Thailand, participated in the experiment, of whom 18 were male and 10 females. Regarding age, most participants were 25–30 years old; only 4 were aged 35–40. Most of the participants were Ph.D. students; seven were in their master’s studies, and only three were post-doctoral students. Participants’ experiences with the discussion forum were classified into three categories: experienced, not experienced, and unsure or did not remember whether they used the discussion forum previously or not. Furthermore, three groups were categorized based on their experience of asynchronous discussion forums: 13 were experienced (G1), 7 had no prior experience (G2), and 8 were unsure whether they used the asynchronous discussion forum in the past (G3). However, all participants had a research background from the same university.

TABLE 5

Table 5. Participants’ demographics and background.

2.5 Design of the interactive discussion forums for conducting navigation experiment

2.5.1 Design of the question sheet

The question sheet used while conducting the navigation experiment was designed based on the BRUSO model, as it is effective for designing a question sheet as well as a survey questionnaire sheet (Peterson, 2000). However, in the navigation experiment, we also needed to select certain questions that should be easy to understand by the recruited participants in terms of concision, relevancy, clarity, subject-specificity, and intent toward the goal. Therefore, we implemented the BRUSO model while selecting questions from the original discussion forum asked by real learners.

• Brief: Ignoring long questions (seven to eight sentences).

• Relevant: Considering only those queries that had a response from the instructor or other learners.

• Unambiguous: Asking six questions depending on the time available for the experiment.

• Specific: Select only those questions classified as examination, content-related, assignment, and certificate.

• Objective: Make sure each question consists of different words.

As we needed to test the keyword selection by participants and usability using the searching technique, we implemented the Prisma method along with the BRUSO model to select questions with the following specific conditions:

• IDENTIFICATION:

Total posts in the discussion forum (n = 819).

Initial six pages only (n = 180).

• SCREENING:

Based on classified keywords, namely examination, content-related, assignment, and certificate.

Several posts related to a particular keyword.

• INCLUDED:

Repetitive queries (similar kinds of questions asked by other learners in different formats).

Selected only popular posts (n = 6) depending upon the views and BRUSO model, as shown in Figure 1.

FIGURE 1

Figure 1. Design of question sheet.

2.5.2 Design and prototypes of the three discussion forum patterns

The main objective of designing an information architecture is to present information that is easy to navigate and understand. To achieve this objective, the context (what we are presenting) and the user (who is processing it) work parallel (Santoso, 2018). Furthermore, four design components were formulated to achieve our discussion forum design goals:

1) the organization of discussion forum posts.

2) a labeling system of discussion forum posts.

3) a navigation system to browse the posts.

4) a search system to search the posts (Neha and Kim, 2023).

Therefore, the discussion forum design was based on nine features related to information visualization and the gestalt principle (Jacob, 2009; Chen, 2010). The nine selected features were platforms used for creating a discussion forum, color theme, alignment of categories, arrangement of categories, repetition of posts, navigation tools, icons, classification of posts, and locking systems. Table 6 describes the three types of discussion forum designs that were delivered to each recruited participant in the experiment.

• Original discussion forum (D1)

TABLE 6

Table 6. Three patterns of the discussion forum.

The discussion forum was already available in the Google Group discussion forum, as shown in Figure 2. There were 819 comments with repeated queries. The combination of the colors was mainly black and white: the background color was white, and the learners’ posts were black. Furthermore, there was a combination of red and blue to highlight some of the exam-related information that was not eye-catching. None of the participants noticed the exam information on d1. All posts were vertically aligned according to the posting date. A navigation tool was provided, but with limitations.

• Redesigned Discussion Forum 1 based on scrolling (D2)

FIGURE 2

Figure 2. The original design of discussion forum in google groups (D1).

We selected four specific design elements in redesigned discussion forum 1:

1. The discussion forum was created in Google Forms to check the scrolling feature derived from D1, as shown in Figure 3. In D2, questions were chosen from the first three pages (up to 90 questions) of the real discussion forum of the academic writing course.

2. The combination of color themes in the discussion forum was only black and white.

3. All posts were vertically aligned and distributed among their desired sections. Each section was ordered alphabetically, and questions with answers were set in the topmost position of each specified section.

4. Some posts were a combination of queries asking two or more questions in the same post, such as assignment-and certificate-related posts; in this case, we considered the initial part of the question. A total of 66 questions were set in the discussion forum in five sections: assignment (7), certificate (25), content-related (4), exam (27), and technical error (3).

• Redesigned Discussion Forum 2 based on categorization label (D3):

FIGURE 3

Figure 3. Redesigned Discussion Forum in Google Forms (D2).

We selected four specific design elements in redesigned discussion forum 2:

1. The discussion forum was customized using Wix software with a locking feature, as shown in Figure 4. The locking features aid in locking the questions which does not require further discussions.

2. The color theme was a black background, white text, and pastel pink links.

3. The alignment of posts (17) was horizontal and alphabetical.

4. All the posts were categorized, and further subcategories were defined with no repetitive posts according to the suggested design principle for asynchronous discussion in MOOC based on the Community of Inquiry (CoI) framework [38]. However, we asked participants to first check the category. If the participant was not able to find the post, then the participant was allowed to use the search technique. Only one of the participants required this search technique.

FIGURE 4

Figure 4. Redesign discussion forum in WIX (D3).

2.6 Conducting navigation experiment

Previous research has reported that navigation, i.e., finding a topic to discuss, is the most difficult task for the usability of a forum platform. A study conducted on OpenEdX reported that navigation was the main usability issue of the forum platform and that the task of searching for discussions of interest to post new questions was difficult. Moreover, navigation within a discussion forum with a large number of replies was found to be much more time-consuming. Another issue of current search tools or functions is that they do not improve the navigation process because of typing issues, scrolling, the nonexistence of spell checkers, and many more. Therefore, this study focused on conducting a navigation experiment with a design-based approach along with Gestalt principles and developing a classification method that distinguishes between CR and NCR posts by using an anticipatory data reduction method to evaluate the study efficiently and effectively with reliable results (Ntourmas et al., 2019, 2021).

Figure 5 describes the execution of the navigation experiment. During this experiment, a hard copy of a question sheet consisting of six questions (taken from an existing discussion forum) was provided to all the participants. The participant’s task was to navigate the provided questions from the discussion forum and later copy and paste them into a soft copy of the answer sheet. In the end, all the participants filled out a google feedback form including the merits, demerits, design ideas, time taken during answering the questions, and number of clicks and scrolls done during searching.

FIGURE 5

Figure 5. Procedure of navigation experiment.

2.6.1 Learner’s experience and interactions with the three discussion forum patterns

Navigation was the task chosen for the experiment and three discussion forums were delivered to each participant for navigating the set of six questions (question sheet). The sequence of distributing discussion forums to participants was based on their preferences. During the experiment three variables were measured through Python programming and later analyzed by SPSS software (Statistical Package for Social Sciences):

1. Time taken in navigating questions from three different patterns of the discussion forum (t1, t2, t3);

2. number of clicks (c1, c2, c3) clicked while searching the questions; and

3. The number of scrolls (s1, s2, s3) performed by each participant in the three different discussion forums (D1, D2, and D3), respectively.

2.6.2 Organization and usability issues of patterns of discussion forum

Participants evaluated the representation of the design or pattern of the three discussion forums under three themes: usability demerits, merits, and design ideas of each discussion forum using a Google form. The feedback from Google Form was taken to ensure the validity and reliability of our results.

3 Results

3.1 Learner’s behavior and interactions with the three discussion forum patterns

A significant difference was found between the three discussion forums. The result was significant at the p < 0.05 level. The One-Sample test revealed a significant difference between the time taken (t), the number of clicks (c), and the number of scrolls (s) performed by participants while navigating in D1, D2, and D3. Altogether, what particularly stands out is the time taken by participants in three different patterns of discussion forums. The time gap is worth highlighting, as t1 > t2 > t3 as shown in Table 7. The observed increase in t1 could be interpreted as the number of clicks and number of scrolls while navigating questions in the discussion forum. Therefore, we were also interested in tracking the number of clicks and scrolls which were also important in the navigation study. Further analysis showed that c1 > c3 > c2 (see Table 8) and s2 > s3 > s1 (see Table 9). Also, the analysis did not show any significant differences between G1, G2, and G3 at p < 0.05 as calculated by using the relative frequency contingency test on the mean difference of time taken in three discussion forums as shown in Table 10.

TABLE 7

Table 7. One-sample t-test for time taken in D1, D2, and D3.

TABLE 8

Table 8. One-sample t-test for number of clicks in D1, D2, and D3.

TABLE 9

Table 9. One-sample t-test for the number of scrolls in D1, D2, and D3.

TABLE 10

Table 10. Time-frequency table for the three group.

3.2 Organization and usability issues in the three design patterns

This section describes the three themes of usability issues in the three discussion forums: (1) Design Demerits, (2) Design Merits, and (3) Design Ideas. Participants reported maximum design demerits for D1, as shown in Table 11. In addition, participants reported D1 as the most difficult for navigating posts, which suggested that more time was needed on a platform with non-interactive features. However, scrolling on the D2 platform was a major challenge for the participants.

TABLE 11

Table 11. Design demerits in D1, D2, and D3.

Tables 12, 13 present the reported design merits and design ideas, respectively. Most of the design merits (15) reported by the participants were for D3. The top three design merits were categorization of posts (10), easy to navigate (9), and searching option (8). All the creative and innovative design ideas reported by participants are relevant when designing asynchronous discussion forums according to the learner’s perspective. Most of the design ideas recommended by the participants in D1 and D2 had already been implemented in D3 such as adding a categorization of posts, adding a short Q&A for NCR, adding an Icon button, adding pinned features, adding the most frequent question according to time relevancy as well as following the literature. However, seven additional innovative design ideas which can be implemented by MOOC designers were suggested by the participants.

TABLE 12

Table 12. Design merits in D1, D2, and D3.

TABLE 13

Table 13. Design Idea in D1, D2 and D3.

4 Discussion

This study investigated the impact of the original and effectiveness of a redesigned discussion forum in MOOC. The results demonstrated that the redesigned discussion forum is more efficient and adopted by participants based on the classification of discussion forum posts and the design of discussion forum patterns.

1. Classified or organized discussion forum posts to improve learner engagement and interaction efficiency: This addresses the first key factor mentioned in the study, which is the classification of discussion forum posts. The study investigated the strategies and methods that can be employed to categorize and structure discussion forum posts in a way that benefits learners. During the navigation experiment, participants took less time to search the question in the redesigned discussion forum because of the categorization of posts. The analysis of learners’ feedback showed that the categorization of posts, easy-to-navigate, and searching options in redesigned discussion forums were important features and should be included in any adopted discussion forum in MOOCs to increase interaction efficiency and learning engagement. Therefore, adopting the post-classification strategy increases content interaction, and further enhances the motivation of learners to interact with instructors and their learning engagement.

2. Specific design patterns for discussion forums in MOOCs that enhance the overall learning experience, and impact learner engagement and usability: The second factor mentioned in the study is the design of different discussion forum patterns. It seeks to explore the various design patterns and their effects on learner engagement, usability, and satisfaction. All the feedback from the participants was collected regarding the usability of the discussion forum and to increase interactivity. The study compared the design of the original discussion forum and the redesigned discussion forum with all the design demerits, merits, and ideas that can be adopted by MOOC designers to improve the usability of discussion and enhance the learners’ engagement. Interestingly, participants also mentioned innovative design ideas that can be adopted by MOOC designers. The innovative design ideas were adding color for different categories, adding symbols, bold font, upvotes, more keyword accession, better categorization, and graphics.

3. Implementation of Learner-centered design elements and usability improvements in MOOC discussion forums impact learners’ navigation efficiency and overall satisfaction with the learning experience: This research question encompasses the third key factor mentioned in the study, which is the evaluation of learners’ perspective. It aims to assess the impact of learner-centered design with improvements on learners’ navigation efficiency and their overall perception of the MOOC. Through a navigation experiment, this study investigated the perceived demerits and merits of existing and redesigned discussion forum design and learners’ design ideas to implement based on learners’ perspectives in order to improve the quality of the learning process in MOOCs as shown in Figure 6. The results of this study reveal that adopted design factors reduced the time taken by learners to explore the discussion forum. For instance, the categorization of posts and search features were found to be useful in exploring discussion forums. Although the search option was available in the existing discussion forum, participants encountered difficulty while typing the correct keyword to search for a post and ended up with a scrolling feature which was more troublesome in discussion forums. The participants who took the maximum time while navigating questions typed several combinations of keywords, showing that these users or learners could not judge which keyword was suitable for searching the post. Therefore, the classification of posts aids in resolving the confusion of learners when exploring discussion forums. Several design ideas were recommended by the participants which can enhance the interactivity with the system. Education MOOC designers can thus adopt the findings of this study to enhance the interactivity among learners and instructors, as they need to pay more attention to user interactivity in online learning to encourage learners and instructors.

FIGURE 6

Figure 6. User feedback in existing and redesigned discussion forum of MOOC.

The number of reported merits or positive feedback points for the redesigned discussion forum (D3) was 41 including the novice discussion forum users, and the frequency of negative feedback points was 6. However, there were only 8 positive and 25 negative feedbacks for the existing discussion forum based on the perceived usefulness, as shown in Table 14. A chi-square test was thus conducted to measure the significance of the redesigned discussion forum in enhancing perceived usefulness and ease of use. The association between the design of the discussion forum and comments showed extremely statistically significant results, with a two-tailed value of p of less than 0.0001.

TABLE 14

Table 14. Contingency table of feedback for the design of the discussion forum.

5 Theoretical and practical significance of the study

All scientific research intends to contribute to the body of knowledge or a theory in the research domain. To achieve this knowledge contribution, the research study encounters two phases that majorly revolve around scientific research. The first phase is the search for understanding and the second phase is for knowing. Improving the current education system, decision making and policy development in the field of education are other factors that contribute to educational research.

The research study contributed the majority to educational design research (EDR). EDR focuses on educational problems in real-world situations with two primary goals for the improvement of education. The first one is to develop knowledge through understanding the concepts, and the second one is to develop solutions through acquiring skills and expertise. EDR is considered a powerful approach because it addresses real needs through the development of a solution to a problem, while also generating knowledge for future development. EDR features the collaboration between stakeholders (e.g., researchers, educational designers, instructors, and learners) to simultaneously develop new theoretical insights into learners’ motivational behavior and practical solutions to serious teaching and learning challenges (McKenney and Reeves, 2021). The Design Research suggests researchers and practitioners have the opportunity to produce interventions of real value with tools, approaches, theories, and products tested in the field and proven to be effective. In this research, we focused on the real problem of interaction among learners and instructors in the discussion forum of MOOCs and provides process information on emerging learning technologies. We implemented several tools-machine learning and Atlas.ti; approaches-UCD and LCD; theory such as CoI framework and products-google forms, google groups, WIX, and TAM to approve the learning system design effectiveness. EDR extends theoretical knowledge through data collection and analysis.

The process of EDR systematically follows three main phases with multiple time repetitions till achieving the desired goal:

1. Analysis-In this study, we analyzed three cases to analyze the whole structure of the discussion forum with different perspectives using various tools and education models.

2. Design-The study adopted educational design using UCD and LCD approaches to find a better solution. Further, the existing design and adopted design were compared for a better learning process and interaction between instructors and learners.

3. Evaluation-Three most crucial statistical factors were measured according to the literature review. In addition, qualitative data were collected to evaluate and validate the redesigned discussion forum.

The Design based research (DBR) model adopted in 2008 (Amiel and Reeves, 2008) enhances understanding about the nature of learning and what facilitates it. The following characteristics are followed in DBR approach (McKenney and Reeves, 2021):

1. Continuous cycle of design, evaluation, and redesign.

2. Real-life learning settings.

3. Aimed at both testing and refining theories and advancing practice.

4. Characterized by mixed-methods studies.

5. Involves designers, researchers and practitioners with different expertise who interact frequently to guide the design, conduct and reporting of DBR.

This study provides a pathway to follow for solving educational design research.

6 Limitations and future scope

This study has three primary limitations. First, this research study was especially focused on the learner-centered approach which eventually can aid in the instructor’s task. However, this doctoral study raised a question regarding the fundamental reasons behind the unbalancing in the learner side and instructor side in the discussion forum. As this study is more inclined toward a learner-centered approach rather than aiding the instructor but in the future, there is a potential to focus on both instructor and learner at the same time. Further investigation can be done by the educational research community through instructor interview or facilitator approach to enhance the research study which supports both instructor and learner at the same time. Secondly, we were not able to experiment with real users/learners of MOOCs or specifically SWAYAM users because of COVID-19 border restrictions. We were also not able to reach SWAYAM facilitators which can be a future approach and improve the design ideas of asynchronous discussion forums according to real situations or demands.

During the navigation experiment, some of the participants were confused with some keywords used in the question sheet. For instance, the word “admit card” which is defined differently in different countries. Although the task was only to search the question in the discussion forum. However, keywords can be explained in the manual of the experiment in future research. Also, the MOOC platform needs to be careful regarding the words mentioned in the consideration of international students. Moreover, the study provides a good set of data that can be used for further analysis. For instance, the set of keywords used by participants while searching posts can be analyzed as follows:

1. Which set of keywords were used more frequently by the participants?

2. Is there any link or connection between the most frequent combination of keywords and time taken by the participant?

3. Is the most frequent combination of keywords adequate to search so that the researcher or facilitator can use them to categorize pre-asked or frequently asked questions?

4. Does the sequence of keywords also matter during searching?

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

Ethical approval was not required for the study involving human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was not required from the participants in accordance with the national legislation and the institutional requirements.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This research was partially funded by KAKEN22K13754.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^https://swayam.gov.in/nc_details/

References

Abdullah, F., and Ward, R. (2016). Developing a general extended technology acceptance model for E-learning (GETAMEL) by analysing commonly used external factors. Comput. Hum. Behav. 56, 238–256. doi: 10.1016/j.chb.2015.11.036

CrossRef Full Text | Google Scholar

Ambadkar, R., and Ambadkar, R. S. (2020). E-learning through SWAYAM MOOCs-awareness and motivation among commerce students minor project-capital structure and profitability of foreign direct investment companies in the Indian manufacturing sector view project research project on-sustainability reporting and corporate financial performance: a study of S&P BSE Sensex companies view project E-learning through SWAYAM MOOCs-awareness and motivation among commerce students. Int. J. Sci. Technol. Res. 9:2.

Google Scholar

Amiel, T., and Reeves, T. C. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Educ. Technol. Soc. 11, 29–40.

Google Scholar

Baber, H. (2021). Modelling the acceptance of e-learning during the pandemic of COVID-19-a study of South Korea. Int. J. Manag. Educ. 19:100503. doi: 10.1016/j.ijme.2021.100503

CrossRef Full Text | Google Scholar

Castro, M. D. B., and Tumibay, G. M. (2021). A literature review: efficacy of online learning courses for higher education institution using meta-analysis. Educ. Inf. Technol. 26, 1367–1385. doi: 10.1007/s10639-019-10027-z

CrossRef Full Text | Google Scholar

Chauhan, J., and Goel, A. (2015). ‘An analysis of video lecture in MOOC’. Available at: https://www.miriadax.net/ (Accessed: 7 September 2023).

Google Scholar

Chen, C. (2010). Information visualization. Wiley Interdisc. Rev. Comput. Statis. 2, 387–403. doi: 10.1002/wics.89

CrossRef Full Text | Google Scholar

Cheung, R., and Vogel, D. (2013). Predicting user acceptance of collaborative technologies: an extension of the technology acceptance model for e-learning. Comp. Educ. 63, 160–175. doi: 10.1016/j.compedu.2012.12.003

CrossRef Full Text | Google Scholar

Chiu, T. K. F., and Hew, T. K. F. (2018). Factors influencing peer learning and performance in MOOC asynchronous online discussion forum. Aust. J. Educ. Technol. 34, 16–28. doi: 10.14742/ajet.3240

CrossRef Full Text | Google Scholar

Cummins, J. A., and Nambudiri, V. E. (2022). Natural language processing: a window to understanding skincare trends. Int. J. Med. Inform. 160:104705. doi: 10.1016/J.IJMEDINF.2022.104705

PubMed Abstract | CrossRef Full Text | Google Scholar

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quar. Manag. Inf. Syst. 13, 319–339. doi: 10.2307/249008

CrossRef Full Text | Google Scholar

Dhar, D., and Yammiyavar, P. (2012). “Design approach for E-learning systems: should it be user centered or learner centered.” In Proceedings −2012 IEEE 4th international conference on Technology for Education, T4E 2012.

Google Scholar

Ezen-Can, A., Boyer, K. E., Kellogg, S., and Booth, S. E. (2015). ‘Unsupervised modeling for understanding MOOC discussion forums: a learning analytics approach’, in ACM International Conference Proceeding Series. Association for Computing Machinery.

Google Scholar

Frau-Meigs, D., Osuna-Acedo, S., and Marta-Lazo, C. (2021). MOOCs and the participatory challenge. Springer Cham.

Google Scholar

Gamage, D., Perera, I., and Fernando, S. (2020). MOOCs lack interactivity and collaborativeness: evaluating MOOC platforms. Int. J. Eng. Pedag. 10, 94–111. doi: 10.3991/ijep.v10i2.11886

CrossRef Full Text | Google Scholar

Hasani, L. M., Santoso, H. B., and Junus, K. (2022). Designing asynchronous online discussion forum Interface and interaction based on the Community of Inquiry Framework. Int. Rev. Res. Open Dist. Learn. 23, 191–213. doi: 10.19173/irrodl.v23i2.6016

CrossRef Full Text | Google Scholar

Huang, Y., and Wang, R. (2021). ‘Analysis of influencing factors of MOOC learners’ Loyalty based on online review text mining’. In 2021 International Conference on Electronic Information Engineering and Computer Science, EIECS 2021. Institute of Electrical and Electronics Engineers Inc.

Google Scholar

Huang, Y., Wang, R., Huang, B., Wei, B., Zheng, S. L., and Chen, M. (2021). Sentiment classification of crowdsourcing participants’ reviews text based on LDA topic model. IEEE Access 9, 108131–108143. doi: 10.1109/ACCESS.2021.3101565

CrossRef Full Text | Google Scholar

Jacob, E. K. (2009). Information architecture 43; ISBN 9781573873406., 1–64. doi: 10.1002/aris.2009.1440430110,

CrossRef Full Text | Google Scholar

Jung, I., Choi, S., Lim, C., and Leem, J. (2002). Effects of different types of interaction on learning achievement, satisfaction and participation in web-based instruction. Innov. Educ. Teach. Int. 39, 153–162. doi: 10.1080/14703290252934603

CrossRef Full Text | Google Scholar

Ma, Z., Idris, S., Zhang, Y., Zewen, L., Wali, A., Ji, Y., et al. (2021). The impact of COVID-19 pandemic outbreak on education and mental health of Chinese children aged 7–15 years: an online survey. BMC Pediatr. 21:95. doi: 10.1186/s12887-021-02550-1

PubMed Abstract | CrossRef Full Text | Google Scholar

McKenney, S., and Reeves, T. C. (2021). Educational design research: portraying, conducting, and enhancing productive scholarship. Med. Educ. 55, 82–92. doi: 10.1111/medu.14280

PubMed Abstract | CrossRef Full Text | Google Scholar

Moore, M. G. (1989). Editorial: three types of interaction. Am. J. Dist. Educ. 3, 1–7. doi: 10.1080/08923648909526659

CrossRef Full Text | Google Scholar

Muthuprasad, T., Aiswarya, S., Aditya, K. S., and Jha, G. K. (2021). Students perception and preference for online education in India during COVID-19 pandemic. Soc. Sci. Human. Open. 3:100101. doi: 10.1016/j.ssaho.2020.100101

PubMed Abstract | CrossRef Full Text | Google Scholar

Neha,, and Kim, E. (2020). “Designing discussion forum in SWAYAM for effective interactions among learners and supervisors” in Communications in Computer and Information Science. eds. C. Stephanidis, M. Antona, and S. Ntoa (Berlin: Springer Science and Business Media Deutschland GmbH)

Google Scholar

Neha,, and Kim, E. (2021a). “A classification method of the learners’ queries in the discussion forum of MOOC to enhance the effective response rate from instructors” in Communications in Computer and Information Science. eds. C. Stephanidis, M. Antona, and S. Ntoa (Berlin: Springer Science and Business Media Deutschland GmbH)

Google Scholar

Neha,, and Kim, E. (2021b). ‘Investigating responsible factors for interaction between learners and instructors in the discussion forum of MOOC’. In 2021 9th International Conference on Information and Education Technology, ICIET 2021. Institute of Electrical and Electronics Engineers Inc.

Google Scholar

Neha,, and Kim, E. (2023). ‘Relevant interaction among learners and instructors in asynchronous academic writing course’. In 2023 11th International Conference on Information and Education Technology, ICIET 2023. Institute of Electrical and Electronics Engineers Inc.

Google Scholar

Ntourmas, A., Avouris, N., Daskalaki, S., and Dimitriadis, Y. (2019). ‘Evaluation of a massive online course forum: design issues and their impact on learners’ support’, in Lecture notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag.

Google Scholar

Ntourmas, A. I., Dimitriadis, Y., Daskalaki, S., and Avouris, N. M. (2021). ‘Classification of discussions in MOOC forums: an incremental modeling approach’, in L@S 2021- Proceedings of the 8th ACM Conference on Learning @ scale. Association for Computing Machinery, Inc.

Google Scholar

Peng, X., Xu, Q., and Gan, W. (2021). SBTM: a joint sentiment and behaviour topic model for online course discussion forums. J. Inf. Sci. 47, 517–532. doi: 10.1177/0165551520917120

CrossRef Full Text | Google Scholar

Peterson, R. (2000). Constructing effective questionnaires. SAGE Publications, Inc.

Google Scholar

Rahiem, M. D. H. (2021). Remaining motivated despite the limitations: university students’ learning propensity during the COVID-19 pandemic. Child Youth Serv. Rev. 120:105802. doi: 10.1016/j.childyouth.2020.105802

CrossRef Full Text | Google Scholar

Ramesh, A., Goldwasser, D., Huang, B., Daume III, H., and Getoor, L. (2013). Modeling learner engagement in MOOCs using probabilistic soft logic.

Google Scholar

Reich, J., and Ruipérez-Valiente, J. A. (2019). INSIGHTS sciencemag.org SCIENCE. doi: 10.1145/3051457.3053980,

CrossRef Full Text | Google Scholar

Santoso, H.B. (2018) The development of a learning dashboard for lecturers: a case study on a student-centered e-learning environment.

Google Scholar

Singh, J. P., Singh, L., and Matthees, B. (2022). Establishing social, cognitive, and teaching presence in online learning—a panacea in COVID-19 pandemic, post vaccine and post pandemic times. J. Educ. Technol. Syst. 51, 28–45. doi: 10.1177/00472395221095169

CrossRef Full Text | Google Scholar

Swan, K. (2001). Virtual interaction: design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Educ. 22, 306–331. doi: 10.1080/0158791010220208

CrossRef Full Text | Google Scholar

Terras, M. M., and Ramsay, J. (2015). Massive open online courses (MOOCs): insights and challenges from a psychological perspective. Br. J. Educ. Technol. 46, 472–487. doi: 10.1111/bjet.12274

CrossRef Full Text | Google Scholar

Wu, B. (2021). Influence of MOOC learners discussion forum social interactions on online reviews of MOOC. Educ. Inf. Technol. 26, 3483–3496. doi: 10.1007/s10639-020-10412-z

CrossRef Full Text | Google Scholar