“Hello, My Name Is Melinda” – Students' Views on a Digital Assistant for Navigation in Digital Learning Environments; A Qualitative Interview Study

Introduction

Progressing digitalisation is finding its way into everyday live, becoming second-nature to the so-called generation of “digital natives” (Prensky, 2001). Personal digital assistants (PDA) like Alexa, Siri, Cortana, or Google Assistant, are a readily accepted part of today's student life, seeking to offer support for most situations in everyday life. Byte magazine, cited in Bayus et al. (1997) proposed the more or less vague definition of PDAs as being “highly portable, easy-to-use computing and communication devices aimed at the mass market.” In addition, by Bayus et al. stated “The purpose of a PDA is to help people communicate more effectively and efficiently in an era of global mobile communications. Furthermore, a PDA should assist people in managing all their information needs. Key functions are thus communications and intelligent assistance.” No clear definition of what a PDA is exactly or what functions justify the term PDA for an electronic device seems to exist.

With the rapid progression of digitalization, similar changes can be found in the context of higher education. Classes are more and more offered as blended-learning formats or even as online modules altogether with massive open online courses as the most prominent example (e.g., a course by Sebastian Thrun on artificial intelligence at Stanford University in 2011, with 160,000 students from over 190 different countries being enrolled for the course (Siemens, 2015). Accordingly an increasing use of internet-based learning can be observed (Cook et al., 2008).

It is only natural to include such PDAs as digital navigators into digital learning environments (DLE) – as already having been done by several deaneries to offer their students a professional support system (Kavakli et al., 2012; Ghose and Barua, 2013; Ranoliya et al., 2017; Dibitonto et al., 2018).

Many universities are already experimenting with solutions to offer help to students under stress. The following main findings can be noted:

(1) Student counseling helps students, but usually does not reach the severe cases or only with difficulty (Kavakli et al., 2012).

(2) Chat users indicate two key motivations for using a bot: productivity followed by entertainment (Brandtzaeg and Følstad, 2018).

(3) Students still prefer telephone, mail or face-to-face communication with their peers for important questions, but trust bots especially when it comes to up-to-date information (Dibitonto et al., 2018).

One of the biggest hurdles for the durable and motivated use of such bots is above all the necessary gain of information, which the students must obtain when using the bots. Many of the early approaches of assistance systems have lost user acceptance because they were not able to provide real added value for the users. A prominent example of this is certainly the widely known clip from Microsoft, a bot that was ahead of its time in terms of user interaction, but on the other hand did not have much content to offer (Dey, 2009). However, there are already efforts to make lecturer knowledge available for the engines (Ghose and Barua, 2013; Ranoliya et al., 2017).

This whole issue of digitalisation might pose benefits such as more work flexibility and better adaptation to individual needs, but also potentially harbors the danger of posing additional stressors, such as increased isolation of individuals (Wildemeersch and Jütte, 2017) or technical problems (Islam et al., 2015). This seems particularly important when one takes into account that students are alarmingly stress-prone with some even showing full-scale psychological conditions, such as anxiety disorders, depression or suicidality (Verger et al., 2009; Stallman, 2010). This has specifically been shown among medical students (Erschens et al., 2018a,b) and students of the science, technology, engineering and math (STEM) subjects (Madany Mamlouk et al., 2018) where studies are generally considered very challenging and demanding.

If PDAs are to be successfully included into the everyday lives of students, they need to be extremely user-friendly and particularly tailored to the user group, making routine usage attractive and not posing additional stress factors. It is well-known that such features significantly contribute to user acceptability, and in turn lead to consequent usage, for example as depicted in the technology acceptance model (TAM) (Venkatesh and Davis, 2000).

We have shown that there are many promising approaches to bringing conversational agents to universities. There is both the therapeutic approach, which takes intensive care of its conversational agents, and the FAQ approach, which is primarily about contact points for quick information. There is not yet a single integrated approach in which an empathic chatbot accompanies the students and, in addition to providing a lot of helpful information about their studies, also provides a further social anchor for a balanced and healthy study.

This is where our project comes in. In the coming years, a conversational PDA called Melinda (My Emotion-screening, Learning-supportive, Integrated New Digital Assistant) should bring these loose ends together and be a valuable and reliable partner for the students, providing them with information and trustworthy support. In order to start the developmental process of Melinda, this study aims to get students' views on two major issues: First, it seeks to get their opinions on working in a Digital Learning Environment (DLE) in general, and specifically with regards to potential new stressors accompanying digitalisation in higher education. Second, it strives to determine what students consider relevant in a PDA serving as a student support system in order to be accepted and used by them.

The bot must be able to bind the students so comprehensively to the use of the system that the emotional component has enough opportunity to ask empathic questions without the bot being shut down immediately by too many users after an initial phase. It is therefore essential to get a clear idea of the requirements and expectations that a specialized user group has of the bot. In this study, we have therefore concentrated on a group of students with a more technical background: German students from the STEM subjects.

Methods

Design

This study presents a qualitative research design of semi-structured interviews. Qualitative research methods are particularly helpful when wanting to get new insights or understand underlying concepts, which can in turn be used for further quantitative prospective studies. They are frequently used in social science and medical research and data collection follows strict rules comparable to those known in quantitative research projects (Dicicco-Bloom and Crabtree, 2006; Mcgrath et al., 2018).

Participants

STEM students from the University of Luebeck, Germany, were asked to participate in the study. We asked for students who were interested in DLE or have already used DLE in general and in regards to their studies. They were all recruited via email. N = 12 students from various years of study took part in the survey. N = 3 were master students (one in the 1st semester, two in the 4th). N = 9 were bachelor students (one in the 1st semester, two in the 3rd, four in the 5th, and two in the 7th). They had various academic backgrounds including medical informatics, molecular life science, mathematics in medicine and life science, robotic and autonomous systems, infection biology, medical engineering and entrepreneurship in digital technologies.

Interviews

The interviews were conducted by a clinical psychologist experienced in qualitative research. Students were asked questions focusing on the DLE as well as stress-promoting or stress-preventing factors. Additionally, students' ideas on the development of a PDA were collected. Example questions of the interview are provided in Table 1 (For full interview guide, please see Supplementary Material).

TABLE 1

Table 1. Interview excerpts.

Furthermore, demographic information, such as gender, age, year of study, and course of study, was assessed.

The semi-structured interview guideline were constructed based on the SPSS-method (Sammeln [collect], Prüfen [check], Sortieren [sort], Subsumieren [subsume]) developed by Cornelia Helfferich (Helfferich, 2011). First, ideas for possible questions were gathered. Second, they were checked for coverage of the topic. Third, the questions were sorted by topic. Last, they were subsumed which means they were ranked and subordinated in the interview guideline.

All data were assessed pseudonymously. The interviews were audio recorded and later transcribed verbatim. Original recordings were deleted after transcription. Transcripts were stored on a secure computer with no connection to the internet.

Data Analysis

The goal of the qualitative content analysis of the interviews was to get an overall image of the arguments brought up by the interviewed subjects concerning factors of stress and coping related to their studies as well as working with DLE. The interviews were evaluated according to Mayring's qualitative content analysis (Mayring, 2010), following the author's general procedural model. In order to attain an overview of the opinions mentioned by the interviewed subjects, summarisation, more precisely inductive category formation, was determined as the technique of analysis. The procedure of inductive category development according to Mayring (2014) was performed by two independent raters, one as part of the team in Luebeck, the other from the team in Tuebingen.

In a first step, the goal of the analysis and the theoretical background were defined according to the interview guide of the semi-structured interviews. Three main topics (stress, coping and the DLE including properties of a possible PDA) were chosen as categories. In a second step, each of the raters started working through the material line by line and formed categories using MAXQDA Version 2018. In order to obtain the same level of abstraction in building the categories, the raters revised them together via video chat and agreed on the final categories by paraphrasing representative examples and building a hierarchy of categories. Following this, both raters independently worked through the material again.

Ethical Approval

Ethical approval was given by the ethics committee of the University of Luebeck (No.18-010). The participation was voluntary. All students provided their written informed consent.

Results

Sample

N = 12 STEM students were interviewed. The average age was M = 27.92 ± 6.44. 50% were female.

Categories Derived From the Interview Material

Based on the analysis of the verbatim transcribed interviews via the SPSS technique described above (see Methods section), the following areas could be distinguished: digital learning environments, stressors, preventive factors and digital assistant. For each of these areas, categories (see Figure 1) and subcategories (see Table 2; for full data please see Supplementary Material) could be identified and example quotations are listed for each subcategory. For full data please see Supplementary Material.

FIGURE 1

Figure 1. Themes derived from student interviews via SPSS method with according categories.

TABLE 2

Table 2. Examples and anchor quotations for each category and its subcategories.

Principal Results

In this study we report on students' views on a conversational DA as a supportive instrument in their everyday lives. From the present interview data, the following categories could be derived: students' general perception of the DLE with their advantages and challenges and specific stressors and preventive factors students perceive within a DLE in a more study-related / administrative as well as in a personal/emotional sense. Finally, students' ideas on characteristics of a new PDA were gathered (see Figure 1).

Despite the many advantages, especially regarding faster and more efficient communication within DLE, students also seem to be aware of drawbacks of increased digitalisation, such as information overload and reduced personal contact. Advantages and disadvantages regarding a DLE are generally reported according to the change of communication happening through progressing digitalisation and e-learning. Further, technical and legal issues were mentioned when students spoke about disadvantages they had experienced.

Stressors mentioned in the interviews were mainly reported in the context of either performance within the medical studies or organizational aspects of university life. Preventive factors were mostly related to motivation, interest and encouragement through feedback from others as well as time off from studies (social contacts and leisure time).

When specifically asked about a PDA, students agreed on basic features such an assistant should have. They stressed the need for simple operations within the system as well as a clear layout and the wish for direct links with other entities (e.g., direct email link to teacher). An important topic was the PDA's role as a kind of secretary. Students wanted the system to prioritize for them, filter information, plan dates and remind them of deadlines. However, students stressed the importance of self-determination within such a system. They wanted to be in control of, for example, remembering functions and general decisions and rebutted the idea of a system telling them rather than suggesting what to do.

Discussion

This study explored students' views on the DLE as well as necessary features of a PDA as a support system within this environment. Our data suggest that students had quite a balanced opinion on working within a DLE. They could reflect on potentials as well as challenges, and apply both to study-related aspects as well as more personal, emotion-related ones.

When asked for desired functions of a supporting PDA within such a DLE, students see potential for support in the area of administrative or study-related concerns, respectively (see Figure 2). They explicitly wish for functions such as time management and calendar, exam registration, recording, reminder-systems, and prioritization of information by the system – all of which should be considered when developing a PDA for students in the sense of a digital secretary. Such a PDA might be able to reduce administrative and study-related stressors particularly in challenging study subjects like STEM.

FIGURE 2

Figure 2. Filled arrows: paths having been described by students in the interviews; dashed arrow: new part to further explore when developing a PDA.

In light of this, students in the present study, in line with previous studies (Flynn and Macleod, 2015; Fares et al., 2016), have to withstand enormous pressure especially before and during exams. However, they usually develop successful strategies to deal with this stress. One factor, mentioned by many of our interviewees, was their thirst for knowledge, a driving force behind their enrolment at a university. Overall, such strategies seem to make most students resilient enough to deal with the academic stress they experience. However, students are more prone to stress-related conditions during their studies and might benefit from early integrated screening and supportive structures (Verger et al., 2009; Stallman, 2010).

This, in turn, led to the idea, that an ideal PDA might additionally be able to identify underlying problems when a student, for example, does not complete homework tasks or has not entered the system for a while. On this basis, it might serve as a helpful agent for screening and treatment of psychological impairments, such as symptoms of depression or anxiety, offering broad and easily available support mostly irrespective of time, location or even language (Fitzpatrick et al., 2017; Fulmer et al., 2018).

A “perfect” PDA within the professional environment should be able to combine these two characteristics. First, it should offer supportive information on routine tasks, such as deadlines for class homework or updates on changes of location. Second, it should simultaneously screen for psychosocial needs of the user and react accordingly. This must be embedded into the routine chat, for example, within the pre-class assignment online, as not all users will have a psychological problem in the first place, making a student-related PDA distinctly different to psychotherapeutic chat bots already in use (Kavakli et al., 2012; Pasikowska et al., 2013). This was mirrored by the fact, that within the interviews no one mentioned the idea of a PDA having the potential to support personal and emotional issues.

Insights gained by the semi-structured interviews and the above-outlined considerations, as also depicted in Figure 2, finally led to the preliminary design-idea for so-called “Melinda” – “my emotion-screening, learning-supportive, integrated new digital assistant” (see Figure 3).

FIGURE 3

Figure 3. Final idea for a new student PDA, Melinda.

However, there are some limitations. If one aims for such a combined PDA, some technical challenges have to be considered. A text- or voice-based request analyzed by a natural language processing (NLP) system can be processed much more efficiently than would be possible for an inquiry using an ordinary GUI-based system, simply because the information sought is already implicitly represented by the verbal request. Thus, digital infobots for university admission are a consistent first use in the context of higher education (Ghose and Barua, 2013; Ranoliya et al., 2017; Dibitonto et al., 2018). However, if communication is not about a mere database request, but rather on a personal or emotional subject, it still poses a great challenge for NLP systems to derive this context from the message. Recent NLP-based frameworks approach this problem using a so-called sentiment analysis (also known as “opinion mining”) that seems to be a promising step in the right direction (Cambria et al., 2013; Poria et al., 2016).

Second, we only interviewed students at one location (University of Luebeck, Germany) from the STEM subjects. Despite the fact that the latter consisted of various sub-subjects, most of them came from studies with a more or less direct link to digital aspects (i.e., computer science), which also limits generalisability. Finally, as always with qualitative research, the data analysis depends on interpretation of the raters and is thus, despite analyzing protocols, more prone to subjectivity.

Despite several technical challenges and potentially emotional barriers, these data presented only preliminary characteristics, and we strongly believe a PDA integrated into the usual DLE of students is a feasible and useful project. We particularly value the idea of not only addressing study-related issues, but also more personal situations and emotional states. As there are already validated reliable screening instruments for psychological burdens, the challenge will be to integrate these into normal everyday conversations.

During the intensive interviews, we gained the impression that both sides of Melinda can certainly strengthen each other: On the one hand a reliable and trustworthy companion, with whom one would like to talk gladly and frequently, and on the other hand a contact person with a comprehensive knowledge of important dates and available teaching and learning material adapted to my needs. In this combination we expect for the first time a setup that has the potential to bind users permanently and to be used in the long term.

The next step thus will be to test a Melinda prototype within the actual study setting to see how students interact with such a PDA, collect some first data in a wizard-of-oz-experiment, and then develop algorithms to individually match the user needs.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by Ethical approval was given by the ethics committee of the University of Luebeck (No. 18-010). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

AH-W and AM were responsible for the designed and conduction the study, as well as acquisition, analysis, and interpretation of data. AH-W drafted the first version of the manuscript. TL conducted the interviews and was involved in analyses and interpretation and revised the manuscript critically. FJ and SZ made substantial contributions to the study design and revised the manuscript critically. All authors read and approved the final version of the manuscript and agreed to be accountable for all aspects of the work.

Funding

This study was supported by the Stifterverband Germany within its Digital Transfer Fellowship programme. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We also acknowledge support with financing publication fees by Deutsche Forschungsgemeinschaft and Open Access Publishing Fund of the University of Tuebingen.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We acknowledge the support from Kathrin Riech and Maren Kissel with transcribing the data.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2020.541839/full#supplementary-material

Abbreviations

DLE, digital learning environment; NLP, natural language processing; PDA, personal digital assistant

References

Bayus, B. L., Jain, S., and Rao, A. G. (1997). Too little, too early: introduction timing and new product performance in the personal digital assistant industry. J. Market. Res. 34, 50–63. doi: 10.1177/002224379703400105

CrossRef Full Text | Google Scholar

Brandtzaeg, P. B., and Følstad, A. (2018). Chatbots: changing user needs and motivations. Interactions 25, 38–43. doi: 10.1145/3236669

CrossRef Full Text | Google Scholar

Cambria, E., Schuller, B., Xia, Y., and Havasi, C. (2013). New avenues in opinion mining and sentiment analysis. IEEE Intell. Syst. 28, 15–21. doi: 10.1109/MIS.2013.30

CrossRef Full Text | Google Scholar

Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., and Montori, V. M. (2008). Internet-based learning in the health professions: a meta-analysis. JAMA 300, 1181–1196. doi: 10.1001/jama.300.10.1181

PubMed Abstract | CrossRef Full Text | Google Scholar

Dey, A. K. (2009). Modeling and intelligibility in ambient environments. J. Ambient Intell. Smart Environ. 1, 57–62. doi: 10.3233/AIS-2009-0008

CrossRef Full Text | Google Scholar

Dibitonto, M., Leszczynska, K., Tazzi, F., and Medaglia, C. M. (2018). “Chatbot in a campus environment: design of LiSA, a virtual assistant to help students in their university life,” in International Conference on Human-Computer Interaction (Palma: Springer), 103–116. doi: 10.1007/978-3-319-91250-9_9

CrossRef Full Text | Google Scholar

Dicicco-Bloom, B., and Crabtree, B. F. (2006). The qualitative research interview. Med. Educ. 40, 314–321. doi: 10.1111/j.1365-2929.2006.02418.x

PubMed Abstract | CrossRef Full Text

Erschens, R., Herrmann–Werner, A., Keifenheim, K. E., Loda, T., Bugaj, T. J., Nikendei, C., et al. (2018a). Differential determination of perceived stress in medical students and high-school graduates due to private and training-related stressors. PLoS ONE 13:e0191831. doi: 10.1371/journal.pone.0191831

PubMed Abstract | CrossRef Full Text | Google Scholar

Erschens, R., Keifenheim, K. E., Herrmann-Werner, A., Loda, T., Schwille-Kiuntke, J., Bugaj, T. J., et al. (2018b). Professional burnout among medical students: Systematic literature review and meta-analysis. Med. Teach. 41, 172–183. doi: 10.1080/0142159X.2018.1457213

PubMed Abstract | CrossRef Full Text | Google Scholar

Fares, J., Al Tabosh, H., Saadeddin, Z., El Mouhayyar, C., and Aridi, H. (2016). Stress, burnout and coping strategies in preclinical medical students. N. Am. J. Med. Sci. 8:75. doi: 10.4103/1947-2714.177299

PubMed Abstract | CrossRef Full Text | Google Scholar

Fitzpatrick, K. K., Darcy, A., and Vierhile, M. (2017). Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Mental Health 4:e19. doi: 10.2196/mental.7785

PubMed Abstract | CrossRef Full Text | Google Scholar

Flynn, D. M., and Macleod, S. (2015). Determinants of happiness in undergraduate university students. Coll. Stud. J. 49, 452–460.

Google Scholar

Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., and Rauws, M. (2018). Using psychological artificial intelligence (tess) to relieve symptoms of depression and anxiety: randomized controlled trial. JMIR Ment. Health 5:e64. doi: 10.2196/mental.9782

PubMed Abstract | CrossRef Full Text | Google Scholar

Ghose, S., and Barua, J. J. (2013). “Toward the implementation of a topic specific dialogue based natural language chatbot as an undergraduate advisor,” in Informatics, Electronics and Vision (ICIEV), 2013 International Conference on: IEEE (Dhaka), 1–5. doi: 10.1109/ICIEV.2013.6572650

CrossRef Full Text | Google Scholar

Helfferich, C. (2011). Die Qualität qualitativer Daten. Wiesbaden: Springer. doi: 10.1007/978-3-531-92076-4

CrossRef Full Text | Google Scholar

Islam, N., Beer, M., and Slack, F. (2015). E-learning challenges faced by academics in higher education. J. Educ. Training Stud. 3, 102–112. doi: 10.11114/jets.v3i5.947

CrossRef Full Text | Google Scholar

Kavakli, M., Li, M., and Rudra, T. (2012). Towards the development of a virtual counselor to tackle students' exam stress. J. Integr. Design Process Sci. 16, 5–26. doi: 10.3233/jid-2012-0004

CrossRef Full Text | Google Scholar

Madany Mamlouk, A., Geick, C., and Lämmermann, K. (2018). “From zero to hero - new methods for motivating students,” in Excellent Teaching - Principles, Structures and Requirements, eds T. Jansen-Schulz, and T. Tantau (Bielefeld: Blickpunkt Hochschuldidaktik), 99–111.

Google Scholar

Mayring, P. (2010). “Qualitative inhaltsanalyse,” in Handbuch Qualitative Forschung in der Psychologie, eds G. Mey, and K. Mruck (Wiesbaden: Springer), 601–613. doi: 10.1007/978-3-531-92052-8_42

CrossRef Full Text | Google Scholar

Mayring, P. (2014). Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. Klagenfurt: Beltz. doi: 10.1007/978-94-017-9181-6_13

CrossRef Full Text | Google Scholar

Mcgrath, C., Palmgren, P. J., and Liljedahl, M. (2018). Twelve tips for conducting qualitative research interviews. Med. Teach. 41, 1–5. doi: 10.1080/0142159X.2018.1497149

PubMed Abstract | CrossRef Full Text | Google Scholar

Pasikowska, A., Zaraki, A., and Lazzeri, N. (2013). “A dialogue with a virtual imaginary interlocutor as a form of a psychological support for well-being,” in Proceedings of the International Conference on Multimedia, Interaction, Design and Innovation: ACM (Warsaw), 16.

Poria, S., Cambria, E., and Gelbukh, A. (2016). Aspect extraction for opinion mining with a deep convolutional neural network. Knowled. Based Syst. 108, 42–49. doi: 10.1016/j.knosys.2016.06.009

CrossRef Full Text | Google Scholar

Prensky, M. (2001). Digital natives, digital immigrants part 1. On Horiz. 9, 1–6. doi: 10.1108/10748120110424816

CrossRef Full Text

Ranoliya, B. R., Raghuwanshi, N., and Singh, S. (2017). “Chatbot for university related FAQs,” in Advances in Computing, Communications and Informatics (ICACCI), 2017 International Conference on: IEEE (Udupi), 1525–1530. doi: 10.1109/ICACCI.2017.8126057

CrossRef Full Text | Google Scholar

Siemens, G. (2015). “The role of MOOCs in the future of education,” in MOOCs and Open Education Around the World, eds C. J. Bonk, M. M. Lee, T. C. Reeves, and T. H. Reynold (New York; London: Routledge, Taylor & Francis Group), 14–17.

Google Scholar

Stallman, H. M. (2010). Psychological distress in university students: a comparison with general population data. Aust. Psychol. 45, 249–257. doi: 10.1080/00050067.2010.482109

CrossRef Full Text | Google Scholar

Venkatesh, V., and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage. Sci. 46, 186–204. doi: 10.1287/mnsc.46.2.186.11926

CrossRef Full Text | Google Scholar

Verger, P., Combes, J.-B., Kovess-Masfety, V., Choquet, M., Guagliardo, V., Rouillon, F., et al. (2009). Psychological distress in first year university students: socioeconomic and academic stressors, mastery and social support in young men and women. Soc. Psychiatry Psychiatr. Epidemiol. 44, 643–650. doi: 10.1007/s00127-008-0486-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Wildemeersch, D., and Jütte, W. (2017). Digital the new normal-multiple challenges for the education and learning of adults. Eur. J. Res. Educ. Learn. Adults 8, 7–20. doi: 10.3384/rela.2000-7426.relae13

CrossRef Full Text | Google Scholar