Potential and Pitfalls of Digital Voice Assistants in Older Adults With and Without Intellectual Disabilities: Relevance of Participatory Design Elements and Ecologically Valid Field Studies
- 1Network Aging Research, Heidelberg University, Heidelberg, Germany
- 2Institute for Educational Sciences, Heidelberg University of Education, Heidelberg, Germany
- 3Institute of Psychology, Heidelberg University, Heidelberg, Germany
- 4Institute of Prevention, Integration and Rehabilitation Research, Ludwig-Maximilians-Universität Munich, Munich, Germany
- 5Institute for Special Needs Education, Heidelberg University of Education, Heidelberg, Germany
While digital voice assistants (VAs) are increasingly becoming part of everyday life (Auxier, 2019), specific user groups that might benefit from such technology are often not considered in research. This applies to older adults (Stigall et al., 2019), a population with pronounced heterogeneity in terms of cognitive, sensory and functional competencies. Given the rapid expansion of digital tools, it is important to consider the use of VAs as a promising tool able to maintain and enhance social participation, autonomy, and leisure activities for a wide range of older adults. There is still a digital divide between younger and older generations (Anderson, 2019) and older adults might benefit to a smaller extent from the advantages of new technologies such as VAs. Older people with intellectual disabilities are at particular risk of digital exclusion (Ehlers et al., 2020) and digital technologies such as VAs have hardly been evaluated for this target group (e.g., Smith et al., 2020). In this opinion paper, we synthesize current research in the context of VAs for older adults. Building on this, we propose specific research designs to provide better insights into the adoption and use of VAs in advanced age.
Benefits of Voice Assistants
An overview of literature provides six clusters of insights. First, VAs offer technology access for individuals who do not use conventional computing devices. Usability problems caused by small font or buttons are eliminated (Ziman and Walsh, 2018; Kowalski et al., 2019; Corbett et al., 2021), which is especially useful for individuals with limited motor, sensory, or cognitive functions (e.g., Yaghoubzadeh et al., 2013; Wulf et al., 2014). Second, in the social domain, VAs enable contact and communication with others, especially for older people with disabilities like limited vision or impaired hand movement (Kowalski et al., 2019; Scherr et al., 2020; Trajkova and Martin-Hammond, 2020). Additionally, the VA itself can be a social companion to some extent (Scherr et al., 2020; Smith et al., 2020; Corbett et al., 2021). Third, a benefit has been identified in the health domain. VAs can assist with daily well-being activities such as health tracking, medication management, or dietary planning (Tsiourti et al., 2014; Nallam et al., 2020; Trajkova and Martin-Hammond, 2020). Fourth, the use of VAs might contribute to enjoyable leisure time experiences, via entertainment features like music, videos, and jokes (Scherr et al., 2020; Corbett et al., 2021). Fifth, VAs can provide support for independent living including aspects of time-structuring (e.g., setting a timer or reminders) and instrumental activities like access to online information (Nallam et al., 2020; Pradhan et al., 2020; Scherr et al., 2020; Corbett et al., 2021). Sixth, taken together, VAs may have a positive effect on a person's agency because VAs can support older adults with intellectual disabilities in better managing different aspects of their everyday lives (Smith et al., 2020).
Challenges of Voice Assistants and Limitations in Current Research on Voice Assistants in Older Adults
Although it boasts good usability in general, problems interacting with VAs are frequently observed, because the user is required to follow a pre-structured form of dialogue, thus limiting the conversational abilities of VAs (Scherr et al., 2020). Older adults often have problems recalling the specific commands necessary to operate the devices (Wulf et al., 2014; Pradhan et al., 2020). Another limiting factor can be the lack of added value, which may result in a preference for devices already used (Trajkova and Martin-Hammond, 2020). VAs are perceived as time consuming and a lack of compatibility is criticized (Kowalski et al., 2019). Furthermore, a barrier for using VAs can be seen in reported fear of losing one's own competences and autonomy, because the VA may take care of a number of tasks without considering the competencies of the user (Kowalski et al., 2019; Trajkova and Martin-Hammond, 2020). In this way, the benefit of support for independent living and a reduction of dependency on personal assistance may also result in a higher level of dependency on VA assistance. Finally, concerns about privacy and data security are potential barriers to using VAs (Nallam et al., 2020; Trajkova and Martin-Hammond, 2020).
On a more general level, two major limitations in research on VAs for older adults exist. There is limited knowledge on VA use in specific groups of older adults. We identified two studies that focus on benefits and challenges for older adults with cognitive impairment (e.g., dementia; Wargnier et al., 2015; Wolters et al., 2016) and two other studies that included older people with intellectual disabilities (Braun et al., 2020; Smith et al., 2020). We also see limitations at the methodological and design level. In particular, we identified six studies that apply field data collection—collecting data in everyday life settings to analyze the use of VAs among older adults (Tsiourti et al., 2014; Kopp et al., 2018; Oh et al., 2020; Pradhan et al., 2020; Scherr et al., 2020; Corbett et al., 2021). Similarly, user-centered research approaches have been infrequently applied thus far.
Recommendations For Future Research
Based on these insights, we derive recommendations for research on VAs (see Table 1). We put emphasis on subgroups of older adults who have been largely neglected, i.e., older adults with cognitive impairment like dementia-related disorders or intellectual disabilities.
Table 1. Recommendations for future research on Voice Assistants (VAs) in heterogeneous groups of older adults.
Need for Participatory Design Elements in Research With Older Adults
We recommend a high level of user involvement in research about commercial VAs for older people. A participatory design strategy should identify the best ways to introduce VA use, benefits as well as limits and risks of VAs to different groups of older adults. In particular, this allows to address the identified challenges (see above) and may ensure that older people can use VAs according to their preferences. We argue that the need for instruction is higher among older adults compared to younger people because they did not grow up with digital technologies. As an example, we propose a participatory conception and implementation of user trainings and manuals1: To realize a participatory design in this domain, older adults should be involved in different co-designing activities such as co-design workshops to develop and discuss own ideas (e.g., Davidson and Jensen, 2013). Thereby, older users will be able to directly and actively influence the design of different kinds of material:
• User trainings: Educational programs and training courses can lower barriers in VA interaction (Czaja et al., 2019). The contents and format should be developed together with the target group and tailored to varying prior technology experiences, disability, and cognitive competencies. Relevant aspects may include the requirements for a successful interaction with VAs (e.g., specific voice commands), the possibilities of using VAs for different purposes (e.g., social domain, health, leisure time), and possible concerns (e.g., privacy issues, losing competencies due to using VAs). We recommend including older users with different skill levels in the training conception to make VAs widely accessible. Due to the different competencies (e.g., in reading and writing, attentional control, executive functions), the presentation and complexity must be adapted individually in each case.
• User manuals: Another aspect of VA learning and adoption are user manuals, an aspect that has so far been insufficiently addressed in research. Well-designed guidelines could help older adults to explore the possibilities of VAs according to their needs and in their own pace. Different versions of user manuals should be discussed with older adults to achieve the best design possible. These group-specific manuals are especially helpful for older adults with cognitive impairments or intellectual disabilities who may have specific needs, like instructions in easy-to-read language (Maaß, 2020) or a visualization of the instructions (Spriggs et al., 2017).
Need for Field Studies in the Everyday Lives of Older Adults
Field studies can contribute to a high level of ecological validity of the research–implying the empirical validity of findings in everyday life settings. Depending on the research questions, researchers should thoroughly consider the appropriate observation period. Existing field studies about VA use report durations from only a few days (Lopatovska et al., 2019), several months (Scherr et al., 2020), up to 1 year (Trajkova and Martin-Hammond, 2020). Our recommendation would be an observation period of at least 4 weeks, which would allow analysis of more routinized interactions with VAs after an initial phase of learning and curiosity.
Despite existing field studies, we claim that the potentials are not yet fully exploited, and should be extended focusing on innovative approaches and heterogeneous user groups:
• Diary studies: In open-ended and closed questions, the users can report their experiences, likes, and dislikes about using a VA. A digital diary format allows to provide additional assistance if necessary. Participants can be reminded of the diary with prompts, and questions can be repeated and adapted to the individual. Participants can be actively encouraged to provide comprehensive feedback on enjoyable, useful, and negative experiences of VA interactions. In addition to this, cultural probes such as self-taken photos, cards with reflection tasks about VA use, and other activities (e.g., creation of relationship or neighborhood maps) can be applied to gain further insights into the everyday lives of the participants and to capture older adults' experiences with the devices in a comprehensive manner (Jarke and Gerhard, 2017).
• Analysis of automatically collected data: Beyond users' self-reports, we see high potential in collecting additional data associated with usage behaviors like back-end data combined with external recordings of audio (Porcheron et al., 2018) or video (Lahoual and Frejus, 2019) data of VA use. These data provide information about which VA functions are used by individuals with different levels of expertise and competences, about used voice commands, and changes of use patterns over time. In this context, researchers should in any case consider ethical concerns of automatically collected data such as a threat of permanent observation or the fear of providing too intimate information. Attention should be paid to the design of the informed consent and continuous support of study participants concerning these aspects should be provided.
• Emotion analysis: State-of-the-art software solutions allow to automatically analyze emotional experiences based on speech and facial expressions (Garcia-Garcia et al., 2017; Dupré et al., 2018). The emotional experience of VA use is another aspect that is important to our understanding of their benefits and challenges for older adults. An analysis of these data allows researchers to study user experiences of VAs in situations when verbal feedback is scarce due to possible cognitive impairments or intellectual disabilities. Still, due to the novelty of the approach, automatic emotion analysis is not always reliable. Regarding older adults with intellectual disabilities, their emotional expression may differ from older adults without disabilities due to more frequent motor impairments (e.g., spasticity), differences in physical appearance (especially regarding genetic syndromes), cognitive deficits (e.g., in perception or appraisal processes), stereotypical behaviors, or earlier aging processes (von Gontard, 2013). Facial expression may be altered so that automatic face recognition cannot detect known patterns. The validity of automatic emotion analysis in this group has not yet been proven (Adams and Oliver, 2011; Martínez-González and Veas, 2019) and should be the focus of future research.
In particular, the triangulation of the different data sources can provide an overarching picture of user experiences of VAs, e.g., by analyzing in which situations the VA is used, how the older person evaluates this interaction, and how it is experienced emotionally.
In this opinion paper, we offer a set of recommendations that may guide future research on VAs for and with older adults. In a nutshell, we posit that future research designs should strongly rely on analyzing VA interactions in everyday ecologies and strictly apply participatory design elements where possible. Data protocols should include a balanced mixture of automatized data including emotional aspects as well as structured and open assessments. From our perspective, considering these recommendations can significantly help to create evidence-based findings able to inform interventions with VAs in heterogeneous groups of older adults. This also contributes to the goal of getting the best out of VA systems to improve quality of life and avoid possible risks.
All authors provided substantial contributions to this article from conception to final approval.
This publication was part of the AI-Aging project (AI-based voice assistance for older adults with and without intellectual disabilities), which was funded by the Baden-Württemberg Stiftung within the funding line Responsible Artificial Intelligence. The article processing charge was funded by the Baden-Württemberg Ministry of Science, Research and Culture and the Heidelberg University of Education in the funding programme Open Access Publishing.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
1. ^We put emphasis on commercial VAs that are available “off the shelf” but offer lower customization and co-designing options. However, in the case of VA design and development, participatory research and co-designing is equally important to better understand users' needs and their preferences. A co-design of VAs will guarantee that style, content, wording and tasks of the VAs will fit the mental models of older adults (Wolters et al., 2016; Fischer et al., 2020).
Adams, D., and Oliver, C. (2011). The expression and assessment of emotions and internal states in individuals with severe or profound intellectual disabilities. Clin. Psychol. Rev. 31, 293–306. doi: 10.1016/j.cpr.2011.01.003
Anderson, M. (2019). Mobile Technology and Home Broadband 2019. Available online at: https://www.pewresearch.org/internet/wp-content/uploads/sites/9/2019/06/PI_2019.06.13_Mobile-Technology-and-Home-Broadband_FINAL2.pdf (accessed May 17, 2021).
Auxier, B. (2019). 5 Things to Know About Americans and Their Smart Speakers. Available online at: https://www.pewresearch.org/fact-tank/2019/11/21/5-things-to-know-about-americans-and-their-smart-speakers/ (accessed May 20, 2021).
Braun, M., Wölfel, M., Renner, G., and Menschik, C. (2020). “Accessibility of different natural user interfaces for people with intellectual disabilities,” in 2020 International Conference on Cyberworlds (CW) (Caen: IEEE), 211–218.
Corbett, C. F., Combs, E. M., Wright, P. J., Owens, O. L., Stringfellow, I., Nguyen, T., et al. (2021). Virtual home assistant use and perceptions of usefulness by older adults and support person dyads. Int. J. Environ. Res. Public Health 18, 1–13. doi: 10.3390/ijerph18031113
Davidson, J. L., and Jensen, C. (2013). “Participatory design with older adults,” in Proceedings of the 9th ACM Conference on Creativity and Cognition, eds E. Y.-L. Do, S. Dow, J. Ox, S. Smith, K. Nishimoto, C. T. Tan, Y. Nagai, S. Ferguson, and T. Hewett (Sydney, NSW: ACM), 114–123.
Dupré, D., Andelic, N., Morrison, G., and McKeown, G. (2018). “Accuracy of three commercial automatic emotion recognition systems across different individuals and their facial expressions,” in 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (Athens: IEEE), 627–632.
Ehlers, A., Heß, M., Frewer-Graumann, S., Olbermann, E., and Stiemke, P. (2020). “Digitale Teilhabe und (digitale) Exklusion im Alter: Expertise zum Achten Altersbericht der Bundesregierung,” in Expertisen zum Achten Altersbericht der Bundesregierung, eds C. Hagen, C. Endter, and F. Berner (Berlin: Deutsches Zentrum für Altersfragen).
Fischer, B., Peine, A., and Östlund, B. (2020). The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist 60, e513–e523. doi: 10.1093/geront/gnz163
Garcia-Garcia, J. M., Penichet, V. M. R., and Lozano, M. D. (2017). “Emotion detection: technology review,” in Proceedings of the XVIII International Conference on Human Computer Interaction–Interacción '17, ed J. M. González-Calleros (Cancun: ACM Press), 1–8. doi: 10.1145/3123818.3123852
Jarke, J., and Gerhard, U. (2017). “Using cultural probes for co-creating a digital neighbourhood guide with and for older adults,” in Mensch und Computer–Workshopband, eds M. Burghardt, R. Wimmer, C. Wolff, and C. Womser-Hacker (Regensburg), 79–85.
Kopp, S., Brandt, M., Buschmeier, H., Cyra, K., Freigang, F., Krämer, N., et al. (2018). “Conversational assistants for elderly users–the importance of socially cooperative dialogue,” in Proceedings of the AAMAS 2018 Workshop on Intelligent Conversational Agents in Home and Geriatric Care Applications (Stockholm).
Kowalski, J., Jaskulska, A., Skorupska, K., Abramczuk, K., Biele, C., Kopeć, W., et al. (2019). “Older adults and voice interaction: a pilot study with google home,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, eds S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos (Glasgow: ACM), 1–6.
Lahoual, D., and Frejus, M. (2019). “When users assist the voice assistants: from supervision to failure resolution,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, eds S. Brewster, G. Fitzpatrick, A. Cox, and V. Kostakos (Glasgow: ACM), 1–8.
Lopatovska, I., Rink, K., Knight, I., Raines, K., Cosenza, K., Williams, H., et al. (2019). Talk to me: exploring user interactions with the Amazon Alexa. J. Librariansh. Inf. Sci. 51, 984–997. doi: 10.1177/0961000618759414
Martínez-González, A. E., and Veas, A. (2019). Identification of emotions and physiological response in individuals with moderate intellectual disability. Int. J. Dev. Disabil. 1–6. doi: 10.1080/20473869.2019.1651142
Nallam, P., Bhandari, S., Sanders, J., and Martin-Hammond, A. (2020). A question of access: exploring the perceived benefits and barriers of intelligent voice assistants for improving access to consumer health resources among low-income older adults. Gerontol. Geriatr. Med. 6:2333721420985975. doi: 10.1177/2333721420985975
Porcheron, M., Fischer, J. E., Reeves, S., and Sharples, S. (2018). “Voice interfaces in everyday life,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems–CHI '18, eds R. Mandryk, M. Hancock, M. Perry, and A. Cox (Montreal, QC: ACM Press), 1–12.
Scherr, S. A., Meier, A., and Cihan, S. (2020). “Alexa, tell me more – about new best friends, the advantage of hands-free operation and life-long learning,” in Mensch und Computer 2020–Workshopband, eds C. Hansen, A. Nürnberger, and B. Preim (Bonn: Gesellschaft für Informatik).
Smith, E., Sumner, P., Hedge, C., and Powell, G. (2020). Smart-speaker technology and intellectual disabilities: agency and wellbeing. Disabil. Rehabil. Assist. Technol. 1–11. doi: 10.1080/17483107.2020.1864670
Spriggs, A. D., Mims, P. J., van Dijk, W., and Knight, V. F. (2017). Examination of the evidence base for using visual activity schedules with students with intellectual disability. J. Spec. Educ. 51, 14–26. doi: 10.1177/0022466916658483
Stigall, B., Waycott, J., Baker, S., and Caine, K. (2019). “Older adults' perception and use of voice user interfaces: a preliminary review of the computing literature,” in OZCHI'19, eds S. Berkovsky, Y. Hijikata, J. Rekimoto, M. Burnett, M. Billinghurst, and A. Quigley (Fremantle, WA: ACM Press), 55–66.
Trajkova, M., and Martin-Hammond, A. (2020). ““Alexa is a Toy”: exploring older adults' reasons for using, limiting, and abandoning echo,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, eds R. Bernhaupt, F. Mueller, D. Verweij, J. Andres, J. McGrenere, A. Cockburn, I. Avellino, A. Goguey, P. Bjørn, S. Zhao, B. P. Samson, and R. Kocielnik (Honolulu, HI: ACM), 1–13.
Tsiourti, C., Joly, E., Wings, C., Ben Moussa, M., and Wac, K. (2014). “Virtual assistive companions for older adults: qualitative field study and design implications,” in PervasiveHealth ‘14: proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg, Germany, May 20-23, 2014, eds A. Hein and S. Boll (Oldenburg: ICST, Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 57–64.
von Gontard, A. (2013). “Genetische und biologische Grundlagen.” in Geistige Behinderung: Grundlagen, Erscheinungsformen und klinische Probleme, Behandlung, Rehabilitation und rechtliche Aspekte, eds G. Neuhäuser, H.-C. Steinhausen, F. Häßler, K. Sarimski, A. von Gontard, J. Buchmann, O. Reis, T. Klauß, S. Wachsmuth, K. Fischer, G. Grampp, S. Elstner, M. Buchka, C. Schanze, T. Frühauf, S. Wendt, U. Hellmann, and K. Lachwitz (Stuttgart: Kohlhammer Verlag), 30–43.
Wargnier, P., Malaise, A., Jacquemot, J., Benveniste, S., Jouvelot, P., Pino, M., et al. (2015). “Towards attention monitoring of older adults with cognitive impairment during interaction with an embodied conversational agent,” in 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology (VAAT) (Arles: IEEE), 23–28.
Wolters, M. K., Kelly, F., and Kilgour, J. (2016). Designing a spoken dialogue interface to an intelligent cognitive assistant for people with dementia. Health Inform. J. 22, 854–866. doi: 10.1177/1460458215593329
Wulf, L., Garschall, M., Himmelsbach, J., and Tscheligi, M. (2014). “Hands free–care free: elderly people taking advantage of speech-only interaction,” in Proceedings of the 8th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational-NordiCHI'14, eds V. Roto, T. Olsson, K. Väänänen-Vainio-Mattila, J. Häkkilä, E. Hvannberg, and O. Juhlin (Helsinki: ACM Press), 203–206.
Yaghoubzadeh, R., Kramer, M., Pitsch, K., and Kopp, S. (2013). “Virtual agents as daily assistants for elderly or cognitively impaired people: Studies on acceptance and interaction feasibility,” in Intelligent Virtual Agents. IVA 2013. Lecture Notes in Computer Science, eds R. Aylett, B. Krenn, C. Pelachaud, and H. Shimodaira (Berlin, Heidelberg: Springer). doi: 10.1007/978-3-642-40415-3_7
Ziman, R., and Walsh, G. (2018). “Factors affecting seniors' perceptions of voice-enabled user interfaces.” in Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, eds R. Mandryk, M. Hancock, M. Perry, and A. Cox (Montreal, QC: ACM), 1–6.
Keywords: speech assistant, smart speaker, aging, old age, digitization, ecological validity, digital diary
Citation: Schlomann A, Wahl H-W, Zentel P, Heyl V, Knapp L, Opfermann C, Krämer T and Rietz C (2021) Potential and Pitfalls of Digital Voice Assistants in Older Adults With and Without Intellectual Disabilities: Relevance of Participatory Design Elements and Ecologically Valid Field Studies. Front. Psychol. 12:684012. doi: 10.3389/fpsyg.2021.684012
Received: 22 March 2021; Accepted: 01 June 2021;
Published: 01 July 2021.
Edited by:Benjamin Weiss, Hanyang University, South Korea
Reviewed by:Maria Klara Wolters, University of Edinburgh, United Kingdom
Copyright © 2021 Schlomann, Wahl, Zentel, Heyl, Knapp, Opfermann, Krämer and Rietz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Anna Schlomann, firstname.lastname@example.org