eXtended Artificial Intelligence: New Prospects of Human-AI Interaction Research

Abstract

Artificial Intelligence (AI) covers a broad spectrum of computational problems and use cases. Many of those implicate profound and sometimes intricate questions of how humans interact or should interact with AIs. Moreover, many users or future users do have abstract ideas of what AI is, significantly depending on the specific embodiment of AI applications. Human-centered-design approaches would suggest evaluating the impact of different embodiments on human perception of and interaction with AI. An approach that is difficult to realize due to the sheer complexity of application fields and embodiments in reality. However, here XR opens new possibilities to research human-AI interactions. The article’s contribution is twofold: First, it provides a theoretical treatment and model of human-AI interaction based on an XR-AI continuum as a framework for and a perspective of different approaches of XR-AI combinations. It motivates XR-AI combinations as a method to learn about the effects of prospective human-AI interfaces and shows why the combination of XR and AI fruitfully contributes to a valid and systematic investigation of human-AI interactions and interfaces. Second, the article provides two exemplary experiments investigating the aforementioned approach for two distinct AI-systems. The first experiment reveals an interesting gender effect in human-robot interaction, while the second experiment reveals an Eliza effect of a recommender system. Here the article introduces two paradigmatic implementations of the proposed XR testbed for human-AI interactions and interfaces and shows how a valid and systematic investigation can be conducted. In sum, the article opens new perspectives on how XR benefits human-centered AI design and development.

Introduction

Artificial Intelligence (AI) today covers a broad spectrum of application use cases and the associated computational problems. Many of those implicate profound and sometimes intricate questions of how humans interact or should interact with AIs. The continuous proliferation of AIs and AI-based solutions into more and more areas of our work and private lives also significantly extends the potential range of users in direct contact with these AIs.

There is an open and ongoing debate on the necessity of required media competencies or, even more, on required computer science competencies for users of computer systems. This digital literacy (competencies needed to use computational devices (Bawden and others, 2008) and computational literacy (the ability to use code to express, explore, and communicate ideas (DiSessa, 2001)), lately has been extended to also include AI literacy to denote competencies that enable individuals to critically evaluate AI technologies; communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the workplace (Long and Magerko, 2020). This debate roots deep into the progress of the digital revolution for some decades now. AI brings in an exciting flavor to this debate since it risks significantly amplifies the digital divide for certain (groups of) individuals just by the implicit connotation of the term. For one, AI’s implicit claim to replicate human intelligence can be attributed to the term “Artificial Intelligence” itself, as John McCarthy proposed at the famous Dartmouth conference in 1956. Some researchers still consider the term ill-posed to begin with, and the history of AI records alternatives with less implicate associations, see contemporary textbooks on AI, e.g., by Russel and Norvig (Russell and Norvig, 2020).

Nevertheless, now AI-applications are omnipresent, the term AI is commonly used for the field, and the term certainly implies far-reaching connotations for many users not experts in the specific field of AI or in computer science in general. Additionally, the reception and presentation of AI by mainstream media, e.g., in movies and other works of fiction, has undoubtedly contributed to shaping a very characteristic AI profile (Kelley et al., 2019; Zhang and Dafoe, 2019). This public understanding often sketches at least a skewed image of the principles and the potentials and risks of AI (see Figure 1). As Clarke has pointed out in his often quoted third law, “Any sufficiently advanced technology is indistinguishable from magic” (Clarke, 1962). Suppose the latter could already be observed for such simple computing applications like a spread-sheet, or more complex examples of Information Technology (IT) like “the Internet”. In that case, it seems even more likely to be true for assistive devices that listen to our voices and speak to us in our native tongue, robots that operate in the same physical realm with us and which—for a naïve observer—seem to be alive, or self-driving cars, or many more incarnations of modern AI systems. From a Human-Computer Interaction (HCI) perspective, it is of utmost importance to understand and investigate if, and if, how the human user perceives the AI she is interacting with. Besides, most AI-systems will incorporate a human-computer interface. With interface we here denote the space where the interaction between human and machine takes place, including all hardware and software components and the underlying interaction concepts and styles.

FIGURE 1

The appearance of an AI at the interface can range from simple in general pervasive effects like the execution of a requested operation, such as switching on a lamp in a smart home appliance, to simple text displays, to humanoid and human looking robots or virtual agents trying to mimic real persons with communicative behaviors typically associated with real humans. Often, AIs will appear to the user at the interface with some sort of embodiment, such as a specific device like a smart speaker, as a self-driving car, or as a humanoid robot or a virtual agent appearing in Virtual, Augmented, or Mixed Reality (VR, AR, MR: XR, for short). The latter is specifically interesting since the embodied aspect of the interface is mapped to the virtual world in contrast to the real physical environment. Embodiment itself has interesting effects on the user such as the well-known uncanny valley effect (Mori et al., 2012) or the Proteus effect (Yee and Bailenson, 2007). The latter describes a change of behavior caused by a modified and perceived self-representation, such as the appearance of a self-avatar. First results indicate that this effect also exists regarding the perception of others (Latoschik et al., 2017) which can be hypothesized to also apply to the human-AI interface.

The media equation postulates that the sheer interaction itself already contributes significantly to the perception and, more specifically, an anthropomorphization of technical systems by users (Reeves and Nass, 1996a). In combination with an already potentially shallow or even skewed understanding of AI by some users, i.e., a sketchy AI literacy, or an unawareness of interacting with an AI by others, and the huge design space of potential human-AI interfaces, it seems obvious that we need a firm understanding of the potential effects the developers’ choice of the appearance of human-AI interfaces has on the users. Human-centered-design approaches would suggest evaluating the impact of different embodiments on human perception of and interaction with AIs and identifying the effects these manipulations would have on users. This approach, which is central to the principles in HCI, however, is difficult to realize due to the large design space of embodiments in, and the sheer complexity of many fields of applications of the real physical world. Another rising technology cluster, XR, accounts for control and systematic manipulations of complex interactions (Blascovich et al., 2002; Wienrich and Gramlich, 2020; Wienrich et al., 2021a). Hence, XR provides much potential to increase the investigability of human-AI interactions and interfaces.

In sum, a real-world embodiment of an AI might need considerable resources, e.g., when we think of humanoid robots or self-driving cars. In turn, systematic investigations are essential for an evidence-based human-centered AI design, the design and evaluation of explainable AIs and tangible training modules, and basic research of human-AI interfaces and interactions. The present paper suggests and discusses XR as a new perspective on the XR-AI combination space and as a new testbed for human-AI interactions and interfaces by raising the question:

How can we establish valid and systematic investigation procedures for human-AI interfaces and interactions?

Four sub-questions structure the first part of the article. Theoretical examinations about these questions contribute to a new perspective on the XR-AI combination space on the one hand and a new testbed for human-AI interactions and interfaces on the other hand (Table 1).

The second part of the present article introduces two paradigmatic implementations of the proposed XR testbed for human-AI interactions and interfaces. An XR environment simulated interactive and embodied AIs (Experiment 1: a conversational robot, Experiment 2: recommender system) to evaluate the perception of the AI and the interaction in dependency of various AI embodiments (Table 2).

TABLE 2

Contribution	Research Question | Experimental Approach
Experiment 1 simulated a robot-human interaction in an industrial context	How do the sense of the robot’s social intelligence influence the robot’s perception and the evaluation of the interaction?
	Manipulation of the conversational ability to vary the sense of social intelligence of a simulated robot
Experiment 2 simulated a recommender system in a quiz game setting	How does the sense of complexity influence the perception of a recommender system?
	Manipulation of the appearance of an embodied recommender system to vary the sense of complexity

Summarizes the experimental approach and corresponding contributions of the article’s second part.

A New Perspective on the XR-Artificial Intelligence Combination Space

How Is Human-Artificial Intelligence Interaction Defined?

Human-AI interaction (HAII) has its roots in the more general concept of HCI, since we here assume that some sort of computing machinery realizes an artificial intelligence. Hence human-AI interaction is a special form of HCI where the AI is a special incarnation of a computer system. Note that this definition does not distinguish between a hardware and a software layer, but uses the term computer system in the general sense, combining both aspects of hardware and software together to constitute a system that interacts with the user.

The second aspect to clarify is what precisely we mean by the term Artificial Intelligence or AI. As we have briefly noted in the introduction, the term has a long history going back at least to the Dartmouth conference in 1956. Typical definitions of AI usually incorporate some definition of the aspect of artificiality referring to a machine or computer system as the executor of some sort of simulation or process which can be attributed to some type of intelligence either in its relationship between input and output or in its internal functioning or model of operation, which might try to mimic the internal functioning of a biological entity one considers intelligent. Note that in these types of definitions, the term intelligence is not defined but often implicitly refers to human cognitive behaviors.

Russell and Norvig demarcate AI as a separate field of research and application from other fields like mathematics, control theory, or operations research by two descriptions (Russell and Norvig, 2020): First, they state that from the beginning, AI included the concept to replicate human capabilities like creativity, self-dependent learning, or utilization of speech. Second, they point to the employed methods, which first and centrally are rooted in computer science: AI uses computing machines as the executor of some creative or intelligent processes, i.e., processes that allow the machine to autonomously operate in complex, continuously changing environments. Both descriptions cover a good number of AI use cases. However, they are not comprehensive enough to cover all use cases, unless one interprets complex, continuously changing environments very broadly. We want to extend this circumscription of AI with processes that adopt computational models of intelligent behavior to solve complex problems that humans are not able to solve due to the sheer quantity and/or complexity of input data, i.e., as typical in big data and data mining.

To further clarify what we mean by human-AI interaction, we use the term AI in human-AI to denote the incarnation of a computing system incorporating AI capabilities as described above. For the remainder of this paper, the context should clarify if we talk about AI as the field of research or AI as such an incarnation of an intelligent computing system, and we will precisely specify what we mean where it might be ambiguous. Note that this ambiguity between AI as a field or AI as a system already indicates the tendency to attribute particular capabilities and human-like attributes to such a system and potentially see it as an independent entity that we can interact with. The latter is not specific to an AI but can already be noticed for non-AI computer systems where users tend to attribute independent behavior to the system, specifically if something is going wrong. Typical examples from user reactions calling a support line like “He doesn’t print” or “He is not letting me do this” indicate this tendency and usually are interpreted as examples of the media equation (Reeves and Nass, 1996b; Nass and Gong, 2000; Wienrich et al., 2021b). However, we want to stress the point that such effects might be amplified by AI-systems due to their, in principle, rather complex behaviors and, as we will discuss in the next sections, due to the general character and appearance of an AI’s embodiment.

How to Classify Combined XR With Artificial Intelligence? The XR-Artificial Intelligence Continuum

As we pointed out, there are various use-cases for and also incarnations of AIs, and in this context, we do not want to restrict the kinds of AIs in human-AI interaction any further. As we have seen, AI are specialized computer systems either by their internal workings or by their use case, and capabilities. As such, if they do not operate 100% independently from humans but will serve a role, task, or function, there will be the need to interact with human users. For some AI-applications, it seems more straightforward to think about forms of an embodiment for an AI, e.g., robots, conversational virtual agents, or smart speakers (examples Figure 2). However, if there is a benefit of AI-embodiment for the user, e.g., if it helps to increase the usability or user experience (UX), or if it helps the user to gain a deeper understanding of an AI’s function and capabilities, then we should consider extending the idea of AI-embodiment to more use cases of AIs, from expert systems to data science to self-driving cars. However, appearance matters, i.e., the kind of embodiment impacts significantly on human perception and acceptance. From an HCI perspective, it is of utmost importance to understand and investigate how an AI’s appearance influences the human counterpart. Only when we understand this influence we can scientifically contribute to a user-centered AI design process, considering responsively different user groups. Note, the term embodiment is not restricted to self-avatars representing humans in a virtual environment. We follow the common understanding of embodiment addressing also “others” in social encounters (also in conjunction with the term other-avatar). The complexity of human-AI interactions and interfaces challenges such investigations (see below). XR offers promising potentials to meet these challenges.

FIGURE 2

Intelligent Graphics is about visually representing the world and visually representing our ideas. Artificial intelligence is about symbolically representing the world, and symbolically representing our ideas. And between the visual and the symbolic, between the concrete and the abstract, there should be no boundary. (Lieberman, 1996)

Lieberman’s quote describes a central paradigm that combines AI with computer graphics (CG). Its focus on symbolic AI methods seems limited today. However, in the late nineties of the last century, the surge and success of machine learning and deep learning approaches was yet to come. Hence the quote should be seen as a general statement about the combination of AI and CG. Today, intelligent graphics or synonymously smart graphics refers to a wide variety of application scenarios. These range from the intelligent and context-sensitive arrangement of graphical elements in 2D desktop systems to speech-gesture interfaces or intelligent agents in virtual environments as assistants to users. All these approaches have in common that a graphical human-computer interface is adapted to the user’s cognitive characteristics with the help of AI processes to improve the operation (Latoschik, 2014).

The combination of AI and XR, more specifically of Artificial Intelligence and artificial life techniques with those of virtual environments, has been denoted by Aylett and Luck (2000) as Intelligent Virtual Environments. They specifically concentrated on autonomous, physical, and cognitive agents and argued that “embodiment may be as significant for virtual agents as they are for real agents.” They proposed a spectrum between physical and cognitive and identified autonomy as an important quality for such virtual entities. We here argue that the combination of XR and AI is significantly broader and propose the XR-AI continuum (see Figure 3).

FIGURE 3

The XR-AI continuum is spanned between two endpoints defined by the overall epistemological perspective and goal of a given XR-AI combination. The continuum ranks XR-AI combinations with respect to the general question of what we want to achieve by a given XR-AI combination, i.e. the epistemological motive. Are we using AI as an enabling technology to improve an XR system, e.g., to realize certain AI-supported functionalities, user interfaces, and/or to improve the overall usability? Or do we use XR as a tool to investigate AI? The first perspective is on-trend typical for the majority of early approaches of XR-AI combinations, e.g., as described by intelligent virtual environments, intelligent real-time interactive systems, or, with less focus on immersive and highly interactive displays, as described by smart or intelligent graphics (Latoschik, 2014). While Aylett and Luck mainly focused on AI as an enabling technology to improve the virtual environment (Aylett and Luck, 2000), the XR-AI continuum also highlights how XR technologies provide a new investigability of HAII.

The proposed XR-AI continuum with its explicit perspective of the investigability of HAII circumscribed by the eXtended AI approaches combines current developments and results from XR and AI. Recent work has motivated to investigate how humans react to AIs during interaction. These approaches include investigations of verbal interaction (Fraser et al., 2018; Rieser, 2019) and its influence on the tendency to anthropomorphize (Strathmann et al., 2020), and behavior design (Azmandian et al., 2019) including emotional intelligence (Fan et al., 2017). (Kulms and Kopp, 2016) explicitly target embodiment effects. There are some early examples motivating the methodological aspects of investigations of HAII, though they concentrate on emulating investigability by desktop applications (Zhang et al., 2010; Bickmore et al., 2013; Mattar et al., 2015). However, none of the approaches motivate to utilize the increased design and effect space of XR, e.g., to provide situated and/or embodied interactions in a simulated real-like context. There is strong evidence that media-related characteristics like interaction, immersion (inclusive, extensive, surrounding, and vivid) (Slater and Wilbur, 1997), or plausibility (Slater, 2009; Skarbez et al., 2017; Latoschik and Wienrich, 2021) have a huge impact on investigated and expected target effects, for example in terms of emotional response and/or embodiment (Yee and Bailenson, 2007; Gall and Latoschik, 2018; Waltemate et al., 2018).

More recently, Antakli et al. (2018) presented a virtual simulation for testing human-robot interaction (Antakli et al., 2018). However, the authors only briefly illustrated the entire design space and the advantages of an XR testbed proposed in the present paper. Very recently, Sterna et al. (2021) pointed to the lack of pretesting in the VR community (Sterna et al., 2021). They presented preliminary work on a web-based tool for pretesting virtual agents. Again, the application does not allow situated or embodied interaction in a real-like context.

Two currently published contributions point to a lack of research referring to the right side of our continuum. Ospina-Bohórquez et al. (2021) analyzed in a structured literature review the interplay between VR and multi-agent systems (Ospina-Bohórquez et al., 2021). It answered two research questions: “What applications have been developed with Multi-agent systems in the field of Virtual Reality?” and “How does Virtual Reality benefit from the use of Multi-agent systems?”. The analysis revealed fruitful combination and application areas. Remarkable, the search also demonstrated that most combinations are intended to make the virtual environment more intelligent referring to the left side of our continuum. The authors conclude that there is a lack of research investigating more complex simulations to examine the user-avatar relationship on the one hand. On the other hand, they stated that “[…] it would be necessary to include the mental state of agents (emotions, personality, etc.) as part of the agent’s perception process, since these factors influence human attention processes in the real world” (p. 17). Both desiderates hints to the significance of the right sight of our continuum. Further, Fitrianie et al. (2020) pointed to a methodological crisis on the evaluation of artificial social agents (Fitrianie et al., 2020). They discuss that most studies use different approaches to investigate human-agent interactions, resulting in a lack of comparability and replicability. They contributed to a solution by looking for constructs and questionnaire items to make the research on virtual agents more comparable and replicable.

In sum, many research can be classified to the left side of the XR-AI Continuum, while using XR as an investigation method for AI needs further research.

The following sections discuss why the extension is necessary and how it contributes to a better understanding of human-AI interactions and interfaces.

XR as a New Testbed for Human-AI Interactions and Interfaces

What can We Learn From the Challenges and XR Solutions Concerning the Investigability of Human-Human Interaction?

In psychology, interaction is defined as a dynamic sequence of social actions between individuals (or groups) who modify their actions and reactions due to actions by their interaction partner(s) (Jonas et al., 2014).

Researchers studying individual differences in human-human social interactions face the challenge of keeping constant or changing systematically the behavior and appearance of the interaction partner across participants (Hatfield et al., 1992). Even slightly different behaviors and appearances influence participants’ behavior (Congdon and Schober, 2002; Topál et al., 2008; Kuhlen and Brennan, 2013). For investigating social interactions between humans, the potentials of XR are already recognized (Blascovich, 2002; Blascovich et al., 2002). Using virtual humans provides high ecological validity and high standardization (Bombari et al., 2015; Pan and Hamilton, 2018). In addition, using a virtual simulation of interaction enables researchers to easily replicate the studies, which is essential for social psychology, in which replication is lacking (Blascovich et al., 2002; Bombari et al., 2015; Pan and Hamilton, 2018). Another advantage of using XR to study human-human interactions is that situations and manipulations that would be impossible in real life can be created (Bombari et al., 2015; Pan and Hamilton, 2018). Many studies substantiated XR’s applicability and versatility to simulate and investigate social interaction between (virtual) humans (Blascovich, 2002; Bombari et al., 2015; Wienrich et al., 2018a, 2018b). Many of them showed the significant impact of different self-embodiments on self-perception, known as the Proteus effect (Yee and Bailenson, 2007; Latoschik et al., 2017; Ratan et al., 2019). Recent results show the Proteus effect caused by self-avatars also applies to the digital counterparts (the avatars) of others. Others demonstrated how XR potentials are linked to psychological variables (Wienrich and Gramlich, 2020; Wienrich et al., 2021a).

Which Challenges and Solutions Arise for the Systematic Investigation of Human-AI Interaction and Interfaces?

In HCI and hence in HAII, at least one interacting partner is a human, and at least one partner is constituted by a computing system or an AI, respectively. However, the focus has traditionally less been on the social aspects but a task level or the pragmatic quality of interaction. Notably, social aspects of the interaction are recently becoming more and more found interesting in HCI (Carolus and Wienrich, 2019). This trend is becoming even more relevant for HAII due to the close resemblance of certain HAII properties with human intelligence (Carolus et al., 2019).

Thus, AI applications are becoming increasingly interactive and embodied, leading to the question of how researchers can study individual differences in human-AI interactions and the impact of different AI embodiments on human perception of and interaction with AI. Similar to human-human interaction, we face the challenge of keeping constant or changing systematically the behavior and appearance of the artificial interaction partner across participants. Such an approach is challenging to realize due to the sheer complexity of application fields and embodiments in reality. Similar to human-human interaction, interactive and embodied AIs lead up to considerable challenges for the systematic investigability of human-AI interactions. However, XR opens four essential potentials shown in Table 3.

From an HCI perspective, eXtended AI (left side of the XR-AI continuum) constitutes a variant of rapid prototyping for HAII (Table 3). Rapid prototyping includes methods to fabricate a scale model of a physical part or to show the function of a software product quickly (Yan and Gu, 1996; Pham and Gault, 1998). The clue is that users interact with a prestage or a simulation instead of the fully developed product. Such methods are essential for iterative user-centered design processes because they supply user insights in the early stages of development processes (Razzouk and Shute, 2012). Computer aided design (CAD), Wizard-of-Oz, mock-ups, darkhorse prototyping, or the Eliza principle are established rapid prototyping methods. XR as a testbed for HAII allows for rapid prototyping of interactive and embodied AIs, for complex interactions, and in different development stages to understand user’s mental models about AI, predict interaction paths and reactions. Besides, multimodel interactions and analyzes can be quickly realized and yield interesting results for the design, accessibility, versatility, and training effects. Of course, XR technology faces some challenges regarding the accessibility of the technology itself (Peck et al., 2020). However, XR technology becomes increasingly mobile and cheaper (e.g., Oculus Quest, Pico neo) and easier to developed (e.g., with Unity). Accessibility and versability refer instead to the advantage of XR reaching regions or user groups that are often cut off from industrial centers or innovation hubs. Moreover, situated interaction in different contexts can be realized facilitating participative design approaches and contextual learning.

In sum, the first part presented theoretical examinations about four sub-questions revealing a new perspective on the XR-AI combination space on the one hand and a new testbed for human-AI interactions and interfaces on the other hand. Hence, the first part show why the combination of XR and AI fruitfully contribute to a valid and systematic investigation of human-AI interactions and interfaces.

The following second part of the present article introduces two paradigmatic implementations of the proposed XR testbed for human-AI interactions and interfaces and show how a valid and systematic investigation can be conducted.

Paradigmatic Implementations of an XR Testbed for Human-Artifical Intelligence Interactions and Interfaces

In the following, we outline two experiments simulating a human-AI interaction in XR. The first experiment simulated a human-robot interaction in an industrial context. The second one resembled an interaction with an embodied recommender system in a quiz game context. The experiments serve as an illustration to elucidate the four potentials of XR mentioned above. Thus, only a few pieces of information are presented related to the scope of the article leading to a deviation from a typical method and results presentation. Please refer to the authors for more detailed information regarding the experiments. In the discussion section, the possibilities and limits of such an XR testbed for AI-human interactions, in general, are illustrated on the paradigmatic implementations.

Paradigmatic Experiment 1: Simulated Human-Robot Interaction

Background

Robots reflect an embodied AI. In industrial contexts, robots and humans already work side by side. Mainly, robots operate within a security zone to ensure the safety of human co-workers. However, collaboration or cooperation, including contact and interaction with robots, will gain importance. As mentioned above, investigating collaborative or cooperative human-robot interactions are complex due to myriads of gestalt variants, tasks, and security reasons. Besides, many different user groups with different needs and motives will work with robots. One crucial aspect of interaction is the sense of social intelligence in the artificial co-worker (Biocca, 1999). The scientific literature describes different cues implying the social intelligence of an artificial counterpart, such as starting a conversation, adaptive answering, or sharing personal experiences (Aragon, 2003; Terry and Garrison, 2007).

The present experiment manipulated the conversational ability to vary the sense of social intelligence of a simulated robot. The experiment asks: How does the sense of the robot’s social intelligence influence the perception of the robot and the evaluation of the interaction?

Method

Thirty-five participants (age in years: M = 22.00, SD = 1.91; 24 females) interacted with a simulated robot in an industrial XR environment (see Figure 4). All participants were students and received course credit for participation. The environment was created in Unity Engine Version 2019.2.13f1. The player interactions are pre-made and imported through the Steam-VR plugin. All assets (tools and objects for use in Unity, like 3D-models) were available in the Unity Asset Store. The Vive pro headset was used.

FIGURE 4

Collaboratively, they sorted packages with different colors and letters as fast and accurately as possible. First, the robot sorted the packages by color (yellow, cyan, pink, dark blue). Second, the robot delivered the packages with the right color to the participant. Third, the participant threw the packages into one of two shafts, either into the shaft for the letters A to M or into that for N to Z’s letters. A gauge showed the number of correct sorted packages. The performance of the robot was the same in all interactions.

Participants conducted two conditions in a within subject design while the order of the conditions was balanced. In the conversational condition (short: CR), the robot showed cues of social intelligence by starting a conversation, adaptive answering, and sharing personal preferences about small-talk topics (e.g., “Hi, I am Roni, we are working together today!”; “How long do you live here?”; What is your favorite movie?”). In a Wizard-of-Oz scenario, the robot answered adaptively to the reaction of the participants. The conversation ran alongside the tasks. In the control condition, the robot did not talk to the participant (short: nCR). The experimenter was present during the whole experiment.

To assess the perception of the robot, we measured the uncanniness on a five-point Likert-scale (Ho and MacDorman, 2010) with the subscales humanness, eeriness, and attractiveness. Furthermore, the sense of social presence was measured with the subscale social of the Bailenson social presence scale (Bailenson et al., 2004). Finally, the valence of the robot evaluation was assessed by the negative attitude towards robots scale (short: NARS), including the subscales S1 negative attitudes towards situations of interactions with robots, S2 negative attitudes toward the social influence of robots, S3 negative attitudes toward emotions in interaction with robots (Nomura et al., 2006). Participants gave their answers on a five-point Likert-scale. For data analyses, an overall score as the average of the subscales was built.

To assess the evaluation of the interaction, the user experience of the participants was measured. Four items assessed the pragmatic (e.g., “The interaction fulfilled my seeking for simplicity.”) and hedonic (e.g., “The interaction fulfilled my seeking for pleasure.”) quality based on the short version of the AttrackDiff mini (Hassenzahl and Monk, 2010). The eudaimonic quality was measured by four items (e.g., “The interaction fulfilled my seeking to do what you believe in.”) adapted from (Huta, 2016). Besides, a the social quality was assessed by four items (e.g., “The interaction fulfilled my seeking for social contact.”) based on (Hassenzahl et al., 2015). Additionally, NASA-TLX with the subscales mental demand, physical demand, temporal demand, performance, effort, and frustration was measured on a scale ranging from 0 to 100 (Hart and Staveland, 1988).

Since the experiment serves as an example, we followed an explorative data analysis by comparing the two conditions with an undirected paired t-Test. Further, some explorative moderator analyses, including participant’s gender as a moderator, were analyzed. For each analysis, the alpha level was set up to 0.05 to indicate the significance and to 0.2 to indicate significance by trend (Field, 2009).

Results

Table 4 summarizes the results. The robot with conversational abilities was evaluated as more human-like, attractive, social present, and positive. The evaluation of the interaction yielded mixed results. The pragmatic quality and the mental effort indicated a more negative evaluation of the conversational robot than the non-conversational robot. In contrast, the hedonic, and social quality, the perceived performance were evaluated more positively after interacting with the conversational robot than the non-conversational robot.

Furthermore, the results showed that gender matters in human-robot interaction (see Figure 5 and Figure 6). Women rated the conversational robot as more positive (significant) and attractive (by trend) than the non-conversational robot, while men did not show any differences. Regarding the interaction evaluation, men showed lower values of the pragmatic quality after interacting with the conversational robot, while women showed no difference for the conversational conditions. Moreover, women rated the interaction with the conversational robot as more hedonic, while men did not show any differences. Finally, the gradient regarding the social quality for the conversational robot was stronger for women than for men. In general, men showed higher ratings for the robot. Women only showed similar high ratings in the conversational robot condition.

FIGURE 5

FIGURE 6

Paradigmatic Experiment 2: Simulated Recommender System

Background

Besides robots, recommender systems are another important application of human-AI interaction. Two effects might influence the perception of such a system. The Eliza effect occurs when a system uses simple technical operations but produces effects that appear complex. Then, humans attribute more intelligence and competence than the system provides (Wardrip-Fruin, 2001; Long and Magerko, 2020). The Tale-Spin effect occurs when a system uses complex internal operations but produces effects that appear less complex (Wardrip-Fruin, 2001; Long and Magerko, 2020). However, systematic investigations of these effects are rarely.

The second experiment manipulated the appearance of an embodied recommender system to vary the sense of complexity. The experiment asks: How does the sense of complexity influence the perception of the recommender system?

Method

Thirty participants (age in years: M = 23.07, SD = 2.03; 13 females) interacted with a simulated recommender system in a virtual quiz game environment (see Table 3). All participants were students and received course credit for participation. The environment was created in Unity Engine Version 2019.2.13f1. The player interactions are pre-made and imported through the Steam-VR plugin. All assets (tools and objects for use in Unity, like 3D-models) were available in the Unity Asset Store. The Vive pro headset was used.

Participants answered 40 difficult quiz questions (Figure 7). They could choose from four answers. Participants decided under uncertainty. First, they had 10 s to guess and log in the answer. Then, an embodied AI recommended via voice output which answer possibility might be correct. The participant could decide whether they want to correct their initial choice. After the final selection, feedback about the correctness was given. Participants passed through a tutorial to learn the procedure, including ten test questions and interactions that the AI recommender system (conditions: correct, complex, and simple, see below). The experimenter was present during the whole experiment.

FIGURE 7

The recommender system was prescripted and split into two factors correctness (between-subject) and appearance (within-subject). The recommender system answered either 100% correct (high correctness, i.e., AI correct) or 75% correct (low correctness, i.e., AI incorrect). A sphere embodied the recommender system (see Figure 7). The sphere’s surface was either white plain, referring to a simple appearance, or metallic patterned, referring to a complex appearance. A pretest ascertained the distinction. The combination of the factors resulted in four conditions (see Figure 8).

FIGURE 8

To assess the perception of the recommender systems, participants rated the perceived competence (e.g., The AI was competent) and friendliness (e.g., The AI was friendly). The 7-point Likert-scales based on (Fiske et al., 2002). Further, they rated the perceived trustworthiness based on (Bär et al., 2011), including the subscales perceived risk (e.g., It is a risk to interact with the AI.), benevolence (e.g., I believe, the AI act to my good.), and trust (e.g., I can trust the information given by the AI.). To evaluate the interaction, the user experience of the participants was measured. Four items assessed the pragmatic (e.g., “The interaction fulfilled my seeking for simplicity.”) and hedonic (e.g., “The interaction fulfilled my seeking for pleasure.”) quality based on the short version of the AttrackDiff mini (Hassenzahl and Monk, 2010).

Since the experiment serves as an example, we followed an explorative data analysis by testing the main and interaction effects with a 2 × 2 MANOVA. For each analysis, the alpha level was set up to 0.05 to indicate significance and to 0.2 to indicate significance by trend (Field, 2009).

Results

Table 5 summarizes the results. Participants interacting with the 100% correct AI perceived more competence, trust, and less risk, independently of its appearance. The complex appearance led up to less perceived risk (a sign of higher trustworthiness). However, the simple appearance was rated friendlier, and more benevolent (in the correct condition). Finally, the interaction with the simple appearance rated as more hedonic (when interacting in the correct condition). Figure 9 and Figure 10 illustrate the results.

TABLE 5

shows the MANOVA-test results. Green indicates significant results and orange significance by trend (p < 0.20). F refers to the test statistic, p indicates the significance, and η² represents the effect size.

FIGURE 9

FIGURE 10

Discussion

AI-applications are omnipresent, and AI as a term certainly implies far-reaching connotations for many users, not experts. HCI approaches would suggest evaluating the impact of different AI applications, appearances, and operations on human perception of and interaction with AIs and identifying the effects these manipulations would have on users (Wienrich et al., 2021b). However, this approach, which is central to HCI principles, is challenging to realize due to the sheer complexity of many fields of applications of the real physical world. Another rising technology cluster, XR, accounts for control and systematic manipulations of complex interactions (Blascovich et al., 2002; Wienrich and Gramlich, 2020; Wienrich et al., 2021a). Hence, XR provides much potential to increase the investigability of human-AI interactions and interfaces.

The present paper suggests and discusses new perspective on the XR-AI combination space and XR as a new testbed for human-AI interactions and interfaces to establish a valid and systematic investigation procedures for human-AI interfaces and interactions. Four sub-questions structured the article’s first, theoretical, part (see Table 6). The second part presented two paradigmatic implementations of an XR testbed for human-AI interactions (see Table 7).

TABLE 7

Contribution	Research Question | Experimental Approach	Paradigmatic Results
Experiment 1 simulated a robot-human interaction in an industrial context	How does thesense of the robot’s social intelligence influence the robot’s perception and the evaluation of the interaction?	The conversational robot was evaluated as more social, human-like, and positive than the non-conversational robot
		The interaction with the conversational robot fulfilled more user motives than the non-conversational robot
	Manipulation of the conversational ability to vary the sense of social intelligence of a simulated robot	Female users evaluated the robot and the interaction differently from male users
Contribution	Research Question | Experimental Approach	Paradigmatic Results
Experiment 2 simulated a recommender system in a quiz game setting	How does the sense of complexity influence the perception of a recommender system?	The simple appearance was evaluated as friendlier and less risky
	—	The interaction with the simple AI was more hedonic than the complex AI (when the AI answered 100% correctly)
	Manipulation of the appearance of an embodied recommender system to vary the sense of complexity	—

summarizes the results of the article’s second part.

Contribution 1: A New Perspective on the XR-Artificial Intelligence Combination Space

Lieberman motivated an integrated combination of AI and computer graphics (Lieberman, 1996). Aylett and Luck (Aylett and Luck, 2000) denoted, more specifically, combinations of AI and XR technologies as Intelligent Virtual Environments. In both views, AI is conceptualized as enabling technology to improve the computer graphics or virtual environments. The XR-AI continuum extends the XR-AI combination space by introducing XR as a new investigability of HAII. It includes two perspectives on XR-AI combinations: Does the combination target AI as an object of investigation, e.g., to learn about the effects of a prospective AI embodiment on users? Or does the combination use AI as an enabling technology for XR, e.g., to provide techniques for multimodal interfaces? Often, XR-AI combinations will serve both tasks to some specific degree which allows to map them to the XR-AI continuum.

The XR-AI continuum provides a frame to classify XR-AI combinations. Such a classification serves multiple scientific purposes: First, it allows to systematically evaluate and rate different approaches of XR-AI combinations and hence it helps to structure the overall field. Second, it hopefully leads to an identification of best practices for certain approaches depending on the general perspective taken, and hence, third, that these best practices provide useful guidance, e.g., for replicating or reevaluating work and results. XR-AI combinations can drastically vary in terms of the required development efforts and complexities (Latoschik and Blach, 2008; Fischbach et al., 2017). However, many approaches targeting the eXtended AI range of the XR-AI continuum seem—in principle—to be realizable either by favorable loose couplings of the AI components or to require a rather limited completeness or development effort of the AI part. Regarding the latter, XR technology allows to adopt lightweight AI-mockups based on the Eliza principle or on Wizard-of-Oz scenarios to various degrees and to extensively investigate the relationship between humans and AIs as well as important usability and user experience aspects before an AI is fully functional. Hence, these investigations can be performed without the necessity to first develop a full-blown AI system and, in addition, an AI development process can be integrated with a user-centered design process to continuously optimize the human-AI interface. These rapid prototyping prospects lay the cornerstone to render XR as an ideal testbed to research human-AI interaction and interfaces.

Contribution 2: A New Testbed for Human-Artificial Intelligence Interactions and Interface

Limitations

Although, we have many indicators stemming from investigating human-human interactions in XR (Blascovich, 2002; Bombari et al., 2015; Pan and Hamilton, 2018) and product testing in XR, simulations only resemble real human-AI interactions. Hence, we cannot be sure that the results stemming from XR interactions precisely predict human-AI interactions in real settings. For establishing XR as valid testbed, comparative studies testing real interaction settings with XR interactions would be necessary. Additionally, the easiness of producing different variants of features might mislead researchers into engendering myriad results stemming from neglectable variation. The significance of the outcome would blow out of proportion. Since XR simulations can be set up in each laboratory, results might increasingly be collected with student samples instead of real users of the corresponding AI application. Thus, it seems important to make XR testbeds attractive and available for practical stakeholders and diverse user groups.

Conclusion

AI covers a broad spectrum of computational problems and use cases. At the same time, AI applications are becoming increasingly interactive. Consequently, AIs will exhibit a certain form of embodiment to users at the human-AI interface. This embodiment is manifold and can, e.g., range from simple feedback systems, to text or graphical terminals, to humanoid agents or robots of various appearances. Simultaneously, human-AI interactions gain massively in importance for diverse user groups and in various fields of application. However, the outer appearance and behavior of an AI result in profound differences on how an AI is perceived by users, e.g., what they think about the AI’s potential capabilities, competencies or even dangers, i.e., how user’s attitude towards an AI are shaped. Moreover, many users or future users do have abstract ideas of what AI is, significantly increasing the embodiment effects and hence the importance of the specific embodiment of AI applications. These circumstances raise intricate questions of why and how humans interact or should interact with AIs.

From a human-computer interaction and human-centered design perspective, it is essential to investigate the acceptance of AI applications and the consequences of interacting with an AI. However, the sheer complexity of application fields and AI embodiments, in reality, face researchers with enormous challenges regarding valid and systematic investigations. This article proposes eXtended AI as a method for such a systematic investigation. We provided a theoretical treatment and model of Human-AI interaction based in the introduced XR-AI continuum as a framework for and a perspective of different approaches of XR-AI combinations. We motivate eXtended AI as a specific XR-AI combination capable of helping us to learn about the effects of prospective human-AI interfaces and to show why the combination of XR and AI fruitfully contributes to a valid and systematic investigation of human-AI interactions and interfaces.

The article also provided two exemplary experiments investigating aforementioned approach for two distinct AI-systems. The first experiment reveals an interesting gender effect in human-robot interaction, while the second experiment reveals an Eliza effect of a recommender system. These two examples of paradigmatic implementations of the proposed XR testbed for human-AI interactions and interfaces show how a valid and systematic investigation can be conducted. It additionally reports on two interesting findings of embodied human-AI interfaces supporting the proposed idea to use XR as a testbed to investigate human-AI interaction. From an HCI perspective, eXtended AI (left side of the XR-AI continuum) constitutes a variant of rapid prototyping for HAII. The clue is that users interact with a prestage or a simulation instead of the fully developed product. In sum, the article opens new perspectives on how XR benefits for systematic investigations are essential for an evidence-based human-centered AI design, the design and evaluation of explainable AIs and tangible training modules, and basic research of human-AI interfaces and interactions.

References

• 1

  AntakliA.HermannE.ZinnikusI.DuH.FischerK. (2018). “Intelligent Distributed Human Motion Simulation in Human-Robot Collaboration Environments,” in Proceedings of the 18th International Conference on Intelligent Virtual Agents, IVA 2018 (New York: ACM), 319–324. 10.1145/3267851.3267867

  • CrossRef
  • Google Scholar

• 2

  AragonS. R. (2003). Creating Social Presence in Online Environments. New Dir. Adult Cont. Edu.2003, 57–68. 10.1002/ace.119

  • CrossRef
  • Google Scholar

• 3

  AylettR.LuckM. (2000). Applying Artificial Intelligence to Virtual Reality: Intelligent Virtual Environments. Appl. Artif. Intell.14, 3–32. 10.1080/088395100117142

  • CrossRef
  • Google Scholar

• 4

  AzmandianM.Arroyo-PalaciosJ.OsmanS. (2019). “Guiding the Behavior Design of Virtual Assistants,” in Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents, 16–18.

  • Google Scholar

• 5

  BailensonJ.AharoniE.BeallA.GuadagnoR.DimovA.BlascovichJ. (2004). “Comparing Behavioral and Self-Report Measures of Agents’ Social Presence in Immersive Virtual Environments,” in Proceedings of the 7th Annual International Workshop on PRESENCE, 1864–1105.

  • Google Scholar

• 6

  BärN.HoffmannA.KremsJ. (2011). “Entwicklung von Testmaterial zur experimentellen Untersuchung des Einflusses von Usability auf Online-Trust,” in Reflexionen und Visionen der Mensch-Maschine-Interaktion – Aus der Vergangenheit lernen, Zukunft gestalten. Editors SchmidS.ElepfandtM.AdenauerJ.LichtensteinA., 627–631.

  • Google Scholar

• 7

  BawdenD. (2008). “Origins and Concepts of Digital Literacy,” in Digital Literacies: Concepts, Policies and Practices Editors LankshearC.KnobelM.New York: Peter Lang. 30, 17–32.

  • Google Scholar

• 8

  BickmoreT.VardoulakisL.JackB.Paasche-OrlowM. (2013). “Automated Promotion of Technology Acceptance by Clinicians Using Relational Agents,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 68–78. 10.1007/978-3-642-40415-3_6

  • CrossRef
  • Google Scholar

• 9

  BioccaF. (1999). The Cyborg's Dilemma. Hum. Factors Inf. Technol.113, 113–144. 10.1016/s0923-8433(99)80011-2

  • CrossRef
  • Google Scholar

• 10

  BlascovichJ.LoomisJ.BeallA. C.SwinthK. R.HoytC. L.BailensonJ. N. (2002). TARGET ARTICLE: Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology. Psychol. Inq.13, 103–124. 10.1207/s15327965pli1302_01

  • CrossRef
  • Google Scholar

• 11

  BlascovichJ. (2002). “Social Influence within Immersive Virtual Environments,” in The social life of avatars (Springer), 127–145. 10.1007/978-1-4471-0277-9_8

  • CrossRef
  • Google Scholar

• 12

  BombariD.Schmid MastM.CanadasE.BachmannM. (2015). Studying Social Interactions through Immersive Virtual Environment Technology: Virtues, Pitfalls, and Future Challenges. Front. Psychol.6, 869. 10.3389/fpsyg.2015.00869

  • CrossRef
  • Google Scholar

• 13

  CameronJ. (1984). The Terminator. Hendale Film Corpoperation, Pacific Western Productions, Euro Film Funding, Cinema '84.

  • Google Scholar

• 14

  CarolusA.BinderJ. F.MuenchR.SchmidtC.SchneiderF.BuglassS. L. (2019). Smartphones as Digital Companions: Characterizing the Relationship between Users and Their Phones. New Media Soc.21, 914–938. 10.1177/1461444818817074

  • CrossRef
  • Google Scholar

• 15

  CarolusA.WienrichC. (2019). How Close Do You Feel to Your Devices? Visual Assessment of Emotional Relationships with Digital Devices. dl.gi.de.. 10.18420/muc2019-ws-652

  • CrossRef
  • Google Scholar

• 16

  CarpenterJ. (1974). Dark Star. Jack H. Harris Enterprise. Los Angeles, California: University of Southern California. 10.5962/bhl.title.84512

  • CrossRef
  • Google Scholar

• 17

  CavazzaM.LugrinJ.-L.PizziD.CharlesF. (2007). “Madame Bovary on the Holodeck: Immersive Interactive Storytelling,” in Proceedings of the 15th international conference on Multimedia MULTIMEDIA ’07 (New York, NY, USA: ACM), 651–660.

  • Google Scholar

• 18

  ClarkeA. C. (1962). Hazards of Prophecy: The Failure of Imagination. Profiles Futur6, 1

  • Google Scholar

• 19

  CongdonS. P.SchoberM. F. (2002). “How Examiners’ Discourse Cues Affect Scores on Intelligence Tests,” in Kansas City, MO: 43th Annual Meeting of the Psychonomic Society.

  • Google Scholar

• 20

  Cruz-NeiraC.SandinD. J.DeFantiT. A.KenyonR. V.HartJ. C. (1992). The CAVE: Audio Visual Experience Automatic Virtual Environment. Commun. ACM35, 64–72. 10.1145/129888.129892

  • CrossRef
  • Google Scholar

• 21

  DiSessaA. A. (2001). Changing Minds: Computers, Learning, and Literacy. Cambridge: Mit Press.

  • Google Scholar

• 22

  FanL.ScheutzM.LohaniM.McCoyM.StokesC. (2017). “Do we Need Emotionally Intelligent Artificial Agents? First Results of Human Perceptions of Emotional Intelligence in Humans Compared to Robots,” in International Conference on Intelligent Virtual Agents, 129–141. 10.1007/978-3-319-67401-8_15

  • CrossRef
  • Google Scholar

• 23

  FieldA. (2009). Discopering Statistics Using SPSS. Thrid Edition. Thousand Oaks, California: SAGE Publications.

  • Google Scholar

• 24

  FischbachM.WiebuschD.LatoschikM. E. (2017). Semantic Entity-Component State Management Techniques to Enhance Software Quality for Multimodal VR-Systems. IEEE Trans. Vis. Comput. Graphics23, 1342–1351. 10.1109/TVCG.2017.2657098

  • CrossRef
  • Google Scholar

• 25

  FiskeS. T.CuddyA. J. C.GlickP.XuJ. (2002). A Model of (Often Mixed) Stereotype Content: Competence and Warmth Respectively Follow from Perceived Status and Competition. J. Personal. Soc. Psychol.82, 878–902. 10.1037/0022-3514.82.6.878

  • CrossRef
  • Google Scholar

• 26

  FitrianieS.BruijnesM.RichardsD.BönschA.BrinkmanW.-P. (2020). “The 19 Unifying Questionnaire Constructs of Artificial Social Agents,” in Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020 (New York: Association for Computing Machinery, Inc), 1–8. 10.1145/3383652.3423873

  • CrossRef
  • Google Scholar

• 27

  FraserJ.PapaioannouI.LemonO. (2018). “Spoken Conversational Ai in Video Games: Emotional Dialogue Management Increases User Engagement,” in Proceedings of the 18th International Conference on Intelligent Virtual Agents, 179–184.

  • Google Scholar

• 28

  GallD.LatoschikM. E. (2018). “Immersion and Emotion: On the Effect of Visual Angle on Affective and Cross-Modal Stimulus Processing,” in Proceedings of the 25th IEEE Virtual Reality Conference (IEEE VR). 10.1109/vr.2018.8446153

  • CrossRef
  • Google Scholar

• 29

  HartS. G.StavelandL. E. (1988). “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research,” in Human Mental Workload. Editors HancockP. A.MeshkatiN. (Amsterdam: Elsevier), 139–183. 10.1016/s0166-4115(08)62386-9

  • CrossRef
  • Google Scholar

• 30

  HassenzahlM.MonkA. (2010). The Inference of Perceived Usability from beauty. Human-comp. Interaction25, 235–260. 10.1080/07370024.2010.500139

  • CrossRef
  • Google Scholar

• 31

  HassenzahlM.Wiklund-EngblomA.BengsA.HägglundS.DiefenbachS. (2015). Experience-Oriented and Product-Oriented Evaluation: Psychological Need Fulfillment, Positive Affect, and Product Perception. Int. J. Human-Computer Interaction31, 530–544. 10.1080/10447318.2015.1064664

  • CrossRef
  • Google Scholar

• 32

  HatfieldE.CacioppoJ. T.RapsonR. L. (1992). Primitive Emotional Contagion. Emot. Soc. Behav.14, 151–177.

  • Google Scholar

• 33

  HoC.-C.MacDormanK. F. (2010). Revisiting the Uncanny valley Theory: Developing and Validating an Alternative to the Godspeed Indices. Comput. Hum. Behav.26, 1508–1518. 10.1016/j.chb.2010.05.015

  • CrossRef
  • Google Scholar

• 34

  HutaV. (2016). “Eudaimonic and Hedonic Orientations: Theoretical Considerations and Research Findings,” in Handbook of eudaimonic well-being (Springer), 215–231. 10.1007/978-3-319-42445-3_15

  • CrossRef
  • Google Scholar

• 35

  JonasK.StroebeW.H.M. (2014). Sozialpsychologie. 6th ed.Berlin, Germany: Springer-Verlag.

  • Google Scholar

• 36

  KelleyP. G.YangY.HeldrethC.MoessnerC.SedleyA.KrammA.et al (2019). “Happy and Assured that Life Will Be Easy 10years from now.”: Perceptions of Artificial Intelligence in 8 Countries. Arxiv Prepr. arXiv2001.00081. 10.1145/3461702.3462605

  • CrossRef
  • Google Scholar

• 37

  KistlerF.EndrassB.DamianI.DangC. T.AndréE. (2012). Natural Interaction with Culturally Adaptive Virtual Characters. J. Multimodal User Inter.6, 39–47. 10.1007/s12193-011-0087-z

  • CrossRef
  • Google Scholar

• 38

  KubrickS. (1968). 2001: A Space Odyssey. Manhattan, New York: Stanley Kubrick Productions.

  • Google Scholar

• 39

  KuhlenA. K.BrennanS. E. (2013). Language in Dialogue: When Confederates Might Be Hazardous to Your Data. Psychon. Bull. Rev.20, 54–72. 10.3758/s13423-012-0341-8

  • CrossRef
  • Google Scholar

• 40

  KulmsP.KoppS. (2016). “The Effect of Embodiment and Competence on Trust and Cooperation in Human-Agent Interaction,” in International Conference on Intelligent Virtual Agents, 75–84. 10.1007/978-3-319-47665-0_7

  • CrossRef
  • Google Scholar

• 41

  LatoschikM. E. (2005). “A User Interface Framework for Multimodal {VR} Interactions,” in Proceedings of the IEEE seventh International Conference on Multimodal Interfaces (Trento, Italy: ICMI 2005). 10.1145/1088463.1088479Available at: http://trinity.inf.uni-bayreuth.de/download/pp217-latoschik.pdf

  • CrossRef
  • Google Scholar

• 42

  LatoschikM. E.BlachR. (2008). “Semantic Modelling for Virtual Worlds – A Novel Paradigm for Realtime Interactive Systems?,” in Proceedings of the ACM VRST 2008, 17–20. Available at: http://trinity.inf.uni-bayreuth.de/download/semantic-modeling-for-VW-VRST08.pdf.

  • Google Scholar

• 43

  LatoschikM. E.RothD.GallD.AchenbachJ.WaltemateT.BotschM. (2017). “The Effect of Avatar Realism in Immersive Social Virtual Realities,” in 23rd ACM Symposium on Virtual Reality Software and Technology (VRST), 39:1–39:10.

  • Google Scholar

• 44

  LatoschikM. E. (2014). Smart Graphics/Intelligent Graphics. Informatik Spektrum37, 36–41. 10.1007/s00287-013-0759-z

  • CrossRef
  • Google Scholar

• 45

  LatoschikM. E.WachsmuthI. (1998). “Exploiting Distant Pointing Gestures for Object Selection in a Virtual Environment,” in Gesture and Sign Language in Human-Computer Interaction, September 17-19, 1997 (Bielefeld, Germany: International Gesture Workshop), 185–196. 10.1007/BFb0052999

  • CrossRef
  • Google Scholar

• 46

  LatoschikM. E.WienrichC. (2021). Coherence and Plausibility, Not Presence?! Pivotal Conditions for XR Experiences and Effects, a Novel Model. arXiv. Available at: https://arxiv.org/abs/2104.04846.

  • Google Scholar

• 47

  LiebermanH. (1996). Intelligent Graphics. Commun. ACM39, 38–48. 10.1145/232014.232026

  • CrossRef
  • Google Scholar

• 48

  LongD.MagerkoB. (2020). “What Is AI Literacy? Competencies and Design Considerations,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16.

  • Google Scholar

• 49

  MattarN.Van WelbergenH.KulmsP.KoppS. (2015). “Prototyping User Interfaces for Investigating the Role of Virtual Agents in Human-Machine Interaction,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Springer-Verlag), 356–360. 10.1007/978-3-319-21996-7_39

  • CrossRef
  • Google Scholar

• 50

  MoriM.MacDormanK.KagekiN. (2012). The Uncanny valley [from the Field]. IEEE Robot. Automat. Mag.19, 98–100. 10.1109/mra.2012.2192811

  • CrossRef
  • Google Scholar

• 51

  NassC.GongL. (2000). Speech Interfaces from an Evolutionary Perspective. Commun. ACM43, 36–43. 10.1145/348941.348976

  • CrossRef
  • Google Scholar

• 52

  NomuraT.SuzukiT.KandaT.KatoK. (2006). Measurement of Negative Attitudes toward Robots. Is7, 437–454. 10.1075/is.7.3.14nom

  • CrossRef
  • Google Scholar

• 53

  ObaidM.DamianI.KistlerF.EndrassB.WagnerJ.AndréE. (2012). “Cultural Behaviors of Virtual Agents in an Augmented Reality Environment,” in 12th International Conference on Intelligent Virtual Agents (IVA 2012) LNCS. Editors NakanoY. I.NeffM.PaivaA.WalkerM. A. (Berlin, Germany: Springer), 412–418. 10.1007/978-3-642-33197-8_42

  • CrossRef
  • Google Scholar

• 54

  Ospina-BohórquezA.Rodríguez-GonzálezS.Vergara-RodríguezD. (2021). On the Synergy between Virtual Reality and Multi-Agent Systems. Sustainability13, 4326. 10.3390/su13084326

  • CrossRef
  • Google Scholar

• 55

  PanX.HamiltonA. F. d. C. (2018). Why and How to Use Virtual Reality to Study Human Social Interaction: The Challenges of Exploring a New Research Landscape. Br. J. Psychol.109, 395–417. 10.1111/bjop.12290

  • CrossRef
  • Google Scholar

• 56

  PeckT. C.SockolL. E.HancockS. M. (2020). Mind the Gap: The Underrepresentation of Female Participants and Authors in Virtual Reality Research. IEEE Trans. Vis. Comput. Graphics26, 1945–1954. 10.1109/TVCG.2020.2973498

  • CrossRef
  • Google Scholar

• 57

  PhamD. T.GaultR. S. (1998). A Comparison of Rapid Prototyping Technologies. Int. J. Machine Tools Manufacture38, 1257–1287. 10.1016/S0890-6955(97)00137-5

  • CrossRef
  • Google Scholar

• 58

  RatanR.BeyeaD.LiB. J.GracianoL. (2019). Avatar Characteristics Induce Users' Behavioral Conformity with Small-To-Medium Effect Sizes: a Meta-Analysis of the proteus Effect. Media Psychol.23, 1–25. 10.1080/15213269.2019.1623698

  • CrossRef
  • Google Scholar

• 59

  RazzoukR.ShuteV. (2012). What Is Design Thinking and Why Is it Important?Rev. Educ. Res.82, 330–348. 10.3102/0034654312457429

  • CrossRef
  • Google Scholar

• 60

  ReevesB.NassC. (1996a). The media Equation: How People Treat Computers, Television, and New media like Real People. Cambridge, UK: Cambridge University Press.

  • Google Scholar

• 61

  ReevesB.NassC. (1996b). The media Equation: How People Treat Computers, Television, and New media like Real People and Places. Cambridge, UK: Cambridge University Press.

  • Google Scholar

• 62

  RieserV. (2019). “Let’s Chat! Can Virtual Agents Learn How to Have a Conversation?,” in Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents, 5–6.

  • Google Scholar

• 63

  RussellS. J.NorvigP. (2020). Artificial Intelligence: A Modern Approach. 4th ed.. Hoboken, New Jersey: Prentice-Hall.

  • Google Scholar

• 64

  SchreierJ. (2012). Robot & Frank. Stage 6 Films, Park Pictures, White Hat Entertainment, Dug Run PicturesNew York: Park Pictures.

  • Google Scholar

• 65

  ScottR. (1979). Alien. Brandywine Productions.Los Angles: Brandywine productions. 10.6028/nbs.sp.480-12

  • CrossRef
  • Google Scholar

• 66

  ScottR. (1982). Blade Runner. The Ladd Company, Shaw Brothers, Blade Runner Partnership. Los Angles: The Ladd Company.

  • Google Scholar

• 67

  SkarbezR.BrooksF. P.Jr.WhittonM. C. (2018). A Survey of Presence and Related Concepts. ACM Comput. Surv.50, 96:1–39. 10.1145/3134301

  • CrossRef
  • Google Scholar

• 68

  SlaterM. (2009). Place Illusion and Plausibility Can lead to Realistic Behaviour in Immersive Virtual Environments. Phil. Trans. R. Soc. B364, 3549–3557. 10.1098/rstb.2009.0138

  • CrossRef
  • Google Scholar

• 69

  SlaterM.WilburS. (1997). A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators & Virtual Environments6, 603–616. 10.1162/pres.1997.6.6.603

  • CrossRef
  • Google Scholar

• 70

  SternaR.CybulskiA.Igras-CybulskaM.PilarczykJ.KunieckiM. (2021). “Pretest or Not to Pretest? A Preliminary Version of a Tool for the Virtual Character Standardization,” in 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (Danvers: The Institute of Electrical and Electronics Engineers), 123–126. 10.1109/vrw52623.2021.00030

  • CrossRef
  • Google Scholar

• 71

  StrathmannC.SzczukaJ.KrämerN. (2020). “She Talks to Me as if She Were Alive: Assessing the Social Reactions and Perceptions of Children toward Voice Assistants and Their Appraisal of the Appropriateness of These Reactions,” in Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, 1–8.

  • Google Scholar

• 72

  TerryL. R.GarrisonA. D. R. (2007). Assessing Social Presence in Asynchronous Text-Based Computer Conferencing J. J. Distance Educ.14, 1–18.

  • Google Scholar

• 73

  TopálJ.GergelyG.MiklósiA.ErdohegyiA.CsibraG. (2008). Infants' Perseverative Search Errors Are Induced by Pragmatic Misinterpretation. Science321, 1831–1834. 10.1126/science.111363410.1126/science.1161437

  • CrossRef
  • Google Scholar

• 74

  WaltemateT.GallD.RothD.BotschM.LatoschikM. E. (2018). The Impact of Avatar Personalization and Immersion on Virtual Body Ownership, Presence, and Emotional Response. IEEE Trans. Vis. Comput. Graphics24, 1643–1652. 10.1109/tvcg.2018.2794629

  • CrossRef
  • Google Scholar

• 75

  Wardrip-FruinN. (2001). Three Play Effects – Eliza, Tale-Spin, and SimCity. Digit. Humanit.1–2.

  • Google Scholar

• 76

  WienrichC.DöllingerN.HeinR. (2021a). Behavioral Framework of Immersive Technologies (BehaveFIT): How and Why Virtual Reality Can Support Behavioral Change Processes. Front. Virtual Real.2, 84. 10.3389/frvir.2021.627194

  • CrossRef
  • Google Scholar

• 77

  WienrichC.GramlichJ. (2020). appRaiseVR - an Evaluation Framework for Immersive Experiences. i-com19, 103–121. 10.1515/icom-2020-0008

  • CrossRef
  • Google Scholar

• 78

  WienrichC.GrossR.KretschmerF.Müller-PlathG. (2018b). “Developing and Proving a Framework for Reaction Time Experiments in VR to Objectively Measure Social Interaction with Virtual Agents,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (Danvers: The Institute of Electrical and Electronics Engineers), 191–198.

  • Google Scholar

• 79

  WienrichC.ReitelbachC.CarolusA. (2021b). The Trustworthiness of Voice Assistants in the Context of Healthcare Investigating the Effect of Perceived Expertise on the Trustworthiness of Voice Assistants, Providers, Data Receivers, and Automatic Speech Recognition. Front. Comput. Sci.3, 53. 10.3389/fcomp.2021.685250

  • CrossRef
  • Google Scholar

• 80

  WienrichC.SchindlerK.DöllingerN.KockS. (2018a). “Social Presence and Cooperation in Large-Scale Multi-User Virtual Reality-The Relevance of Social Interdependence for Location-Based Environments,” in IEEE Conference on Virtual Reality and 3D User Interfaces (Danvers: The Institute of Electrical and Electronics Engineers), 207–214.

  • Google Scholar

• 81

  YanX.GuP. (1996). A Review of Rapid Prototyping Technologies and Systems. Computer-Aided Des.28, 307–318. 10.1016/0010-4485(95)00035-6

  • CrossRef
  • Google Scholar

• 82

  YeeN.BailensonJ. (2007). The Proteus Effect: The Effect of Transformed Self-Representation on Behavior. Hum. Comm. Res. 33, 271–290. 10.1111/j.1468-2958.2007.00299.x

  • CrossRef
  • Google Scholar

• 83

  ZhangB.DafoeA. (2019). Artificial Intelligence: American Attitudes and Trends. SSRN J.10.2139/ssrn.3312874

  • CrossRef
  • Google Scholar

• 84

  ZhangH.FrickerD.YuC. (2010). “A Multimodal Real-Time Platform for Studying Human-Avatar Interactions,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 49–56. 10.1007/978-3-642-15892-6_6

  • CrossRef
  • Google Scholar

• 85

  ZimmererC.FischbachM.LatoschikM. E. (2018b). “Space Tentacles - Integrating Multimodal Input into a VR Adventure Game,” in Proceedings of the 25th IEEE Virtual Reality (VR) conference (IEEE), 745–746. 10.1109/vr.2018.8446151Available at: https://downloads.hci.informatik.uni-wuerzburg.de/2018-ieeevr-space-tentacle-preprint.pdf

  • CrossRef
  • Google Scholar

• 86

  ZimmererC.FischbachM.LatoschikM. (2018a). Semantic Fusion for Natural Multimodal Interfaces Using Concurrent Augmented Transition Networks. Multimodal Tech. Interaction2, 81. 10.3390/mti2040081

  • CrossRef
  • Google Scholar