There is no unified definition of MR (Speicher and Nebeling, 2019). The most well-known and most cited definition in academic research is that of Milgram and Kishino (1994). The authors formulate MR as “a particular subset of Virtual Reality (VR) related technologies that involve the merging of real and virtual worlds somewhere along the “virtuality continuum” which connects completely real environments to completely virtual ones.”. Another key feature of MR environments, according to Milgram and Kishino (1994), was that in “a Mixed Reality environment [...] objects are presented together within a single display”. Finally, the authors themselves defined their primary work as: "non-exhaustive examples of existing display systems in which real objects and virtual objects are displayed together.”. Milgram and Colquhoun (1999) reiterate the original notion of Milgram and Kishino (1994) by explicitly excluding the extrema on this continuum (“completely real” and “completely virtual”) from their MR definition. MR stated as such, therefore only includes environments where real and virtual content is being mixed. Further, Milgram and Colquhoun (1999) extend the conceptualization of MR by showing the mixed reality combination space, presenting different combinations of real and virtual content within the same display. They firstly show how different types of hybrid displays may be mapped onto the MR combination space, but also go on to “extend the concept somewhat” by describing a journey along with the combination space within a single application. They never explicitly state such a transition as an MR application, but use it as an example of why the terms “AR” and “AV” cannot always clearly distinguish different states on the virtuality continuum and formulate the need for the broader definition of MR.

Others conceived of MR as even broader by including non-technological mediated phenomena such as realities perceived in dreams or drug experiences. Examples are seen in the work of Hillstead (2017), Mann (2002), and Mann and Nnlf (1994). However, these ideas are out of the scope of this paper because the present considerations focus on technic-mediated perceptions of reality.

More recently, Speicher et al. (2019) analyzed the usage of MR in academia. Through expert interviews and a literature review, they tried to gather different views on the term MR. Researchers used the definition of Milgram and Kishino (1994) most widely. Other conceptualizations included MR either as a synonym for AR, a strong AR, a combination of AR and VR, a type of collaboration, or an alignment of environments. The authors concluded that MR could mean many different things, depending on the context. Further, they recognized the need for researchers to clearly describe their understanding of the term MR within the context of their work. For that purpose, Speicher et al. (2019) described five main MR dimensions: 1) the number of environments (one or many), 2) the number of users (one or many), 3) the level of immersion (not, partially, entirely), 4) the level of virtuality (not, partially, entirely), 5) the degree of interaction (implicit, explicit). In addition, they introduced input and output as lower-level dimensions. Although Speicher and colleague’s work contributes to the clarity of MR in academic research, transitions (combinations of real and virtual content) are not explicitly discussed. However, as introduced by Billinghurst et al. (2001) , transitional interfaces or the MR headset’s see-through function cannot be classified with the dimensions. Transitional interfaces allow the interpolation between physical real and virtual environments, and they meet the definition of MR applications as introduced by Milgram and Colquhoun (1999). In addition, the dimensions remain unclear in their descriptions of space and interactions in MR, metaphors being essential for discussions about the meaning of the place-illusion and plausibility-illusion in MR.

Only a few scientific publications address the intersection between spatial presence and MR. Wagner et al. (2009), for example, compared three different MR applications. The applications, MapLense, a mobile AR system, TimeWarp, an augmented reality game, and MR Tent, were not classified by the authors. Schaik et al. (2004) examined a collaborative MR application, i.e., Dessert Rain. The authors argued that evaluating these MR applications using the standard methods established in VR scenarios (see above) is not necessarily meaningful. Despite the recommendation to use other factors such as ecological and cultural factors, they did not provide any solution to conceive spatial presence in MR.

Moreover, the work of Schaik et al. (2004) and Wagner et al. (2009), despite their scientific contributions, demonstrated, again, a problem in the research of MR applications. There is no unified definition of the term MR, and because of this scientific investigation collected under the term may vary considerably in content and or scope. Also, contributions such as that of Billinghurst et al. (2001) are mostly hidden to researchers because they lack MR as a label. The understanding of the term MR remains unclear, and the investigated MR applications incomplete. For example, to the author’s best knowledge, the spatial presence was not the object of research for interfaces allowing the interpolation between physical real and virtual environments as defined by Milgram and Colquhoun (1999).

The emergence of virtual presence is often determined by the allocation of attention to the virtual environment and the occlusion of the physical environment. Wirth et al. (2007) define this allocation/occlusion process as reference setting using the (virtual or physical) environment as referential cues. However, virtual reality is only one possible variant of the mixed reality continuum (Milgram and Kishino, 1994). Other variants have less immersive features and continuously interfere with the real physical space. Those variants that allow for fluent transitions between virtual and physical realities question the interplay between virtual and physical referential cues and their consequences for corresponding outcomes (e.g., the sense of transportation, the sense of realism). The present work aims to analyze the construct of presence and the interplay of virtual and physical referential cues in the context of MR. In the following, we derive a working definition of MR implying new conceptional ideas for the reference setting beyond full immersive VR and environmental cues. In our opinion, detaching from environmental entities as anchor cues is the essential assumption to discuss the spatial presence in MR. Based on these new ideas, we identify three research desiderata, including research questions about the referential power of entities occurring in MR, the corresponding dimension of transportation (i.e., place-illusion in VR), and the dimension of realism (i.e., plausibility-illusion in VR) in MR. Finally, we suggest an experimental setup to reveal the research heuristic behind experiments investigating presence in MR. In sum, the current work presents an alternative conceptional idea of reference setting in MR which raises the question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere? It also presents a research heuristic to investigate the idea and resulting question.

Similarly to Mann (2002) and Hillstead (2017), the scope of the present work refers to the role of spatial presence concerning their fourth area, i.e., computer-generated realities. The role of spatial presence in the first order, second-order, or mediated realities such as the sense of spatial presence during dreams, reading books, or watching movies are outside of the scope.

In the following of the present work, the working definition of MR is considered as concrete computer-generated virtual reality traversing the modified Reality-Virtuality-Continuum (short mRVC). Similar to the continuum defined by Milgram and Colquhoun (1999), the poles of the mRVC are defined as natural or physical reality (R) and virtuality (V). While the poles R and V represent abstract and theoretical forms of reality, the positions between them refer to concrete forms of reality illusions (Figure 1). Each of these concrete forms of reality illusion defines a mixed reality illusion at one time point. MR must include a merging of real and virtual qualities.

In addition to Milgram and colleague’s definitions, firstly, each reality illusion includes entities reaching from real to virtual manifestations at one time point. Entities can refer to spatial environments (i.e., space-based) or concrete objects (i.e., object-based). Secondly, referential cues determine the selection of the reference frame, e.g., allocentric vs egocentric and inside-out vs outside-in. Cues are stemming from the presented entities themselves but also from introduction texts or other framing experiences. We only refer to referential cues stemming from the entities presented as real or virtual manifestations in an MR experience. Thus, spaces can be entities and can serve as cues simultaneously. In previous views, for example, virtual objects represented in virtual spaces refer to VR. Real objects represented in virtual spaces refer to an AV and virtual objects in real space to an AR. Based on the views about virtual presence presented above, the sense of virtual spatial presence has been implicitly anchored in the virtual space. Thus, the virtual spatial environmental entity (space-based) is the cue determining the frame of reference. However, assuming the spatial environmental entity is the cue determining the frame of reference reduces convincibility in MR (Figure 1). Why should the virtual environment in AV serve more likely as an anchor cue than the real object user interacting with? Thus, the view presented here overcomes the distinction of environments and objects by introducing entities possessing different degrees of referential power.

The referential power is defined as a weight indicating the probability of each entity or a class of entities (such as the spatial environment) to be selected as a referential cue. These probabilities are not independent of each other but are context-sensitive, and are a result of a given spatial configuration of entities and the location of the user relative to these entities, i.e., her current perspective she has on the entities. For example, an egocentric outside-in view of an entity that does not convey any self-location - imagine a manipulation of a CAD object of an engine part in front of the user - will most likely not elicit a space-based sensation of a being-there. In contrast, simple entities resembling floor tiles, geometrically arranged in congruence with a user’s perceived floor will most likely elicit a space-based sensation. When multiple entities are perceivable by the user, the question becomes how their referential power compares, how strong the respective cues are in a relation, how congruent they are to each other, and–in the case of the various XR variants defined by the mRVC, the ratio between the real physical and the simulated entities and their relative coverage of a user’s field of view.

It follows from the preceding that spatial environmental entities and objects theoretically can serve as referential anchor cues but that the actual effect is dependent on a combination of various conditions as given for a specific scene and entity configuration. In our opinion, detaching from environmental entities as anchor cues is the essential assumption to discuss spatial presence in MR. From this point of view, we discuss in the following subsection how we can operationalize the degree of real or virtual entity manifestations of MR experiences as one possibility to determine objectively the position of the mRVC. Subsequently, we discuss three research desiderata addressing the referential power of entities.

Although the authors agree with Milgram and Colquhoun’s (1999, p. 8) assertion that " [...] determining whether an image should be considered augmented reality or augmented virtuality is also not necessarily a matter of simply summating the respective areas of real and virtual images […]", the mRVC visualized areas defined by the amount of real and virtual picture portions at one time point (Figure 2, upper panel). Figure 2 (lower panel) shows a stepwise interpolation of Varjo demonstration on the mRVC (Varjo, 2019). The mRVC uses a naive percent-visualization of real and virtual picture portions, with 100% referring to reality. A higher portion of cues engendering the perception of real entities leads to a more left-side position on the mRVC. In contrast, cues engendering the perception of virtual entities lead to a more right-side position on the mRVC. Notably, experiences refer to a continuous stream of feelings, thoughts, and actions (Kahneman et al., 1999). Hence, this position finding does not reflect user’s experiences but helps to define and classify reality illusion and thus support systematic testing. Further, such a definition cannot capture subjective attentional allocation to particular areas of interest (e.g., interaction areas). Nevertheless, the possibility to define and classify reality illusions might be one crucial step in enabling a systematic investigation of the entity’s referential power and spatial presence in MR.

In the following, we consider the relationship between the main aspects of presence in VR, i.e., the place-illusion, and the plausibility-illusion, and the entities occurring in the mRVC, i.e., space-based and object-based, to discuss the concept, measures, and design of spatial presence in MR by considering the referential power of the different entities.

More or less directly, considerations about the presence in VR insert the virtual space as an anchor or reference. Often, the real environment serves as a kind of comparison-reference that should enable users to report their virtual experience (e.g., in the IPQ the user is asked whether “the virtual world seemed more realistic than the real world.”, Schubert et al., 2016). Nevertheless, the virtual and real environments are considered as opponents–only one can be the only (or at least the dominant) reference to judge the experience’s plausibility. Others framed the selection of reference as an attentional allocation process. If the virtual environment obtains (more) attention, users feel (more) present in it. In contrast, MR allows for a fluent transition between virtual and physical realities questioning the view of opponent realities. On the other hand, overcoming the distinction between environments and objects (by introducing entities possessing different degrees of referential power) leads to the RD1, including the following research questions:

C-RQ 1.1: Do users have/need a dominant reference frame in MR experience?

The authors identify three views on the RD1 described in the following.

1) One view to answer the research questions might be the view of Wirth et al. (2007). The authors assumed the place-illusion, i.e., the self-positioning in a real or virtual environment, determines the primary egocentric reference frame (PERF). Following this assumption, the user’s positioning in a real space would lead up to the user’s perception as part of the real space (space-PERF = R). In contrast, positioning the user in a virtual space would lead up to the user’s perception as part of the virtual space (space-PERF = V). However, in mixed realities, besides the self-positioning, the positioning of the objects further determines the perception of the illusion. Consequently, the positioning of objects in the real space would also lead up to the perception that they are part of the real place (left side of the mRVC, see Figure 1). In contrast, the positioning of objects in the virtual space would lead up to the perception that they are part of the virtual place (right side of the mRVC, see Figure 1). Thus, one view might be that an MR illusion emerges by integrating virtual objects into the real space (space-PERF = R). Then the user accepts the virtual objects as a part of the real space. In contrast, another MR illusion emerges by integrating real objects into the virtual space (space-PERF = V). Then the user accepts the real objects as a part of the virtual space. In this view, space-based entities would possess more referential power, become more likely to be referential cues, and determine the PERF more likely than object-based entities.

Both views would assume one dominant reference frame, either the quality of space-based or object-based entity. The former view would result in similar considerations about MR measures and design as it is in VR. In contrast, the latter view would result in new considerations about measures and design in MR. However, established spatial-related presence measures in VR assess transportation dimension and ask more or less after the degree of being in the experienced VR. In our opinion, transportation plays a unique role in experiencing MR (see 4.2). Instead of referring to the sense of transportation, we would argue for including direct questions about the reference frame or rather about the referential power of entities. One operationalization could be to assess the user’s expectations or breaks in expectations relating to the place-illusion emerged by a real or virtual space or the place spanned by the object-based entities. Those questions change the balance when a real and a virtual (space-based, object-based) PERF would make a huge difference.

It would be essential for design to know if users form a PERF and if the space-based or the object-based entities determine the reference frame. Then the MR application should include easy to perceive cues indicating clearly to a (space, object) R-PERF or a (space, object) V-PERF. For example, if users form a space-R-PERF, objects should cast a shadow in a way that would be plausible under real light conditions.

3) A third view of answering the research questions might be that users alter the PERF continuously or just do not have one. Then probably, the amount of real and virtual picture portions during the experience determine the position on the mRVC. In this view, space could be seen as the sum of its objects. The higher the number of virtual entities (including space as one cue), the more likely the user experiences a position on the right side of the mRVC (Figure 1). Then users would expect either real or virtual entity behavior and would not experience breaks by transitions. Measuring the absence of an experience is challenging (see below). For design, in contrast, this possibility enables much openness.

The dimension of transportation refers to the place-illusion in VR (i.e., the sense of being there, Steuer, 1992). Although Lombard and Ditton (1997) distinguished between three different types of transportation, spatial presence in VR has been mainly connected to the first type, i.e., self-transportation to another place. However, in MR, self-transportation might only be one aspect of transportation (egocentric inside-out). The second type, addressing the transportation of another place and objects to the user, probably draws more attention to objects-based entities (egocentric outside-in). The sense of spatial presence might also be determined by the feeling of sharing the MR space with virtual or real objects emphasizing the RD2, including the following research questions:

C-RQ 2.1: Does the sense of spatial presence in MR include the first and the second type of transportation?

The authors detected three views on the RD2 described in the following.

1) The established self-transportation probably refers to the views regarding the rest frame described above. In VR, transportation is closely related to a change of space within the VR experience (e.g., from virtual Europe to virtual Africa) or between the real and the virtual space (e.g., at the beginning of an experience when putting on the head-mounted display (HMD)). In MR, the sense of transportation can be similarly described when users select one dominant spatial (space-) reference frame. Then metaphors indicating a change of space would be similarly appropriate (e.g., portals). Thus, similar questions measuring spatial transportation might also be valid.

The dimension of realism refers to the plausibility-illusion (i.e., the sense of being there plus; Skarbez et al., 2017). In VR, plausibility addresses mainly objects, entities, and events that make sense in the place accepted as real (PERF). MR includes incongruences between the space-based and object-based entities by definition emphasizing the RD3, including the following research questions:

C-RQ 3.1: What can plausibility mean in MR defined by incongruences between the space-based and the object-based entities?

The authors identify the following three views on the RD3.

1) The sense of plausibility probably refers to the views regarding the rest frame described above. In VR, the plausibility-illusion is closely related to the extent to which an entity makes sense in the place that has been accepted as real (PERF). In MR, the sense of plausibility can be similarly described when users select one dominant spatial (space-) reference frame. Then incongruent entities (i.e., virtual objects in a real space or real objects in a virtual space) would be perceived as plausible if they behave coherently to the dominant place-illusion (i.e., the selected space-PERF). For example, in AR, virtual objects should fall downwards and not hang in the air. Measures and design considerations might be similar to corresponding considerations in VR.

In sum, in MR, object-based entities might play an additional role in MR to the space-based entities in VR. Similar to in VR experiences, when users form a dominant reference frame, the question arises as to whether the space-based or the object-based entities determine it. Then, questions around the dimensions of transportation and realism could either be similar to VR or draw more attention to the object-based entity. Finally, users might not need or have a dominant reference frame or alternate it continuously. Then the construct of transportation could be doubted in general. The dimension of realism might also change since exceptionally incoherent sets of circumstances and not coherent sets might define the plausibility of MR.

The research desiderata and corresponding considerations lead up to the final question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere?

Since immersion, defined as objective system factors, influences the place-illusion in VR (e.g., Skarbez et al., 2017; Wienrich and Gramlich, 2020), it might also play a significant role in MR (except the characteristic of inclusiveness). One good practice in VR research is to hold confounding immersive factors constant such as the kind of display. For example, when researchers want to investigate the sense of agency in dependency of the virtual embodiment, the same HMD should be worn in all embodiment conditions to control undesirable impacts of the HMD. Similarly, experiments investigating merging real and virtual entities in MR should follow the same good practice. Thus, if researchers explore the feeling of spatial presence in MR, the display properties of the system should remain the same over the application runtime to avoid confounding immersive factors. Currently, there is only one way to use the same system to represent both fully immersive virtual environments, as well as the real environments, and each possible merging in between. This possibility is the use of see-through HMDs, i.e., fully immersive VR glasses, which are able to display the real environment through cameras.

The questions about the reference frame in MR might be the most significant one of our considerations presented above. Hence, we present a paradigmatic experiment concerning the first research desideratum in the following.

The task of participants might be to search and merge objects by color. For that, they have to move around, find objects, and bring them to an object with the same color. During the experiment, participants would experience at least six transitions on the mRVC. Each transition reflects a systematic variation, either of a space-based or object-based entity (independent variable). To control for order-effects, the order of transitions would be balanced. One order of transitions is shown in Table 1. Before and after each transition, questions including the set, or the change of reference frame would be assessed (first-order dependent variable). Notably, the question must be assessed without switching off the HMD due to the above-described reasons. In addition, the sense of transportation (i.e., sense of place-illusion) or the sense of realism (i.e., sense of plausibility-illusion) could be assessed (second-order dependent variables).

The results of the experiments would bring primary answers to C-RQ 1.1 ( Do users have/need a dominant reference frame in MR experience?) and C-RQ 1.2 (Does the space-based or object-based entities determine the reference frame in MR experiences?). Furthermore, the results would indicate how the sense of transportation (i.e., place-illusion in VR) or realism (i.e., plausibility-illusion in VR) are evaluated in MR.

Plenty of considerations, models, measures, and investigations target the understanding of the sense of presence in VR. However, full-immersive virtual reality is only one possible variant of the mixed reality continuum (Milgram and Kishino, 1994; Milgram and Colquhoun, 1999). Other variants have less immersive features (are less inclusive) and continuously interfere with the real physical space questioning the applicability of the accumulated knowledge about virtual presence and corresponding outcomes. The current work presents an alternative conceptional idea of reference setting in MR which raises the question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere? It also presents a research heuristic to investigate the idea and resulting question.

To achieve the aim, we 1) presented a short review of definitions, dimensions, and measurements of presence in VR, and 2) presented the state of the art views on MR. Furthermore, we 3) derived a working definition of MR implying new conceptional ideas for the reference setting beyond full immersive VR and environmental cues. In our opinion, detaching from environmental entities as anchor cues is the essential assumption to discuss the spatial presence in MR. Based on these new ideas, we 4) identify three research desiderata, including research questions about the referential power of entities occurring in MR, the corresponding dimension of transportation (i.e., place-illusion in VR), and the dimension of realism (i.e., plausibility-illusion in VR) in MR. Finally, we 5) suggest an experimental setup to reveal the research heuristic behind experiments investigating presence in MR.

Five main limitations characterize the present contribution.

1) The first limitation refers to the focus on spatial presence. As mentioned above, presence is a broad construct, including, besides spatial presence (focus here), social presence (e.g., Lee, 2004), or cognitive presence (Nunez and Blake, 2001), for example. Furthermore, this paper considers spatial presence in the context of technologically mediated realities such as VR, AR, AV, and MR and excludes senses caused by dreams or drugs. Consequently, the discussions are restricted to the conceptional focus set by current debates of spatial presence in VR. The place-illusion and plausibility-illusion (Slater, 2009; Skarbez et al., 2017) and corresponding dimensions (Lombard and Ditton, 1997) combined with the idea of a primary (space-based) egocentric reference frame (Wirth et al., 2007) guided the presented research questions. Thus, scrutinizing current views on presence in VR was out of the scope here. Future work should examine other sub-constructs that refer to presence to consider their application for evaluating MR experiences. Similarly, revisited views on the concept of presence, such as the view of Latoschik and Wienrich (2021) should be incorporated in the future.