Staging Location-Based Virtual Reality to Improve Immersive Experiences

Location-based virtual reality (LBVR) is promising to offer the full potential of virtual reality (VR) because it is not completely virtual, as in space independent VR, which usually relies on a head-worn display and hand-held controllers only. Therefore, LBVR can include and arrange the physical surrounding according to a particular application or include parameters, such as objects or light conditions, of the physical surrounding into the virtual environment. While LBVR is drawing a lot of attention in the creative industry and a lot of LBVR-installations are already entertaining millions of users, not much research has been employed in this direction. In this work, we offer an overview containing relevant challenges and accompanying solutions that need to be considered in an LBVR-installation. Received on 19 January 2020; accepted on 28 January 2020; published on 24 February 2020


Introduction
While there is still a debate going on if immersive virtual reality (VR) technologies will become part of our daily lives such as TVs, computers or smartphones, VR is already widely used and already entertaining millions of users-not in our homes, but at public or semi-public spaces.In contrast to VR used in private settings, those location-based VR (LBVR) installations have additional requirements but also offer additional possibilities.They can consist of a customdesigned space, arranged to fit the particular needs of the application.LBVR can improve your experience because part of the physical world is becoming part of the VR experience.This allows items from the physical world to be rediscovered or virtual objects to be touched.In contrast to LBVR space independent VR is usually completely virtual-except for the worn display and hand-held controllers.VR unfolds its full potential only in those cases when the physical surrounding is also taken into consideration.LBVR offers to combine head-mounted display (HMD) with e.g.roller coasters, bumper cars or other activities that are seemingly impossible to experience at home because it requires complex hardware constructions.Therefore, for high-end VR installations, museums, exhibitions, and amusement parks are perfectly suitable for staging VR installations in the most engaging way [12].It has been demonstrated that consisting and correctly mapped mixed environments where virtual and real are intermingled-if done right-can improve presence and overall experience [11,13].In the context of LBVR, one is able to create spectacular, immersive experiences, whereby the complete isolation of the physical outside world at the same time is a major hurdle for wearing HMDs.This has to be overcome, especially for cases where one might be under possible observation such as semi-public spaces.Besides a "leap into the unknown" VR technology tends to give rise to uncertainty in regards to possible motion sickness and also to hygienic concerns that prevent many potential visitors from using an HMD in public settings.

(Re)Connecting Both Worlds
Building artificial worlds that can be entered and experienced is an old dream of mankind.The first realizations of this dream were already implemented as panoramic paintings more than 200 years ago.Later, all kinds of tricks, constructions, and projections of images were used so that the public could feel a kind of movement through time and space.As early as in the year 1900 the Maréorama, as well as the Cinéorama, were created as entertainment attraction for the 1900 Paris Exposition.The former was created by Hugo d'Alesi using combined moving panoramic paintings and a large motion platform to create the illusion of a ship cruise, the latter was created by Raoul Grimoin-Sanson using 10 synchronized 70 mm film projectors to create the illusion of a hot air balloon ride [23].While the first installations were done using large scale canvases with projections and thus generating a joint multi-person experience, the majority of today's immersive experiences relies on HMDs and thus today's experiences are isolating users from their surroundings and company [25].This gap between the headset user and the audience exists because in those cases the virtual content is not presented to the public in an appropriate way-or even worse-not at all.In those cases where the virtual world is shown to the public, it is common to present it in a first-person view on a screen mounted in the vicinity of the person using the HMD.This ego-perspective, however, makes it difficult for others to understand what is happening because this type of visualization lacks to articulate the spatial layout of the current scene, the image itself is unsteady and offers a viewpoint that is unfamiliar to non-video game players.LBVR has the potential to reduce this gap by (re)connecting both worlds: humans, objects, as well as the surrounding in the real can also be presented in the virtual and/or vice versa.The scientific community is catching up on the topic of LBVR lately; e.g.there has been a first workshop dedicated to discussing the upcoming challenges using HMDs in shared and social spaces.Here topics such as bystander in-and exclusion, privacy and safety concerns as well as augmentation by using projections have been discussed [7].In addition, theories regarding the interaction with large displays and HMD have been introduced: To describe the interaction of immersive VR installations in public places, the audience funnel [26], which has been introduced in relation to public displays, needs to be adapted.While Mai and Khamis [21] proposed to replace "subtle interactions" (which we refer to implicitly interacting) by "get in touch with the hardware" to emphasize that the bystanders need to familiarize themselves with the hardware by inspecting and even touching it, we think [30] that both steps need to be considered in such cases where the HMD is accompanied by large screens.
A person who is going to use an LBVR-installation can take on different roles which we briefly describe next (see Figure 1): It is important to note that not every participant will go through every role.Some roles might be skipped.For instance, a passer-by (role 1) might eventually skip roles 3 to 6 and instantly become a companion (role 7) of the headset user (role 5) or he/she might skip 3, 4 and 6 and only interact within the virtual world.In each role the user has particular needs that have to be supported to allow for an optimal flow through the funnel.An overview is given in Table 1

Attraction
The way that visitors are attracted by and perceive an installation has a major effect on the users' attitude and intention to enter the immersive virtual environment.Therefore, the first goal of any installation is to attract attention and foster comfort.This can be reached by following common knowledge in designing exhibition stands [3].The location of a good stand has to be carefully chosen and its design has to catch someone's eye from across the venue.To promote your LBVRinstallation, the story has to be integrated into the exhibition stand design and always keep in mind whom you want to reach.It is also important to understand the impact of decoration and to make sure that the lighting conditions match the aims of the overall concept.

Narration
The connection with the virtual environment presented on the HMD should start already long before its actual use to prepare for what is happening in the virtual world.For instance, in the waiting lines of theme parks, the general theme is already taken on and presented before the final experience in order to foster engagement within the queue [2,20].This is building up a storyline or characters which can be built on in the virtual world.Constructing narrative coherence already before use can help to support the suspension of disbelief (a state in which the mind forgets that it is being subjected to entertainment with e.g.impossible physics) [15].For instance, if one is already prepared to have superpowers before entering the virtual world via HMD, one is not surprised to be able to fly like superman.

Transition
In situations where the physical space or objects are of particular interests, such as in museums, gyms [18] or other institutions the user might want to stay connected with the physical world by finding part of the surrounding or objects also in the digital space.
The digital environment is re-sampling the physical environments to establish a connection between the physical and virtual content and foster an easier transition between the two worlds; e.g. a vitrine showing ancient objects or the resemblance of a warrior in full dress1 (see Fig. 2).After this connection has been stabilized the warrior can become alive and start interacting with the visitor.Note that this is different from augmented reality (AR) where the virtual world is merged with the real world.It is interesting to note that a virtual environment matching reality requires a more complex creation process compared to a fictional environment and a higher level of accuracy.When the participants are located in the same area in reality, differences are much more noticeable.Almost all the 3D models need to be custom created and positioned while a fictional environment offers more freedom and the possibility to use already created 3D objects.
Archaeology in Baden.The Badisches Landesmuseum (Baden State Museum) aims to improve its exhibition about the region's archaeological heritage via LBVR 2 .In order to establish the connection between the virtual and real environment and thus to enable HMD users to locate and orient themselves in VR, the museum is pursuing the approach of showing a shelf with exhibits from the historical era in a physical space as well as in VR, see Figure 2. The idea behind this approach is that the objects and a warrior are displayed in exactly the same place with exactly the same proportions.The historically appropriate content creates an additional bridge between the exhibition, which is perceived as a passive viewer, and the VR-experience, which is retrieved as an active user.

Immersion
Most of today's LBVR-installations aim to improve the experience within the virtual environment.Therefore, the physical environment is set up to re-sample part of the digital environments to make the VR experience more immersive-e.g.haptics can be introduced by physically setting up walls, objects, etc. (see Figure 3) or velocity and acceleration can be introduced through motion platforms (for instance Hologate Blitz3 ) or roller coasters (for instance VRCoaster4 ).While a lot of efforts, in this regard, are taken to improve the immersive experience, those setups cannot support the overall experience outside the virtual world.

Connection
Some content requires to create a shared space where it becomes possible that some information is exchanged between the headset user and his/her companions.This exchange is important because, according to a study conducted by the German association for cultural management e.V., most museum visitors come in groups to have an "eventful day" and "to undertake something together" [19].This requires a space where part of the experience is identical to both and where physical objects, as well as virtual content, are used to enhance both, the perception of the real as well as the virtual environment.For instance, virtual objects can be altered according to objects in the real world.It becomes even possible, for instance, that a physical present chair that would not make sense in the virtual environment, might be replaced by a proxy in the form of a tree stump which is moving accordingly [5].The experience of isolation while using an HMD in public was demonstrated by Mai et al., who interviewed their respondents in qualitative interviews about their feelings during use [22].They found that different spatial layouts and amounts of passive bystanders had no significant influence on measured presence.This, however, might change if passive bystanders are encouraged to become active participants.
"Share VR" by Gugenheimer et al. allows participating in a joint experience between headset users and their companions through floor projections [8].They found that the engagement into the narrative of both, the headset users as well as their companions, could be increased."ReverseCAVE" [14] is an interesting concept to engage bystanders via wall projections and especially helpful for social media purposes since it enables visitors to take photos of VR content.Other concepts that try to facilitate cooperation between headset users and non-headset users include multi-touch-tables [34].
Super Nubibus.The installation Super Nubibus5 presents a VR ride in a hot air balloon over the town of Karlsruhe in the year 1834.It has been presented on various occasions including the ZKM Karlsruhe [30].Users can experience the ride using a physical replica of a balloon basket and an HMD.By placing the basket precisely in the virtual environment, it was possible to ensure that the virtual and haptic representation matched exactly: a concept proven to be very valuable, see Section 3.3.The basket also allowed the headset user to lean on the railing and hold on to it.The user could start the ride and determine the height of the balloon by pulling a physical rope that would ignite the burner's flame in the virtual environment and light up two powerful PAR spotlights to simulate the burner's heat on the user's head.Furthermore, a wind machine was activated once the ride started.It is interesting to note here that while in real hot air balloon rides there is nearly no air movement, people who have never experienced it, assume there are.Thus, the expectation of most users is fulfilled by the introduction of the wind machine even though it is not physically precise.
A floor projection has been set up around the balloon basket to show the virtual content from a bird's eye perspective (see Figure 4A).For passers-by, this resulted in a panoramic view of a balloon basket that conveyed the impression of being levitated over a landscape.Additionally-to allow for a control group-a TV monitor has been set up in 3 m height (see Figure 4B) that exactly reproduced the HMD ego-perspective view.A detailed description of the setup can be found in [10]. 5www.super-nubibus.de

Extend of Similarity between Both Worlds
LBVR offers to include or resemble parts of the physical environment within the virtual environment.As capturing or modeling realistic, real-world objects for virtual 3D environments is a complex and timeconsuming task, the question arose, what factors are crucial and how detailed the virtual world has to resemble the real world for different aspects (see Figure 5).In particular in those cases where a physical representation can directly be compared.Therefore, we discuss in this section how similar the two worlds have to be according to different aspects.

Similarity of Geometry
The similarity in geometry and texture determines if two objects-one real and one virtual-are perceived as the same object or as two independent objects.Simeone et al. [32] investigated fundamental shape parameters that are necessary for a credible integration of physical objects in VR.They found that not only the (assumed) center of gravity of the object is of particular interest, but especially differences in size seem to have a very significant disturbing effect.The authors specifically name the problem of an undersized virtual representation, in which the test persons already touched the physical object, but in the digital environment the distance between hand and object could still be perceived.The opposite case (oversized virtual representation or undersized physical object) was not perceived as particularly disturbing.
Regarding basic factors of discomfort Siess et al. [29] asked various questions concerning what particular effect or feature of VR actually caused uneasiness among VR users.A majority of the participants stated that a significant factor for their perceived discomfort being asynchronous mapping between the displayed image and the underlying technical implementation.Around 17% deprecated insufficient graphics (i.e.bad textures or imprecise shadow rendering) as a source of discomfort.

Similarity of Light Conditions
Lighting has the potential to give a distinct character to scenes.Especially in environments exposed to natural or changing light sources the difference between real and virtual light conditions can vary.This could have a disturbing effect in particular in those cases where the real surrounding is represented in virtuality.Therefore, the question arises if people can benefit with respect to presence or comfort by resampling natural light conditions such as the sun within the headset.To study this effect we developed a test environment as described in the example Karlsruhe Campus in Section 2.3 and applied different lighting situations.The hypothesis was that people feel more comfortable, experience a higher degree of presence, and can adjust faster to a virtual environment when the light conditions regarding the sun's positions or color temperature match the current daytime and sunlight outside.More details on the test setup can be found in the Appendix.
Similarity of Color Temperature.Color temperature describes light in a spectrum that is commonly described by the analogy of warm (< 5000 K) to cool light (> 5000 K) referencing to the heat radiation of most natural warm-colored light sources.It can have an effect on how images in immersive virtual environments are perceived by different users and user groups (male vs. female) [31].
To understand what effect the current light condition of the surrounding has on LBVR we performed a user study: Subjects in group A were exposed in VR to the light color temperature measured before the test from the physical environment. 6Group B experienced a color offset from the current measure which was chosen randomly from 6 different strength levels-1000 K, 1500 K, or 2000 K in either a warmer or colder direction to the current real sunlight.This was done to avoid possible bias to warmer or colder color temperatures in general.

Similarity of Shadows.
Shadows are ubiquitous phenomena in the real world, supporting people to subconsciously estimate current time, sunlight intensity and in some cases the current season of the year.Therefore, the potential of enhancing the perception of virtual worlds by adding natural shadows needs to be researched.Slater et al. [33] examined dynamic shadows in an immersive virtual environment concerning spatial perception and presence.This was inconclusive to the effect of shadows on depth perception.However, the experiment suggests that for visually dominant subjects, the greater the extent of shadow phenomena in the virtual environment, the greater the sense of presence.
Similar to the color temperature test, participants were either exposed to the current sun position or an offset of 2 or 4 hours into the future or the past.As the time range of well-visible shadows ranged from about 07:00 until 20:40 at the approximate test date in the virtual environments, participants close to these times were not tested with an offset ranging out of these boundaries.Figure 7 shows different sun positions in the virtual test environment.Summary.Altering the color temperature as well as the sun's positions in a virtual environment does not influence people in a meaningful way for almost every case tested.Additionally, when participants were asked for the current time in the virtual environment, a noticeable number of people could not actively differentiate between different shadow positions and connect them to time.Therefore, these conditions do not need to be considered in the resembled VR environment.

Similarity of Visual and Tactile Appearance
Also, when further modalities such as touch are addressed in regards to resemble the physical surrounding, there are several findings that can improve the LBVR experience.If we take a look at Figure 3 one can imagine, that a physical room setup was chosen that will be quite similar to the virtual counterpart in case of arrangement (position and rotation) and size.But what is commonly not taken into consideration is how the visual surface (appearance in VR) is matching with the haptic surface (structure in the physical world).In the given case the visual surface resembles stone or wood and the haptic surface cardboard.To investigate if immersion and believability are affected by a mismatch between these conditions Hepperle and Wölfel conducted a user study, see Appendix for more detail and Figure 8.They found that several real materials are not only a better match for their specific replica but also match well for other virtual materials.
The swiss' material archive lists 18 subordinate material groups [24]. 7Neglecting those that are in most cases not relevant for building virtual environments such as wax, colorants, solvents or minerals etc., one can see that by only using real aluminum, and real acrylic one would be able to depict seven out of the ten relevant materials.Virtual leather, virtual fabric, and virtual wood could not be well represented.Here it might be argued that the former two materials are elastic and the latter has a rough surface structure while the other surfaces are solid and plain.

User Acceptability and Involvement
As LBVR-installations are also social spaces, the willingness to use and actual use of the installation are strongly influenced by social opinion.Therefore, the social dimension has to be treated as an important factor in designing LBVR-installations to foster the willingness of the user to get involved and interact with the installation.Besides relying solely on established  social norms, special care has to be taken, because with novel collaborative technologies also novel social norms come [28].This effect is particularly important in those cases where the user is publicly exposed to the general audience while not knowing what is going on in the environment-a very common setup in any situation where HMDs are used.

Social Acceptability
Social acceptability is the outcome of a collective judgment or collective opinion of a project, behavior or technology.It is so strong, that the use of technology is negatively affected if it is expected not to follow social norms.How the user is believing to be perceived and judged by bystanders is an important fact and may result in discomfort and tension [6,17].
As pointed out by Eghbali et al. [4] "the isolation of the user from the others can create a form of social gap that may affect the social acceptability in public context."We identified three social gaps8 due to the use of HMDs: • spectator blindness: When only seeing a person wearing an HMD and interacting with the environment without a "window into the virtual world", such as a monitor or projection, bystanders cannot understand what is happening.
• leap into the unknown: The potential user of the HMD does not know what to expect from the upcoming experience and might let him/her not put on the headset.
• environment blindness: Using an HMD causes the loss of knowledge of the physical space which can cause being afraid about how to move safely through the environments and being watched by others.
While spectator blindness and leap into the unknown might sound very similar at first, we want to point out the difference between the two.While spectator blindness is addressing those persons who have no intention to use the HMD and are only interested in watching others, leap into the unknown is only affecting those users to get prepared for their own VR experience.However, both have in common that they are caused by not communicating what is happening in the virtual world to its physical surroundings.This drawback can be overcome-as already discussed in Section 2.5-by the introduction of screens and "non-digital" staging.
Environment blindness for the user is caused by the lack to communicate from the "outer world" into the virtual world.While the physical environment is usually stable and changes are not immediately important (e.g.cars driving by in the distance) while being enclosed in the virtual world, humans change positions and emotions rapidly and might be of immediate importance to the user (e.g.Is my kid still waiting for me?Are others laughing about me acting strangely?).
To the best of our knowledge, there are no solutions available that are able to reduce the problem of environment blindness without drastically influencing the HMD users' immersion and behavior.One idea is to represent bystanders as avatars inside the virtual world, however, this could have severe influences as has been observed by Kinatender and Warren by examining the influence of bystanders on evacuation behavior in real and virtual environments [16].Another idea is to look outside the virtual environment using a "flashlight" to blend the real into the virtual environment [9].While this approach is promising, the authors named several limitations that have to be overcome before it can be presented to end-users including performance issues and visual fidelity.

User Involvement
Placing the right prompts and incentives to engage visitors to get involved and to interact with the LBVR-installations is a key element to improve their success [27].Hepperle et al. [10] investigated key factors to increase user involvement by observing the Super Nubibus installation, as described in Section 2.5, and by asking users to fill out a questionnaire.The results are presented in Figure 9. Watching others using an LBVR-installation is the most influential method to tear down the barriers to enter-the honeypot effect seems to be effective also for staged VR installations.The second  strongest influence found was the floor projection which performed significantly better than the use of a monitor mirroring the image of the VR headset from a first-person perspective.More than twice as many bystanders were motivated to participate by the floor projection compared to the monitor.A nearly equal influence was attributed to others who suggested the installation as well as persons who actively pulled in passers-by and spectators.
Interest in the LBVR-installation was more attributed to the floor projection (µ: 5.43, σ : 1.43) 9 than the monitor (µ: 4.47, σ : 1.78).HMD users interacting with the installation were equally rated to raise interest than the floor projection (µ: 5.30, σ : 1.52).Comparing correlations between the parameters revealed that users who were attracted by the floor projection were also more likely to recommend the installation to others and to share the opinion that the floor projection established a connection between VR and the real world.

Design Recommendations for Staging Location-Based Virtual Reality
Based on the previous discussions and findings we can summarize the following design recommendations particular for LBVR as an extension to general recommendations for public screening (e.g.avoid unsuitable content such as nudity and violence, protection of data privacy) [1,26]: • stage the environment and queue line and thereby create valuable interactions for passersby, waiting visitors, and spectators with the VRinstallation • define a clear separation between the different interaction zones to increase felt safety and make sure they cannot be broken easily 9 on a Likert scale from 1 (strongly disagree) to 7 (strongly agree) • prepare users beforehand what kind of experience they have to expect-nobody likes the leap into the unknown • make sure bystanders can understand what is going on and why the person wearing the headset is acting like that-sharing the ego perspective of the HMD user is common today but third-person views are more suitable • foster bystanders to take on a particular role, a system operator or friends, next to the wearer of the display can improve the feeling of safety and guidance • allow communication between the HMD user and bystanders or the environment without the need to remove the display • match the location between the objects which can be viewed and touched and make sure they are not drifting over time • match the visual perception of a surface with the haptic perception to a certain degree (some materials can be a good substitution for others) • letting users quickly blend realities can help to overcome environment blindness but might negatively affect immersion and behavior While some aspects show a great impact in LBVR other aspects are not important and might need less consideration in setting up an LBVR-installation: • realism is overrated and consistency in location is more important as high-poly models • exact congruence of the physical lighting conditions with the rendering of the virtual representation seems to be not helpful

Conclusion, Limitations and Future Work
We have investigated different location-based specific aspects to improve the experience of LBVR.While some aspects are very specific for one particular installation such as motion platforms (e.g.underwater, hot-air balloon rides, bike cycling) we have focused on aspects quite common to many LBVR-installations.Staging has the potential to not only convince passers-by to participate but also has a direct influence on the actual HMD user's well-being (e.g.not feeling isolated from the physical surroundings while maintaining a sense of presence).By articulating the content to the public, a connection is created between the otherwise isolated headset user and the audience.This reduces the fear of putting on an HMD and-at the same timeimproves the immersive VR experience itself.This is because the headset user is no longer overwhelmed by the unexpected content, but can mentally prepare for the upcoming environment and content already before entering the experience.
As the possibilities, as well as various effects, are nearly endless we were able to focus on some common aspects which are shared among different LBVRinstallations.Specific topics have not been addressed and need further investigations.For instance, it has been observed that motion sickness can be reduced by merging the virtual and physical world accordinglynot only in those cases of motion platforms.
LBVR has many possibilities to improve VR experiences.By now many LBVR-installations do not consider many points discussed here and thus are not using the full potential to offer "magic moments" through LBVR.This might be due to cost and effort issues, but we believe that there is also a lack of understanding of how to use the potential lying in this.Therefore, we have also included design recommendations into this publication to support a good overview of what has to be taken on to improve LBVR-installations.

Figure 1 .
Figure 1.Audience funnel for location-based virtual reality.

Figure 2 .
Figure 2. The warrior in the vitrine is present in the real and the virtual environment.Courtesy of Oliver Langewitz and ArcTron.

Figure 3 .
Figure 3.The physical environment is setup to support the virtual content.

Figure 4 .Figure 5 .
Figure 4. A: the setup as seen from behind; B: view at the booth from the rest area

Figure 6 .
Figure 6.A participant during the test of color temperature and shadow.

Figure 7 .
Figure 7. Image of a campus building in different daytimes.

Figure 8 .
Figure 8. Collage of a real material (left: acrylic) and its virtual counterpart (right: aluminum).

Figure 9 .
Figure 9. Questionnaire answers on "I used the installation because ..." (more than one answer possible).

9
EAI Endorsed Transactions on Creative Technologies 02 2020 -02 2020 | Volume 6 | Issue 21 | e5 Spectator: a persons who observes the installation and the headset user, he/she can communicate with other spectators about the installation but do not yet interact with users or the installation 1. Passer-By: a person who is uninvolved, not aware, or not paying attention to the installation 2.

Table 1 .
Description of what kind of support has to be offered at what kind of user role.