Indoor Navigation using Augmented Reality

This paper introduces the idea of Indoor Navigation using Augmented Reality. With the increase in complex building structures, people of di ﬀ erent age may ﬁnd it di ﬃ cult to navigate within such structures. This paper provides a solution for this trouble using available and accessible resources. Using this application, a shop within a shopping mall, a speciﬁc room in a hotel etc.,can be easily located and the user is provided with reasonably accurate visual assistance through their smartphone to reach his preferred location. The proposed system is inspired by augmented reality and 2-D Visual markers are the fundamental elements used in this system. Multiple visual markers are placed within the complex building structure and on scanning these visual markers using the application, users are led to their destination using arrows displayed on the screen. The application allows the user to choose their preferred destination and also permits them to change their destination in between.


Introduction
In the steady rise of technology, Augmented Reality has turned out to be one among the most exciting and promising technologies the world has ever seen so far. Even with all these evolving technologies, the problem of Indoor localization and navigation still remains without an adequate solution. The fact that GPS does not give an idea about the current altitude, lies as a major obstacle to this problem. Multiple products have entered the market but the most popular ones are that which use sensors and other hardware of smartphones to get the surrounding information and uses it to localize users. Sensors such as the accelerometer, magnetometer and gyroscope are mostly commonly embedded into the smartphones to obtain a quicker estimation of the user's location. But currently, thanks to the development of smartphones having cameras with high performance and computational abilities, we can use Computer Vision systems that rely on marker-based algorithms to estimate the position of the user. By accomplishing this, we can also help the user to move forward, thus assisting him visually using Augmented Reality to

Related Work
Rhuta Joshi [1] conducted a survey on indoor positioning and navigation techniques using augmented reality. Several existing methods are reviewed to create a navigation system that can be implemented in handheld devices such as smartphones. A review on the latest tools that can enhance the implementation is of Augmented reality is also done. In this system, there are two phases: the owner setup and visitor navigation. The owner (can be the admin) provides a floor map and destinations of the indoor structure to the navigation system. Each destination can be identified as a node in a graph that is used for location identification with the corresponding coordinates. For the visiting user to use the application, he/she just has to scan a nearby destination point and can acquire his/her current location by using landmark detection algorithms. The user can then select from a list displaying the destinations within the particular building. On selecting the destination, the shortest path is calculated and the user is then directed. Figure 1. Implementing augmented indoor navigation system using direction guides.
Rudiger Pryss [2] explains the use of the Augmented Reality Engine Application (AREA) project in which a kernel that enables location-based mobile augmented applications is developed . In projects developed using this kernel, a feature enabling tracks was required since it can help in assisting users in activities such as mountaineering. By conducting experiments, AREA gives better results than commercial location-based mobile augmented applications like AR GPS Navigator. Except for the static scene of Android, AREA performs well in all other scenarios.
Buti Al Delail [3] evaluates a indoor navigation and localization system that combines inertial tracking and augmented reality. This system makes the user aware about his/her location and makes it easier to find other places with reasonable precision. An image recognition system is used such that it can be run in any campus, given a database exists for recognizing the placed markers and also for determining the location and displaying the required information in Augmented Reality. It also shows that inertial navigation provides a satisfactory approach to keep in track of the user's position while indoors. The system described can be deployed in any hardware devices with the required sensors and resources available in the modern day's smartphones.
Gaetano C. La Delfa [4] states the use of markerbased and markerless computer vision approaches. A qualitative performance evaluations is done on three 2-D visual markers which are suitable for real time applications. The selected markers are Vuforia marker, AprilTag and ArUco marker. These markers are tested in different light conditons and floor patterns while taking into account the three important parametersthe size, the detecting speed and the distance from the camera of the phone. The analysis led to a conclusion that the Vuforia marker has better performance when compared to the others if the size of the marker is greater than or equal to 5x5 cmexp2 . AprilTag and ArUco performs well in any marker size and any tested light conditions and floor patterns in real-time environment. The aim is to minimize the marker size and to enhance the speed while giving satisfactory user experience. Jaewoo Chung [5] explains how pedestrians may lose focus while paying attention to their device and not on their actual path, which often can cause them harm. A mindful navigation system is proposed which can reduce errors and improve the user experience. An augmented reality system is used to test the hypothesis with the directions directly projected on the user's path while he/she moves forward. It also states that both the performance and overall user experience will improve once the user feels like he/she is in control of their path. This system brings forward improved user experience, decreased travel time, confusion, errors and also helps in landmarks being more noticed than before.

Existing Systems
In recent times, indoor local references such as Wifi/Bluetooth based localization or Inertial Measurement unit are used for local navigation purposes. Inertial Measurement Unit consists of gyroscope, compass recording and accelerometer. With all these sensors, the expected accuracy is not attained during navigation. Currently existing AR devices like Google Glass and Google Cardboard are supported heavily by hardware that some smartphones lack. To overcome this shortcoming, client-server architecture is used for intensive processing.

Augmented Reality
Augmented reality (AR) is a technology which extends our physical world and adds layers of digital knowledge within. AR appears in a real world environment, adding sounds, videos, and graphics to it. AR is a view of the actual real-world environment, with computergenerated images superimposed, thereby altering the understanding of reality. In other words, both virtual world and real world in combined to give an enhanced experience to the user. Using AR, 3D virtual models are directly projected onto real-time objects, leadin to various virtual reality technologies influencing our social life and entertainment. Typically, AR apps connects digital animation to a special marker or identify the location with the help of GPS in smartphones. Augmentation happens in real time, for example, overlaying scores to live feed sport events within the context of the environment or in weather channel studio where weather reports can be displayed using images next to the news reporter. In our proposed system, augmented reality is used to display arrows marks or directions which appears to be on the floor and can be viewed by the user through his phone. For this, we make use of the following software development kits : • Unity

• Vuforia
Unity. Unity is a platform that offers the requisite collection of features for game developers to create games faster and efficiently. Unity is a game development system that supports many key areas and puts them together. We can bring in 2D, 3D, art and assets, assemble these materials into scenes and environments, add audio, lighting and other special effects, physics and animation, interactivity, and game play logic, and edit, monitor, and optimize content for user target platforms. Unity 3D game engine is a games magic warehouse, and pioneers of augmented reality. Exploiting the power of this cross-platform tool will create practical and immersive solutions to take gaming to a new level. Unity enables the user to create both 2D & 3D games. The engine provides a primary C# scripting API, for both plug-in Unity editor and games themselves, as well as drag and drop functions. In the proposed system, unity is used to create the application as it is a good platform to create both AR and VR platforms.
Vuforia. Vuforia SDK can be used as a kit that creates augmented reality applications for mobile devices. It uses computer vision technology to acknowledge and track 3D objects and planar images in real-time. The image recording functionality permits developers to place and align virtual objects, like 3D models and other graphics, when viewed via a smartphone camera, in regard to world objects. The virtual object then tracks the image's position and orientation in real time, in order that the viewers view of the thing corresponds to the target's perspective. Therefore the virtual model appears to be part of the real-world environment. The Vuforia SDK supports various 2D and 3D target forms including 3D Model Target, markerless Image Targets etc. The recent version-Vuforia Unity 2019.2 of the vuforia engine enable it to collaboratively work with Unity on a single platform to bring enhanced experienced to the both the developer and user. Both head-worn and hand-held devices are included in this. Additionally, it can also detect special markers called Frame marker, which we have used in our indoor localization project. 512 such markers exist as an archive. Each one of these markers has a unique ID in the binary pattern and needs an area free of graphical objects around it with a good contrast to the black frame.

Methodology
The proposed system is made using position markers. These custom markers can be scanned from any angle using the camera of the user's smartphone. Several such markers are placed either on the floor or stuck on a wall and spread across the site. They would be positioned particularly at all entrances and all other points of interests so that the application can easily identify them. Upon scanning, the user is directed to his/her destination using the visual displays shown on their phone. The application provides reasonably accurate visual assistance to the user by focusing on the features of the system and on how to exploit these features to increase the performance : -the fixed size of marker within the frame, which depends on the distance from the camera to the floor and also makes it faster to find the marker, resulting in improvement in the detecting speed.
-Somewhat uniform background pattern of the floor, which can be used to improve the decoding speed of the algorithm.
-Chances for the marker to be at the upper part of the frame, since the user moves forward while navigating.
-Marker postions should be known beforehand so that the currently decoded marker should be one 3 EAI Endorsed Transactions on Creative Technologies 12 2020 -03 2021 | Volume 8 | Issue 26 | e1 within the previously decoded marker's boundary.
The following characteristics of the chosen markers must be considered for a better performance: 1. Small size -the most suitable compromise between size, detection speed and robustness must be found to enhance the algorithm.
2. Unvarying performance in changing light conditions -On changing the light conditions that may occur to switching on/off the lights, shadows etc, the application shouldn't hinder in performance 3. Robustness to blurred images-which may occur due to the fast movements of the user.

Figure 2. Flowchart of user
The above flowchart shows the navigation done by the user. The application first ensures if the user is registered or not. If yes, then he/she is required to login with the necessary details. If not, he/she will have to sign-up to use the application. On logging in, the user is required to select the destination within that building/floor. After entering the destination, visual assistance in the form of arrows are shown to the user, leading them to their destination. If the user has to change their path in between, then the user can enter the new destination and the user will then again be assisted to their new destination.

Backend Design
Firebase is used as back-end service. It is basically an application development platform. The back end service provides an API that synchronize and store the application data on Firebase's cloud.
Firebase provides backend authentication service for the application. The ready-made UI libraries and easy-to-use SDKs are used for authenticating users to the application. Authentication is supported using passwords, emails, phone numbers etc. The application's user identities are securely saved in the cloud and provide the same personalized experience equally to all the users. Thus a user have the options to avail sign-up and log-in to his account. In the proposed system, firebase is used to store user registration and login details. These details are by default stored in tables for easier access. Another added advantage is that the admin can keep an eye on the user activities through the firebase dashboard.

Figure 3. App
We developed an unity application that helps in indoor localization and assists users that need help navigating within the indoor structure. Navigation assistance is provided using arrows displayed on the smartphones. This application has a very simple and user friendly UI. Inorder to use the application, the users are required to sign in. Firebase authentication services allow registered users to sign in and non-registered user to sign up to this application. All user data is stored in the firebase inside tables for easier access. 4 EAI Endorsed Transactions on Creative Technologies 12 2020 -03 2021 | Volume 8 | Issue 26 | e1 The sign-in button allows the user to use the app after successfully logging in. Existing users can login using the required details and non-existing users can sign up by filling in the required details.  On signing into the application, the available destinations within the building structure are shown to the user and the user can choose his/her preferred destination. On selecting the destination, the first marker is scanned using the camera and then visual assistance is provided using arrows to direct the user.
The arrows appear one by one as the user moves and directs him to his destination. If the user desires to changes his destination in between, he/she can do so by clicking on the back button and the list of destinations will once again be displayed to the

Conclusion
The proposed system can be designed and developed using the above discussed technologies. The priority of this application is to assist users in unfamiliar surroundings within a complex indoor structure. Users can select their desired location within the indoor structure and the application then displays directions that can be viewed through the user's smartphone using augmented reality. This constantly keeps the users interactive with the application and leads them to their preferred destination. The application is made in such a way that users of all age can use it with ease. In the nearby future, we plan to further upgrade the system by making navigation available to users worldwide. Existing users can rate the paths taken within an indoor structure so that others can choose the best rated pathway to their destination. Audio assistance can also be made available to users who prefer it.