Innovative services for informing visually impaired persons in indoor environments

Movement and informing visually impaired persons is difficult because mobile application solutions and services are unable to work in indoors environments (location based service using GPS technology). Main precondition for active participation in the daily living and needs of visually impaired persons is increased degree of mobility. The aim of this research is to increase the quality of life of visually impaired persons and the degree of mobility in indoor environments by applying modern information and communication technologies (Cloud Computing, Fog, IoT, AAL / ELE platform). User requirement will be used to define functionalities of service for informing visually impaired persons in the example of large retail chains. A conceptual system architecture model will be proposed for providing information service with the aim to provide real-time information to users. This research will also show simulation testing of the proposed architecture with Arduino Uno and Raspberry Pi 3 platforms for collecting information and informing end users.


Introduction
Previous research in the area of indoor navigation and information of the visually impaired users is based solely on the possibilities of using ICT technology such as AIDC (Automatic Identification and Data Capture). For the purpose of informing users, various possibilities of assistive technologies are often used, in which the most important are mobile devices and devices adapted to provide various sound information. Most of the previous research does not exclude the use of the white cane as basic tool for informing the user, but rather it is being upgraded to various forms of assistive technology, such as the application of various sensor devices [1], [2]. Using VLC (Visible Light Communication) technology, it is also possible to transmit data, so some research is focused on exploiting LED technology for indoor user informing [3], [4]. With the development of technology and the ever-increasing of new smart solutions application, a concept of new ICT technologies application in indoor environments, such as AAL (Ambient Assisted Living), is created. The basic prerequisite for the development of new services based on the AAL concept is the use and use of sensors in the IoT (Internet of Things) environment. Research in the field of IoT concept application in indoors are different, from the possibility of providing user assistance to informing systems in the Smart Home environment [5], [6]. In indoor environments, it is also possible to inform users by using a mobile camera and integrating it into the WSN (wireless sensor network) [7], [8], [9]. It is also possible to inform users about the environment and objects surrounding it by using various sensors [10]. The lack of all mentioned researches is clearly defining user requirements that are the basis for the development of new services in the area of providing real-time information for the indoor moving users. In this paper, based on collected information on user needs, the user requirements are defined EAI Endorsed Transactions on Internet of Things for the delivery of indoor user informing services (e.g. shopping centers, food and drinks markets and other forms of in-house shopping). This paper is a follow-up of the research on sensor technology (IoT) use for users informing [11].

Defining user requirements
For the purpose of defining functionalities and guidelines for the development of user informing services, it is necessary to define their requirements for indoor informing. Content and features of informing services have to meet all elements of the UD (universal design) in addition to user requirements.

User requirements research
For the purpose of defining user requirements, a research was conducted among people with damaged vision in which group belongs the blind and visually impaired person. In the city of Zagreb (Republic of Croatia), according to currently available data, 1,971 persons with vision impairment belong to persons with severe vision damages (both eyes 411) and those with a certain degree of visual impairments [12]. The research was conducted in collaboration with the Up2Date association (which has 60 active members and 30 volunteers) on a sample of 81 users, of which blind persons were 37.65%, visually impaired 21.18% and other disability rates 41.7%. When moving independently (without a guide or assistant) in indoor spaces users are faced with many challenges, such as locating the requested object, navigating indoors, finding an exit in an emergency and getting information of user environment. Currently, the most common way of informing users indoors is the tactile line of guidance (bank, public and other institutions). Shopping centers and marketplaces in the city of Zagreb do not have tactical lines of guidance and thus the users are extremely dissatisfied (63%) because it reduces their degree of mobility. Information about indoor facilities (shops, information desks, toilet, etc.) that surround the users and notification about their presence is considered important to 86% of users. Information on possible obstacles (work on movement routes, benches, decorative elements, trash bins, etc.) that can be found in motion is considered very important and important to 78% of users. A user who moves independently has difficulties in identifying objects and obstacles, thereby further endanger the safety of movement. Intendent movement is associated with determining the user location, and 83% of users consider informing about their location important and very important. In the indoor space, personal belongings can often be lost where the user with visual impairment can have difficulties to notice the loss of a particular item (money, wallet, other personal items), and 51% of users would like to be informed about such a situation. Everyday activities, in which group belongs users' indoor movement, requires additional physical effort for the user if he is in the new and previously not described space. New services whose basic functionality would be to inform the user about the environment and the possible notification on the following activities would raise the degree of user mobility. Users are interested in introducing new services (87%) designed to meet user requirements and UD elements. According to the obtained data, 93% of users use a smartphone for daily communication. For the purposes of introducing new informing services, users have the desire to receive the requested information through the devices seen in Figure 1 (32% is mobile application installed on user's mobile device). A number of users chose more options, where is also seen the presence of mobile device and associated applications.

Figure 1. Possible ways of informing users
Today's smart devices have the ability of using NFC technology for the purpose of contactless payments. This technology is also present in card business for faster and more efficient purchasing. Presented functionality would be used by 81% of users.

Defining service functionality and user requirements
The design of the system required for providing indoor user informing service can be based on the AAL / ELE (Ambient Assisted Living / Enhanced Living Environment) platform. After identification of the AAL / ELE-based facility, basic requirements have been defined [13]: • Open-the system should allow enhancements to the new and already existing technologies. It should allow internetworking of the heterogeneous segments on a single platform.
• Virtualized -the system should be virtual at different levels in order to allow better administration and service development. • Generic -the system needs to be as generic as possible. This means that the services should not be specific but more natural for the end-users.
• Scalable -the system should be scalable and should be capable to be installed in very small scale and be expandable to the large scale at later stage. • Module-based -the system should be proposed in modules that are interoperable and could map, merge, split, interact using standard interfaces and protocols. • Customizable -the system should be as customizable as possible. Based on the generic nature of the initial set of functions the end-users and stakeholders should be capable to change configuration in order to serve their personal preferences, peculiarities, local customs, religion specific requirements etc. • Granular -the system should work at different levels of granularity allowing the end-user to change it. • Security and privacy should be included in all layers and components of the system.
Based on the current research in the IoT technology application field [14] and collected data on user needs in this research, the following customer requirements are defined: • The possibility of using service on the smart device, • Obtaining all relevant information about the environment, • Real-time informing, • Using TTS (Text to Speech) option.
Based on user and basic requirements, guidelines for development and functionalities of indoor user informing service are defined. The development guidelines can be divided into two categories: • Content customization must be enabled for all levels of visual impairment such as increased content fonts allowing the ability to work with TTS options and content in multiple contrast colors.
• Ease of use according to UD principles needs to meet the following areas: • Ensure equal means of use for all users, • Usage flexibility, • Mode of use choosing (left or right handed), • Simple and intuitive use, • Perceptual information and • Low level of physical effort.
The functionality of user informing service in indoor environment should have the following capabilities: • Informing users about possible barriers -is defined as the real-time information about all the relevant obstacles (e.g. benches, garbage bins, storage boxes, ornamental plants and the other) on the user's route. • Notification on nearby objects -defines all the spaces for which the user has expressed interest. When using this functionality, it is necessary to allow enough time for user to decide on further movement direction. • A short description of each facility -according to UD recommendations it is required to provide the user a simple description of each facility and its activity. • Item description -an item is considered any subject of business in a facility. By informing the user on an item it is possible to create an imaginary image of what user is purchasing or has a desire to purchase. • Simple registration of lost things -functionality that can raise a feel of person's safety when moving. • Notifications about possible discounts -the information that is needs to be provided to a user when entering the facility. This functionality is only active in the case of a user's predefined desire. • Information on accidents -information that enable safe routing to emergency exits.
The functionality of user informing service will be based on the proposed system architecture and technologies such as Beacon BLE, Fog, and CC (Cloud Computing), which will ensure the delivery reliability of all the requested information to the end user.

Analysis of information delivery technologies
Delivering data and information from the user's environment that is moving through indoor environment can be based on AIDC technologies, sensors, VLC technology, and Beacon transmitters. Table 1 shows the advantages and disadvantages of an individual technology for the purpose of delivering data and information from the environment of the user moving indoors. Depending on the real conditions in which the Beacon transmitters are placed indoors, the signal may change its value, which can cause problems when determining the exact location. The capabilities Beacon transmitters have in the indoor environment are: • Launching the application on MD when Beacon signal is identified, • Send feedback in the form of a push message, • User guidance and routing using the Beacon transmitter map, • Ability to work on iOS devices, with iBeacon communication protocol and • Eco-mode (power and message sending interval reduction) can extend battery autonomy.

Conceptual System Architecture Proposal
For the purpose of delivering the information to the user in the indoor environment, this paper proposes system architecture based on the Fog / CC platform applicable in the shopping centers, shown in Figure 2. The aforementioned platform enables the system operation in the IoT environment and according to the AAL / ELE concepts with the aim of delivering real-time information to the end user. It also enables interaction and participation of all stakeholders important for creating information.
The conceptual system architecture is based on the information and communication solution SAforA, where Beacon transmitters also use the data collected from the sensors [11]. The mentioned Fog / CC platform is applied indoors where the Fog node is based on a private or public CC service using client -server and peer-to-peer architecture. The user's terminal device has the ability to connect to the Fog node via the Radio Access Network (RAN) network or WiFi network. The interaction between all levels of Fog / CC depends on the service-level agreement (SLA) that needs to be sufficient for providing an adequate service level [13]. An example of user identification and informing service delivery using conceptual system architecture in indoor environment is shown in Figure 3.

Figure 3. Process of objects identification and informing users
When using a Beacon transmitter, the user must activate the BLE signal on his MD. Upon identifying the user's MD, the Beacon transmitter (Bn) triggers an application installed on the user's MD, after which a check is performed with the Fog node. Fog node will return a defined database record (according to a specific ID from the Beacon transmitter) to the mobile application, based on which information is created for the end user. Using Fog technology enables faster delivery of real-time information (measured in milliseconds) that do not require further processing, analysis and forwarding. With Fog technology, higher reliability and lower latency are expected. Fog nodes also have elements such as database, user base and beacon transmitter base from CC architecture seen in Figure 2. Fog to CC Communication is executed at precisely defined time intervals and can be expressed in days, weeks, months, depending on system requirements. In order to allow additional information creation and network access for all stakeholder groups (Associations, public institution, crisis management systems, navigation maps service providers and others), it is envisaged to establish the system according to the PaaS (Platform as a Service) model. When entering each data is adapted to M2H (Machine-to-Human) communication in a form acceptable depending on the user's degree of disability. M2H communication also enables the adaptation of the information according to the Braille line [15]. All data required for Beacon transmitters (Beacon transmitters base) is stored in the database. Based on stored dana all the information needed to inform the end user are created. The user database, which contains user account information and their settings, is located in the CC architecture section that belongs to the service provider. The user database provides the information necessary for verification and customization of the content according to the user's priority. Content customization is based on user characteristics and functionality choose by individual users. The user accesses their data through a mobile application on a mobile device or via a web application using an Internet browser. The Web application is used exclusively to define the user profile and to view the availability of the Beacon transmitter. Comparing Beacon and GPS technology, GPS has the ability to route the user to a specific facility where further routing and information can be made by using a Beacon transmitter. Using NFC technology, the user must be close to the tag itself and activate the NFC reader on his MD if it is integrated (not all devices have NFC readers). In indoor environment, it is recommended to use beacon transmitters on the WiFi network. The reason for this is a check that takes place with the Fog node where it is possible to achieve faster data transmission compared to the 3G / 4G network usage.

Proposed solution testing
Arduino Uno Development Environment, Raspberry Pi3, and Computer Server (CC) were used to test the proposed system architecture. Testing and monitoring was carried out in the Laboratory of development and research of information and communication assistive technology at the Faculty of transport and traffic sciences. Operation monitoring equipment used in this research is following (Figure 4): • Arduino UNO,  Three beacon transmitters were used in the testing, one was used for initialization while others were used as location beacon transmitters. Each Beacon transmitter has its own MAC address, the Arduino Uno component reads the specified address after which the BeaconID is send to the Fog node. The Fog node searches the BeaconID in its database that consists only of the data of a particular store, with the purpose of faster processing of CC requests. This database contains information related to the local area in which the user is located. When the Fog node receives the BeaconID that is not in its database, it forward the query to the CC. There are large amounts of data in the CC that require extra processing to extend the functionality of location-based services whereby the BeaconID is related to that data (for example, looking for an item in a specific facility). The CC will, after receiving such information, perform additional processing and, for example, extract the expiration date of the item from the database and calculate the duration, which will be forwarded back to the application on the user's MD. In addition, the CC in its database contains additional information about articles related to BeaconID. The CC can also include a location in the facility, e.g. where a particular item can be found and forward the user instructions to that part of the facility. Case study in testing the proposed architecture in IoT environment is a supermarket as a facility that offers local discounts. This information is related to one of the Beacon transmitters in the area called the initialization Beacon transmitter. The purchasing application on the user's MD will apply this discount in pricing and inform the user when receiving the discount information. Such information as well as basic information about the location on which the item or user is located is obtained from the Fog node. The information flow in the carried-out simulation is shown in Figure 5.   Figure 6 shows the Fog node simulated on the RPi device. When receiving an ID from the initialization Beacon, Fog node will check for specific services available only at that location, such as a discount on all products (marked in red frame).  If there are no additional services, the Fog node will only forward basic location information. When receiving an ID that is not in its database, Fog node sends a query to CC (connection marked in blue frame) that performs additional processing of its database records. After processing the data CC transmits the expanded information to Fog node which then forwards it to the end user (marked in green frame). The Figure 6 shows information about the item mass, volume and the expiration date. In this way, the user will get additional information that is also related to his current location and his environment.

Conclusion
Increasing the mobility of the user is the goal of the proposed service for indoor user informing. With using Beacon technology in the Internet of Things concept and the Fog / Cloud Computing environment, it is able to deliver real-time information in a reliable and secure way to the end user. The application of Beacon technology in indoor environment represents modern upgrades of today's RTLS (Real-Time Locating Systems). Using the Fog concept allows faster response of the system itself for the purpose of delivering information, while CC architecture performs data collection, processing and analysis. Fog will deliver only data that do not require further analysis and processing as demonstrated by testing of the proposed system architecture operation. By designing services according to guidelines and defined functionalities, it is possible for a user to have a simpler functioning in performing their daily activities without using a special assistant. By applying new information and communication technologies, it is possible to include all relevant stakeholders in creating information for the end-user information. Such a form of information-communication technology is also a model of assistive technology in indoor environments.
The delivery of proposed service in the Internet of Things concept has the ability to use other sensor technologies proposed by the SAforA solution, whereby the expected information delivery is 24/7.