Interacting with and via mobile devices and mobile robots in an assisted living setting

Using robotic home assistants as a platform for remote health monitoring offers several advantages, but also presents considerable challenges related to both the technical immaturity of home robotics and to user acceptance issues. In this paper we explore tablets and similar mobile devices as the medium of communication between robots and their users, presenting relevant current and planned research in humanrobot interaction that can help the telehealth community circumvent technical shortcomings, improve user acceptance, and maximize the quality of the data collected by robotic home assistants. Received on 30 October 2014; accepted on 23 March 2015; published on 18 May 2015


Introduction
Ageing population and chronic diseases pose a number of challenges in terms of health care.The cost of maintaining high quality health care services with high staffing ratios is not viable given the economic struggles institutional health care faces.The challenge presented is to build environments that would: minimize shorttime hospital admissions, prolong independent living at home, and relieve part of the burden of the carers.Advancements in ICT offer solutions to the aforementioned challenges: a smart home environment allows monitoring Activities of Daily Life (ADL), usually, via static sensors.Monitoring in such environment offers the opportunity of timely symptom/incident detection and promotes independent living.However, static sensors can only cover predetermined ranges and can also create a "big brother" feeling to the people that live in the smart home.Using robotic home assistants as a platform for remote health monitoring is a growing research area in healthcare at home.Robots' mobility offers the advantage of being able to collect data from angles and ranges that cannot be easily obtained from alternative setups and, furthermore, robots can be used more pro-actively ensuring that data is collected regularly.At the same time, a robotic assistant collects data in a less obtrusive way.The sensors are localized on the platform and so the user can tell whether he/she is in the range of the sensors.As a result the feeling of being constantly watched is alleviated [1].
However, no matter how strong a case might be for deploying robotic home assistants, user acceptance and feasibility within the robotics state of the art are also critical for success.Since health monitoring mostly targets elderly users, advanced Human-Robot Interaction (HRI) strategies and methods should make service requests easy and natural, create opportunities to unobtrusively collect health monitoring data, and avoid making unrealistic demands on the robot's cognitive capabilities.
In the rest of the paper, we first discuss the challenges and opportunities that come along with the development of robotic smart home and especially under the prism of an assisted living environment (Section 2).We then present the literature on the factors that influence the success of HRI (Section 3).In Section 4, we describe a simple demonstration that helps to better understand the demands emerging from a robotic smart home environment in terms of the communication between humans, a robot, and fixed automation.We conclude by presenting a research plan for developing the necessary technologies that will allow home healthcare community to successfully grasp the opportunity offered by the rapid advancement of mobile devices and home robotics (Section 5).

Challenges and Opportunities
Having a complex system such as a robotic smart home imposes a wide variety of challenges in different levels of the system and in different research and scientific areas.Researchers from different disciplines need to communicate and make contribution in order to achieve a mature and ready-to-be-used system.
To tackle the challenges underlying the implementation of such system we attempted to break down the conceptual design in a number of steps and questions.We start by asking what are the functionalities of the robotic smart home and go on by defining who executes them.How we satisfy the functionalities, answers the question of who does what.Finally, further details of the system defining when decisions are made and where sensing and actuating occur must be decided.Figure 1 presents a schematic diagram of the Conceptual Design of a robotic smart home.
The functional goal of the system imposes the specific actuating and sensing functionalities accommodated.In the setting of assisted living the functionalities set are: • House works and service tasks completion

• Health data monitoring
The mobile actors and static components that will execute the functionalities must be identified next.Examples of actors could be the humans that are assisted and assist, a robot and an application on a mobile device.The components are any part of the fixed home automation that offer sensing, processing and actuating services.A question related to this step is how the actors and components communicate with each other.
At a next step we must define who does what.Each actor and component is characterized by a degree of autonomy.The autonomy not only mirrors what an actor can do but also what an actor wants to do.For example, a healthy human might demand from a robot to execute tasks not because the user is not able to carry them out, but because he/she wants to have them done; by contrast, another user might rely entirely on the robot for the execution of the same task.Given the autonomy of each actor a hierarchy is defined based on which each actor takes over the tasks to execute.Each actor should take over tasks in accordance with their capabilities and in a way that does not leave gaps in Finally, questions of when and where are used to define further decision making and sensing/actuating details of the system.
In this paper we use a simple experiment (see Section 4) to explore how we can enhance acceptance of the robot by users while acquiring close-up footage.This question is relevant to the communication and mutual acceptance of the actors (Human -Robot) (Figure 1 -box "Who?").Moreover, we aimed to include in the demonstration how home automation could communicate with the mobile platform.

Ethical Considerations
When talking about advantages and disadvantages of assistive robots and monitoring activities of daily life, it is necessary to take a look at ethical concerns and human values.Mittelstadt et al. [2] describe ethical issues raised by Personal Health Monitoring, which uses electronic devices to record data within homes.Their findings suggest several different ethical issues concerning: • privacy Similarly, Sharkey and Sharkey [3] identify probable risks and possible advantages of different robot uses caring for elderly.They emphasize that the quality of life of elderly always needs to be preferred over convenience of a system and identify six different ethical issues that designers as well as users need to be aware of: • the potential reduction in the amount of human contact; • an increase in the feelings of objectification and loss of control; • a loss of privacy; • a loss of personal liberty; • deception and infantilisation; • the circumstances in which elderly people should be allowed to control robots.
According to Sharkey and Sharkey, there are three types of robots that can attract various of these ethical issues: those that assist elderly and their carers, those that monitor a user's behavior and health and those that provide companionship.Monitoring robots such as those in our proposed idea, can increase safety for elderly people, remind them of their medicine and make it possible for medical staff, family or friends to visit an older adult virtually.On the other hand, this can lead to reduction of human contact and companionship, as virtual visits cannot substitute for sharing space and giving warmth.Another issue of monitoring robots is the autonomy of the robot, especially when caring for elderly people with mental illnesses [3].The main question here is to what extent a robot should restrain the freedom of elderly people and how authoritarian it should be, especially when protecting them.Also, the ethical issue of infringing privacy applies to monitoring robots, especially in situations where an elderly person takes a bath or gets dressed.This raises the question of who should receive access to collected data and information and how long it should be stored.Cavoukian et al. [4] suggest rules that need to be followed to secure data and privacy.Communications should be encrypted, secured and more reliable.Also, all involved parties need to ensure confidentiality, especially when communicating wirelessly.The collected data of a person need to be uncorrupted in order to ensure quality of health care.
The reduction in the amount of human contact is also an ethical issue that needs to be considered when designing assisting robots like our robot platform.In addition to taking over tasks from medical staff, assistive robots can also do daily tasks such as cleaning or carrying laundry and thus reducing daily encounters and valuable social interactions of elderly with different people [5].Another issue for assistant robots is the target group designers have in mind: are these robots designed to assist elderly people or rather to improve lives of carers and cut costs?[3] Furthermore, an important issue, especially for the concept we are proposing, is the amount of control an elderly person should have over the robot and who allows or assesses this empowerment, especially for people with mental illnesses.Sharkey and Sharkey stress the importance of the right balance between giving elderly people mobility and protecting them from risky situations.Overall, ethical issues concern the liability concerning assistive robots: who is responsible if a robot lets an elderly person crash into a wall; fall onto the floor or out of bed.Who is held responsible if an elderly person controls a robot the wrong way resulting in damage and injury of other patients or staff?
To sum up, there are various different ethical concerns depending on the nature of the robot and its tasks.However, there are also several reasons that argue for a positive contribution of robot technology, especially if it is introduced appropriately.Assistant robots can empower elderly people in their mobility, reduce stressing or embarrassing situations such as bathing or toileting and monitoring robots can prolong independent living in homes through virtual visits and data collection [3].Furthermore, robot technology reduces costs and workload for medical staff.When used and designed appropriately, robot technology can enable elderly people to live independently in their homes, improve their wellbeing and at the same time protect their human rights, physical and psychological welfare.Here, Rashidi and Mihailidis [6] stress the importance of the design and the design process of the system.Also, training and adequate introductions to the system can help to make people feel more comfortable using it and avoid mistakes.According to them, all stakeholders like older adults, system developers, researchers, caregivers, physicians or medical staff need to be included in the process of testing usability and user experience issues when designing systems for ambient-assisted living.In addition, Sorell and Draper [7] emphasize the importance of older people's attitudes when developing an ethical framework for the evaluation of carebots.Sharkey [8] introduces a framework using the Capability Approach to assess positive and negative effects of robot care for elderly people on human dignity.However, she concludes that the framework needs to be studied further to specify particular situations and concerns which indicates that more research has to be done within the field of ethical issues on robot care as robot technology is growing.

Technical Considerations
As discussed in the introduction, there are several advantages in using robots in telehealth applications.In this paper, we explore how besides being fit for its telehealth application, human-robot interaction will also alleviate the feeling of objectification, loss of privacy and personal liberty that have been identified as key ethical issues in elderly telehealth robotics.
More specifically, we explore HRI using tablets, smartphones, and similar mobile devices to communicate with a robot that is used as a telehealth data collection platform.One immediate advantage is that the data collected from the robot's sensors can be complemented by data collected from the mobile device's sensors.Collecting telehealth data from mobile devices, laptops, and desktop computers is well-studied and significant results are immediately applicable [9].
The further advantage and challenge is to design the robot/mobile system is such a way that not only do they complement each other in data collection, but also that this complementarity stresses the unobtrusive and natural character of the system, so that the system as a whole is both more useful and more easily accepted than either the robot or the mobile device alone.Suppose that the user communicates with a robot via an application to request a service, such as that the robot brings them something.There are multiple ways to set up the system, depending on what we want users to perceive as being their "sentient" interaction peer: • The robot presents itself as the cognitive system, with user and robot using the app as a communication channel.
• The robot presents itself as the cognitive system, and offers the app as a means of interaction.
• The app presents itself as the cognitive system.The user interacts with the app, and the app might decide to control the robot (or other devices) in order to achieve a goal.
Using an application via a mobile phone or a tablet to interact with the robot has multiple advantages in this respect.To begin with, it is possible to develop applications adapted toward personal background (e.g.gender, age, culture) and cognitive and physical abilities (e.g.special apps for disabled users).Besides from interacting with the robot, an application in combination with the sensors that are integrated (or can be integrated) in a mobile device can be used to monitor activities of daily life (ADL).This can be a valuable tool for both informal care-givers and medical professionals who can use data collected by such applications.In addition, an application on a mobile device that is not part of the "robot body" allows multiple users to have access to the same robot.In a scenario where a robot is used in a hospital one robot could be serving multiple users.Last but not least, the design and implementation of an app is cost efficient.

Background
HRI literature suggests that the effectiveness of interaction depends on factors related to user personality and background, the characteristic of the robot, the medium of interaction, as well as the communication strategies such as direct or indirect speech.
From the perspective of the users, the range of factors that influence the acceptance of healthcare robots by elderly people has been found to be very wide: age, needs, gender, experience with technology (and robots in particular), cognitive ability and education, culture, role and anxiety and attitudes towards robots [10].Several studies have mapped how user characteristics and background determine what robot behaviours are appropriate.Takayama and Pantofaru [11], for instance, determined that robot gaze has a different influence on women than men and that people who have experience with robots or have pets feel more comfortable when being approached by a robot on short distance.
Robot embodiment plays a key role in its assistive effectiveness [12].Research shows that users find an embodied robot more appealing than a virtual agent [10,13].It is important that the robot's embodiment fits its abilities and intelligence in order for the user not to get confused [12,14].If a robot looks simple, users do not expect it to perform on a high level.If it looks technically complex, the user will expect the robot to perform at a high level.Moreover, recent research has demonstrated that social robots are more easily accepted when they conform with stereotypes that match their occupational role (e.g.healthcare, security) to the "gender" and "personality" that they are designed to possess [15].
Fischer et al. [16] found out that physical embodiment and a robot's degrees of freedom influence human-robot interaction on different levels.A robot's embodiment affects the interpersonal domain, meaning in which way the robot is perceived as an interaction partner for the user.The degrees of freedom affect the user's evaluation of suitability for a certain task.
Different interfaces have been integrated in order to realize a non-expert's interaction with the robot.Most of these interfaces integrate human-human interaction features such as natural language, voice recognition, gaze and gestures [17][18][19].Other assistive robots are controlled by the user with the help of handheld devices [20][21][22].Panek and colleagues [23] use a LED projector unit on the back of an Aldebaran Nao robot to enhance assistance.
Using tablets as an HRI medium has been a natural development, as tablets are becoming familiar, in fact, ubiquitous devices.Assistant robots like the Care-O-Bot [24], Pearl [25], HOBBIT [26] are equipped with tablets that can be used to communicate with them.Closest to our setup is the assistive robot CASERO that can be controlled by a tablet to conduct simple carrying tasks [27].
An application offers the advantage of personalization.For example, Granata and colleagues [28] explored the pictures that should be used when designing a tablet interface of the Kompai robot for people with cognitive disorders.
However, there are still many parameters that can be manipulated to enhance the communication such as use of language and degree of interaction.For example, the role of politeness conventions and conveying the contextually appropriate ways of communication has been widely investigated for human-human interactions in various settings.However, we know little about how politeness should be integrated in HRI to increase user acceptance and satisfaction.Even though not all human-human interaction features can be applied to a human robot interaction, Torrey et al. [29] suggested that natural human-human assistive interactions can help to plan effective human robot assistants.On the other hand, Salem et al. [30] found out that the interaction context (goal directed vs open dialogue) has a greater influence on participant perception of the HRI than the use of verbal politeness strategies.
The set-up of testing the aforementioned parameters seems to play a crucial role to the results.Strait et al. [31] claimed that results differ when communication strategies are tested via indirect interaction scenarios where the user watched the interaction passively such as in online questionnaires than when users actually experience a real interaction with a robot.Bainbridge et al. [32] investigated the advantages of interaction with physically present robots instead of video displayed agents.Their findings indicated that physical presence of a robot can influence trust, respect and possibly other factors of social interaction.
There is need for more and in-depth investigation to understand how users communicate through apps with Robots.An application, especially when used by elderly citizens, needs to be user friendly.However, Salvini et al. [33] differentiate between usability and user's willingness to interact with the robot as two separate aspects of HRI: If a robot is not accepted by elderly users, the user-friendliness of the design is useless.Another advantage of using an application is the ease of accessibility and use: end-users can download the application on their own, familiar, smart phones or tablets.Ease of accessibility can influence liking the system [34].At the same time, the development and distribution of an application is easier and more costefficient.

Robocoffee: A Simple Demonstration
We set-up a simple demonstration 1 to gain insights into: a) the factors that influence user perception when interacting with a robot (actor-actor interaction), b) how we can manipulate these factors, c) the challenges entailed in the interaction between a robot that carries out a daily life task and home automation.Specifically, we focused on studying whether the experience of interacting with a robot via an application could be influenced by using formal or informal language.
In our demonstration a user asked from a robot a cup of coffee.The steps of the demonstration are as follows (Figure 2): • The user follows the on-screen instructions in a mobile device application to place the order.
• The robot approaches the user and the user places the cup on the robot.
• The robot navigates to the coffee container.
• The robot interacts with the coffee container and an electric valve opens to pour the coffee.
• The robot returns to the user, identifies the person and delivers the coffee cup.
The mobile device application was used as a medium of communication between the user and the robot.Text and picture in the application were combined according to the principle of redundancy, repeating the same key idea in the text and in the picture [35].During the ordering phase the user was asked to provide a closeup video.At this instance, we acquired a close-up video with the pretext of face recognition upon delivery of the coffee cup from the robot.In this unobtrusive way, vision data were collected that could be used for health data analysis by experts.By integrating this instance of data collection we aimed to promote health data collection, without infringing personal liberty and privacy.
In the second step the robot approached the user. 2  Besides the text on the application asking the user to place the cup on a tray, the robot produced a beep sound 1 The software for the RoboCoffee demo has been developed on GitHub and is publicly available at https://github.com/roboskel/RoboCoffee 2 More information about the platform used can be found at  to catch user's attention.Once the robotic platform approached the user, the cup had to be placed inside a box integrated on the robot.The box housed two sensors; a force sensitive resistor sensor that measured the weight of the coffee poured inside the cup and a proximity sensor that signalled the presence of a cup into the box.
Following, the robot navigated to the coffee container, it aligned the cup (i.e. the coffee box) under the electric valve by reading an AR code attached on the body of the container.Proper alignment signalled the opening of the electric valve.The electric valve was controlled from a relay mounted on the coffee container.A Hall effect flow sensor with PWM output, attached on it measured the volume of the coffee poured.The relay was triggered by a Raspberry Pi Server and the flow meter was monitored from an Arduino.The two of them communicated to guarantee robust communication with the robot.
When the cup was filled with the predefined amount of coffee, the robot was then allowed to navigate back to the position of the order and look for the user.As the vision part of demo was not of our primary interest, we approached the problem of identifying the customer as a simple one-class classification problem.Color histogram features extracted from the face and t-shirt of the customer were compared to the features extracted from people standing next to the customer upon delivery of the coffee.Although, we tried this simple implementation as a proof of concept for a complete implementation of the demo, the vision detection and identification were not part of the demo that participants tried.
Five participants (mean age: 27.5, age range: 24-32, 2 female) participated in the study.They were recruited by word-of-mouth at the Institute of Informatics and Telecommunications, NCSR "Demokritos" and were all participants of the 2014 International Research-Centred Summer School. 3All participants had a computer science/engineering background and had worked with robots before.
The participants were instructed to use the mobile application to order coffee from the robot.All the steps were followed as described previously except for the visual identification part.Moreover, the user and the coffee container were always at the same room and position.All five participants, ordered coffee by using two different versions of the application.One version used formal language, while the other informal.For example, while the user was waiting for the coffee to be delivered the app in the formal version would display "Your order is being processed.Thank you for your patience", while in the informal version "I am now getting your coffee, please bear with me".
In terms of interaction between the mobile platform and home automation, we managed by using commercially available, cost efficient solutions to build a basic interaction between the two actors.More delicate implementations will be needed in larger-scale installations.However, our demo confirms that this sort of communications are at a ready-to-use state.
As mentioned in the beginning, the main focus of this demo was to gain some insight about how people perceived the interaction with the robot via the mobile device application.After trying the demo the subjects were debriefed about their experience.The first part of the debriefing concerned aspects of the application.Specifically, they were asked: • whether they noticed any difference in the two versions of the application.• if the steps to follow while using the application were clear.

EAI
• if they found the application user-friendly.
• if they felt that they interacted with the robot.
All of the participants thought that the application was user-friendly and it was easy to follow the steps.However, none of the participants thought that the two versions of the application were any different.Moreover, they had problems in conceiving the application and the robot as an integrated system.That was mainly because the robot and the mobile device were physically apart.Participants felt that the robot was only communicating with the app and that the participants only interacted with the application.Moreover, some participants stated that they had trouble giving useful feedback given the set-up of the experiment.

Concluding Remarks and Research Plan
At the core of the work described here is our exploration of the human and automated actors involved in assisted living and the autonomy each actor enjoys.Besides the capabilities of each actor in what they can possibly undertake, ethical and medical considerations should also be considered.On the one hand, the automation should not step in to fill all vacuum of responsibility and decision making even when technically possible, in order to avoid creating feelings of objectification and loss of control.On the other hand, this should be balanced against medical necessities, such as following medication regimes.
The basic idea is that privacy, personal liberty and control can be improved by our design, without compromising the medical data collection that necessitates deploying the system in the first place.
We have presented ideas and a preliminary experiment on using a system that integrates a mobile robot, home automation, and a mobile device application as a data collection platform for telehealth applications.In our experiment, we developed a demonstration where a system collected the footage it needed in order to be able to identify the person who ordered the coffee without scaring "customers" away into preferring the non-robotic alternative of getting coffee themselves.This combined the utility of getting the coffee with the functionality of obtaining close-up face footage, useful for remote health monitoring.
Our research plan involves investigating how humans communicate with the robot through an application.We observed that most of the people that participated in the demonstration above did not feel that they interacted with the robot but with a mobile device app.As mentioned above, a reason for this was that the mobile device and the robot were physically apart.As we wish to keep the mobile device as a separate component, we plan to enhance the interaction with the robot by adding more dominant behaviour to it, such as more prominent sounds or even personalised welcoming and offering-the-coffee messages.Moreover, manipulating the language to influence user perception needs further investigation.Finally, a more home-like experimental environment (rather than the lab environment) could allow for a more natural interaction.
Furthermore, we find it interesting to carry out further experiments exploring how people perceive the interaction with the system and how this perception influences what they expect from it.Put simply, we will explore if people have higher expectations from systems presenting themselves as robots than from systems presenting themselves as a mobile app, even when identical functionality is offered by both systems.

Figure 1 .
Figure 1.Conceptual Design of a Robotic Smart Home.The decisions are made in a top down way.

Figure 2 .
Figure2.A graphic depiction of our mobile device for HRI demonstration, showing the process of placing an order, having the order carry out the necessary steps to fulfil it, identifying the person who placed the order, and making the delivery.