Smart Mirror Based Home Automation Using Voice Command and Mobile Application

The smart mirror helps humans to use their time effectively. It serves as the simple mirror and allows the user to see important updates and control electrical appliances simultaneously. Smart mirror is developed to let the user do multi-tasks while he was leaving from home. This system consists of two applications one is the smart mirror application developed using python and Tkinter library. The second one is smart mirror mobile application developed using the IONIC mobile development framework. Firebase firestore real-time database is exploited to show the user the latest news, weather updates, the current date, and time. It also allows the user to add his to-do list items. The user can control home appliances using voice commands and mobile applications. There is also a security module that captures the image of an intruder and sends the message to the mobile application. Smart mirror is developed to reduce human efforts and to allow the users to use their time effectively.


Introduction
It is seen from past to present that the world transforms globally from manual to automatic, that is self-monitored and controlled mechanism is built in the devices. In present world, smartness found in every object from a small to large objects that enables automation, by reducing human intervention for processing and control. Internet of things (IoT) connects the things such as smart phones, cameras, laptops, televisions … etc. with each other through internet for performing different tasks in short period of time without using the work force of human being. One of the advance technological devices is smart mirror that provides the multilevel functionalities in order to ease the working capacity within limited period. The Smart mirror is a connection to sensor, actuators, cameras, Wi-Fi, Bluetooth or other devices for automated tasks in house hold activities, educational platforms, industries , hospitals and more [1]. The smart mirror makes humans life easier by saving user time in performing their daily tasks. Effective time management is necessary in the current world. The smart mirror saves users time through multitasking and interaction with not only people but also to the other objects. Today the centralized principle of technological world is to automate, each conceivable thing for simplicity in life, providing security, saving electricity and time [1]. The smart mirror has the benefit of using new technology while fitting efficiently into most individuals' daily life. It is essential to make the smart mirror simple, clear, and as practical as possible. To integrate technology and the need for information into the regular routine of everyone, the smart mirror will be used. During their usual morning and night routines, users can communicate and obtain the data they want by using smart mirror. That data used to obtain useful results and make mirror smart by applying techniques discussed in [2]. Smart mirror is a device that saves time in doing daily tasks [3]. It shows the current date and time on the Mirror so that users can know the time and save ourselves from getting late. Smart mirror shows the current news and daily tasks that user can manage through the smart mirror mobile application. Automation is the use of control systems and information technologies to reduce the need for human efforts in the production of goods and services [4]. It also helps to control electrical appliances using voice commands. It also has a mobile application through which the user can also control electrical appliances from anywhere in the world with an internet connection. Smart mirror also provides the user with the security feature. It detects the motion and sends the data to the mobile application So that the user can know if there is someone in his house [5]. This smart mirror objects to reduce and possibly eliminate the need for the user to make time in their daily routine to check for the information they need. Home automation is the term use for automatic control of home appliances and their remote management. The main objective of this paper is to represent the system that is design to acts as a smart mirror and provide the communication to home appliances through voice commands and mobile applications. Smart mirror approaches are also supporting home automation systems which widely comprise IoT, control systems and special purpose sensors [6][7][8][9].
The rest of the paper is organized as: Section 2 presents the literature review based on the works previously done by researchers. Section 3 presents the overview of existing system with the help of working, block diagram and flow chart. Section 4 presents the complete detail of hardware used for the system. Section 5 presents the advantages of smart mirror based home automation using voice command and mobile application. Section 6 presents the Result and Discussion of the proposed system. Section 7 presents the conclusion of the system.

Literature Review
A broad range of features and uses is covered by initiatives and products similar to a smart mirror. There have been considerably more initiatives than real goods [10]. Some blame can be placed on the fact that the smart home is still an emerging market and is constrained by the cost of production to keep the goods out of the ordinary consumer's reach. The systems researched are found below: "Home Automated Smart Mirror as an Internet of Things (IoT) Implementation" was a system developed by Jose and Jane. This system possesses the ability to display date and time, the current weather condition, and outside temperature. This system is just the simple representation of smart mirror which shows the information scraped from the internet [11].
A smart mirror to monitor children by using Raspberry Pi technology is the optimal system for monitoring. This Raspbian magic mirror display useful information such as the date, time, weather, and daily reminders, also helps parents to monitor their children and assist them with their studies, and to organize their daily routines [12]. Another system named smart mirror using Raspberry Pi was developed. This smart mirror aims to display the information to the user like time, date, weather calendar, reminders, and news-updates [13]. The android based home automation system using Raspberry pi was eminent development in the field. The purpose of this system is to control all the home appliances through the smart phone. This concept is taken from the concept of automation from industries work done in [14]. This system consists of two applications one is the smart mirror application which is developed using python and Tkinter library for raspberry pi [15] and the second one is smart mirror mobile application. The user can increase or decrease the speed of the fan, turn-on or off the light, and many more appliances at home through smartphone or tablet. So, this is implemented using Raspberry pi and relay. The devices are control through WIFI or GSM [16]. IoT based solar house monitoring and control system is presents in [2], which uses IoT based system for data collection and control devices of home. A System is developed with voice controlled home automation using Natural Language Processing (NLP) and Internet of Things to automate the basic operations by sending the voice command through mobile devices to appliances. This system uses Arduino board for establishing communication. The application sends commands and receives responses through Wi-Fi [3].
A project of Smart Cities is presented in [4] that use mobile application for monitoring with the help of smart devices. The application controls various devices of home such as light, AC, fan … etc. for fast management and control. A complete review of applications is discussed in [5] that highlights the use of Arduino Uno microcontroller, Global System Module, Light Emitting Diode technology (LED), Raspberry Pi, passive InfraRed (PIR) sensors in different research papers. This paper is also presents the applications of Smart mirror for displaying time/weather/news/road traffic information. This paper also focus on the application of smart mirror in scheduling email read and reply, to send emergency calls, improving people's life, facial recognition and interaction through Artificial Intelligence (AI), for detecting medical condition and mood of a person. This paper highlights the use of smart mirror in fashion field for virtual judgment of dresses.
The microcontroller which is prominent for these types of IoT projects is Raspberry Pi technology exercise in advanced applications. This specific project has highlighted featured with interactive media, personal knowledge that backing end user in the daily enterprise. Human mechanization with IoT accords colossal aid. It is all about sensors which are being used today to make our life easier and relaxing. It is observed in all cases of home automation devices to provide real time information which is broadcasting internationally in the digital world [10]. The interactive machine is designed to support the contextual data such as image, video in voice [17]. Internet of things made possible to communicate devices with each other simultaneously. The quality in this project is lied in dual working mainly first one is the surface which is reflective known as mirror and second one is interactive screen known as smart mirror which updates us and makes our life much more easier.
An integration system of Internet of Things and Wireless technologies is designed for secure smart home automation that is managed by Artificial Intelligence voice controlled system [10]. Using human face expression detection and pattern recognition it is possible to observe the feeling tone of a person that they observe in their different kind of moods such as nervousness, unhappy, joyful, annoyed, irritable... etc. Using smart mirror it is also possible to detect human mental condition and mood. Google cloud and virtual machine used in cloud computing for detection of feeling and emotions of a person to make it possible to generate better feeling in frustrated or tensed person. A smart mirror based system is discussed in [11] for analysis of user emotions and encouragement of positive feelings. In today's world most of the people want to lose weight and try to be fit and healthy in order to maintain a suitable life style. In social media platform, websites, Gym and other places, it is observed that a number of fitness trainers and software, use to provide fitness training. A trend of online classes is also common for weight loss and in maintenance of healthy life style. A smart mirror based system is presented in [12] that personalize user training for health and fitness. This system provides a virtual teacher for the user and makes corrections in case of wrong steps. A survey was conduct, to find the intention of users for using smart mirror [13].

An Overview of Existing System
In the current existing systems, a mirror is designed that acts as a smart mirror, this mirror is made of a frame and the LED monitor is placed behind the two-way mirror. The frame is developed using wooden and nails. There is a two-way mirror in front of the wooden frame and the LED monitor is placed and the back of the wooden frame. Multiple features allow the user to save time and increase his productivity. The user sees temperature, news, clock, and time updates. Voice-controlled electrical appliance system is also implemented using Google speech API. The mobile application is also developed for controlling smart mirror when the user is not at home. Firebase firestore is used as a means of communication between the IONIC mobile application and the Raspberry Pi. The main purpose of building a mobile application is that the user can also interact with a smart mirror even he is not at home. User can control electrical appliance using the mobile application, push or delete to-do list items and can see security messages which were sent to users when raspberry pi detects any intruder. The user can also delete security messages and will be able to add new tasks.
There is also a section from where the user can see the about section of the mobile application [18].

Figure 1. Working of the System
As clearly seen in the above picture we have divided this system into four parts of better understanding. These parts are mobile application, services/backend, smart mirror application, and hardware used in the system.
The mobile application and smart mirror applications are communicating through a firebase firestore real-time database. There are three modules in the mobile application control devices module, the to-do list module, and the security module. These all three modules interact with the firebase and use firebase to update, store or retrieve data which afterward can be used to update, store or retrieve data in a smart mirror application using the python programming language. Similar work done on many IoT based concept like smart traffic system with geofencing in [19].
The weather widget fetches all the weather updates from OpenWeatherMap API and the news widget fetches all the news information from NewsAPI. BBC News and Sports news are used for news widget. The clock widget displays the current date and time of the system.
The Voice command module allows the user to gives voice commands for controlling electrical appliances using the USB microphone. The control device module consists of a relay module to control electrical appliances in the response to commands from mobile applications or voice commands module. The security module consists of a PIR Sensor and PI Camera which helps to detect the motion of the intruder and sends a message to the firestore which is then fetched by mobile applications. It also stores the image of an intruder in the raspberry pi. This client server process is well described in cloud process paper [20]. There are several distinguished approaches which are supporting/can support AI to boost the enactment [21][22][23]. Nevertheless, IoT and computing methods are involved in the evolution of AI [24,25]. and smart mirror turned off until some object appears in front of the mirror.

Application of Smart Mirror
Medical field also uses the IoT based technology to tackle difficult problems and save their time one of them is smart mirror one author develop it to detect the facial expression and clue to determine the condition of patient [26]. Overall this application is general that every field will get benefit from it and apply to different environments such as hospitals to detect patients by self. In the field of fashion a smart mirror performs the role of 2D visualization in order to wear and see the clothes before apply on human, the function of smart mirror is to analyse estimate and recommend wearing [27]. Smart mirror is observed under consideration in academia also for research purposes to collect the data like in one work is done to recognize facial expressions and comeup with mood like happy, sad and angry [28].
Last but not the least the field it is used is Sport field in the way to recognize the fitness of the player and mood of the player by setting some parameters to check either player is in which condition regarding the health which also encourage the sports persons to update their physical activity [29]. It is seen from the figures that represent that overall smart mirror application I used in general field with 57.5% in medical it is 15.7 % fashion field around 15%. Academic and sports uses only 5% which quite less.

Flow Chart
This project flowchart is operating on the base of condition which is primarily start of the device. When this work done the next step to install the mobile application and start surfing on it. So as we start to run this mobile app it must be connected with internet to further go inside the features of the app otherwise it stops to go further in next step. After availability of internet on mobile device, the app requires to select or choose the job one at a time and end user will see the updated regarding this on the smart mirror while using mirror and this will both save the time of the user along with aware of the updates chooses by end user.

PIR Sensor
PIR sensor allows us to detect the motion mostly used for detecting human motion. The HC-SR501 PIR sensor is used in this system to detect the motion of humans. If the sensor detects an intruder it sends the data to firebase which will then be fetched by the mobile application and Shows the user the message that 'Motion is detected' at the same time it triggers the Camera Module to take the picture and store it locally in the Raspberry Pi [30]. Pyroelectric infrared (PIR) sensors are considered to be promising devices for device-free localization due to its advantages of low cost, energy efficiency, and the immunity from multi-path fading [31].

USB Microphone
Raspberry Pi 3B+ 3.5mm jack does not support audio-in, it only supports audio-out [32]. So, for the purpose to make the raspberry pi listen to the user's audio USB Microphone is used. It is made of plastic along with metal and provides noise cancellation with the ease of just a USB plug and play connection.

Relay Module
Relay module is used in this system to control electrical appliances using Mobile application and voice commands [33]. Users can connect from a single device to up to many devices using a relay module. A relay module is a device that provides the function of an electrical switch. It is operated using an electromagnet which is powered by a low voltage signal from any microcontroller. Below is the picture of the 5V DC four-channel relay module.

LED Screen
LED Screen is used in this system for showing the displays to the user. It is placed behind the mirror so that it can act as a smart mirror. This work is done in [34] to perform multifunctional tasks.

Advantages
Following are some advantages of smart mirror based home automation using voice command and mobile application: • The first and main advantage of this system is to help the user save its time while getting ready in front of the mirror. The user can save its time by seeing weather, news, and time while getting ready.

•
The home automation feature allows the user to control its electrical appliances using Voice Command and Mobile Application.

•
The use of Google speech API for text to speech and speech to text conversion for controlling the electrical appliances is one of the key features of Smart mirror.

•
The mobile application is developed using the IONIC framework which allows the development of mobile applications for android phones, IOS phones, and web applications with just a single code base and few changes. • Security Module is another good feature that provides the user the ability to get alerts if someone broke into his house.

•
The mobile application allows the user to interact with the smart mirror even he is not at home.

•
The use of a firebase real-time database makes the system updated to the real-time behaviour of the system.

Results and Discussion
Today the technological worlds centralize principle is to automate each conceivable thing for simplicity in life, providing security, saving electricity and time. This paper consists of two applications one is the Smart mirror application which is developed using python and Tkinter library for raspberry pi. The second one is smart mirror mobile application which is developed using the IONIC mobile development framework. Smart Mirror is developed to reduce human efforts and to allow the user to use their time effectively. It detects the motion and sends the data to the mobile application So that the user can know if there is someone in his house. This smart mirror objects to reduce and possibly eliminate the need for the user to make time in their daily routine to check for the information they need [35]. And finally the mobile application and smart mirror applications are communicating through a firebase firestore real-time database. There are three modules in the mobile application control devices module, the to-do list module, and the security module. These all three modules interact with the firebase and use firebase to update, store or retrieve data which afterward can be used to update, store or retrieve data in a smart mirror application using the python programming language. Nevertheless, this application can be remarkably improved by using some key image processing approaches [36][37][38].

Conclusions
Smart mirrors are the most innovative technology of this decade. These are providing benefits to the human in many ways like allowing them to save their time, managing their routines, getting important updates and making their life much easier. These can be used in every aspect and field of life by utilizing them in a proper way they can be added with many features. The goal of the smart mirror is to reduce the time of the user while doing multiple tasks. The user can save time by seeing all the important information on the screen while doing his task. The feature of home automation makes it very convenient and comfortable for the user to control electrical appliances in their home. The user can even interact with the mirror when he is at work or not at home with just a mobile application. In today's world, the security of the house is one of the most important features of any system, therefore, it is also implemented a security module in this system. Smart mirror makes it easy for everyone to use their time effectively; there can be a lot of new features that can be added to enhance the user experience. The system is designed in a way to reduce human efforts and provide benefits to humanity. There are certainly a lot of technology integration opportunities in the home, but a smart mirror is one of the best places to start.