mBeacon : Accurate , robust proximity detection with smart phones and smart watches using low frequency modulated magnetic fields

We describe a proximity detection method that leverages the internal magnetic field sensors in smart phones and smart watches to allow reliable, robust detection of proximity to predefined regions of interest within a 30-50cm radius. For marking the regions, low cost, unobtrusive, easily deployable magnetic field beacons have been implemented. We evaluate our system in 5 different use cases in semi-naturalistic lab experiments inspired by different application domains.


INTRODUCTION
Proximity is a key piece of information in ubiquitous computing applications.As a consequence much research has been devoted to gather this information (see related work below).In this paper we focus on the question how to detect the proximity of a mobile consumer device such as a smart phone or a smart watch to a predefined region of interest such as a certain table, part of a workspace, shelve in a cupboard or part of a clothes rack.
Current state of the art in proximity detection with mobile con-sumer devices are Bluetooth 4.0 (Bluetooth Low Energy) beacons (e.g.iBeacons).While the detailed accuracy of such systems depends on the specific setup and the environment, it is in general well above 1 m and thus unsuitable for higher precision applications such as distinguishing between neighboring desks or doors, detecting which shelve someone is reaching for, or tracing hand positions when browsing through a rack of clothes.
Motivated by such application scenarios, in this paper we describe a solution based on low frequency ( 10Hz) modulated magnetic fields with the following properties: • It leverages the internal magnetic field sensors in smart phones and smart watches and performs all necessary processing on the device.
• It allows highly reliable, robust detection of proximity within a 30-50cm radius.
• The regions of interest can be marked using cheap, unobtrusive, easily deployable hardware.
Our approach is based on the insight that the limitations of state of art Bluetooth beacons based proximity detection systems can be traced to the physical principle behind radio frequency (RF) signals.By virtue of being electro-magnetic waves they are susceptible to reflections and refractions.In addition the 2.4GHz frequency band used by Bluetooth is strongly absorbed by the human body.By contrast, static magnetic fields are neither reflected nor refracted, and are hardly absorbed by human body.While massive metal objects can significantly disturb magnetic fields such disturbances are localized (essentially bending of field lines within the object) and have little effect on the signal further away (as opposed to reflections and refraction of RF signals).Using modulated rather than static fields allows us to deal with magnetic noise that is nearly always present in the environment.The above considerations are well known and have previously driven the development of different magnetic positioning and tracking systems mentioned in related work.The contribution of this paper is to show how to leverage them for proximity detection within the limitation of the magnetic field sensors typically found in smart phones and smart watches and practical considerations related to the generation of the field with easily deployable, low cost setups.

RELATED WORK
Our work is related to several research directions.First, is the general area of indoor localization [4], where however only systems that rely on the built-in sensors of a popular consumer wearable products like smart phone or a smart watch (which is a focus of this paper) are relevant.This includes WiFi based systems [17], Bluetooth beacons [19], and inertial navigation [10].While much progress has been made in all of these technologies (including the fusion of all three above modalities [6]), reliable recognition of proximity within a range of 30 cm-50cm is still beyond the state of art.This is particularly true in dynamic environments where people and objects influence the RF signal propagation (while having less effect on static or ultra low frequency magnetic fields).
Second, there is a considerable amount of research on using magnetic fields for object tracking, positioning, and proximity detection.Again, vast majority of work to date is not applicable to smart watches and smart phones.This includes passive tag tracking as proposed e.g. by [12], a broad range of approaches based on standard passive RFID tags (which require a dedicated RFID reader) [2], [14], as well as various systems that use oscillating magnetic fields for positioning [16] or [15].The latter use field frequencies in the range of kHz, which the sensors in smart phones cannot detect.For example in [9] a 125kHz oscillating magnetic field is used to detect proximities to given beacon emitters using magneto inductive coils.The Location-log system described in [18] is also based on 125kHz magnetic fields.The authors use a special purpose NFC hardware at 125kHz to measure the induced voltage in a receiver coil which is attached to the mobile phone.The external coil allows to measure magnetic fields with lower field strength values but requires more space and is not IC based and, therefore not as wide spread as electronic compass sensors which are basically in every smart phone.In order to retrieve information about objects persons currently interact with, a sensor modality attached to the hand or close to it is necessary.Gloves with integrated sensors (survey of this technology can be found for example in [5]) or skeleton models [11] allow to gather this information, the additional hardware typically reduces social acceptance of these approaches.Sensor integrated in watch-like devices as in Activity Trackers or Smart Watches are more and more common and therefore a good starting point.
In [13] the fields generated by power lines embedded in walls are used for proximity estimation with special purpose sensors.
Closest to our work are approaches that rely on ambient magnetic field strength distributions for indoor localization as presented in [7].The authors use the magnetic compass sensor attached to a robot to generate a map of the ambient magnetic field.In contrast to our proposed system the authors use the ambient magnetic field and the magnetic field anomalies arose due to disturbances.The accuracy was around 30 cm.The authors of [3] also use the ambient magnetic field to achieve a localization accuracy below 1 m.Both approaches require elaborate fingerprinting and assume a static magnetic environment, which is a significant practical difference to our system.
On a technical level similar to our approach is the work by Jiang described in [8].The authors use magnetic fields to transfer data between a computer and a mobile phone using the on-board compass sensor.The system covers data transfer in close range below 1 cm with a data rate of 44 bits per second.Similar range limitation applies to NFC technology included in many modern smart phones.By contrast, we consider proximity detection on the range of 50 cm.

SYSTEM DESIGN AND IMPLEMENTA-TION
Our system consists of three components: an RF based iBeacon for rough proximity detection to reduce the overall power consumption of the magnetic field coils, a magnetic field coil with attached driving hardware to modulate the id of the region of interest, and a mobile device (either smart watch or mobile phone equipped with a magnetic field sensor) sampling the environmental magnetic field to enable the demodulation of the id of the region of interest embedded in the artificial magnetic field.

Basic Principle
The basic idea behind our approach is to mark the relevant regions of interest with magnetic beacons that are modulated with a region specific ID-code.Whenever the user's phone or smart watch comes within a predefined range it receives and recognizes the code.The range is determined by the peak strength of the magnetic field B(d) and its strength drops as the distance to the coil drops.
The design space of our system is given by the limitations of the magnetic field sensors incorporated in mobile consumer devices such as smart phones and smart watches (e.g.compass sensor AKM AK8963).Typically, such sensors have a sampling rate of 10-20Hz and a measurement range between ±1mT and ±5mT .The base signal that they continuously detect is the earth magnetic field (30µT at the equator, 49µT in Europe).
To determine the sufficient number of windings, current and the shape of the magnetic field coil, we implemented a python script which allows to simulate the magnetic field behavior of the coil.The script supports 2 coil shapes, a circular shaped coil and a frame shaped coil.Using a finite element approach, the field fraction of the positions on the coils are calculated for a given position around the coil in 3d space.To accurately determine the id of a coil, the artificial magnetic field strength at any supported position around the coil should be at least as high as the natural magnetic field.The script is available at www.dfki.de/ei/mBeacon/script.py.

Hardware
From the above considerations the magnetic beacon utilizes a rectangular 40cm × 30cm coil with 20 windings, which can be easily integrated in the environment.As magnetic fields permeate most materials, the coils can be hidden for example under a table, behind a cupboard or within a door frame.The coil is driven by a current limited power supply which is able to deliver a peak current of about 4A at 15V. Figure 2 depicts the field strength trajectory of a frame shaped coil with 40 × 30cm area to which a current of I=4A is applied 1 .Using this approach also assures that the generated magnetic field is not overdriving the compass sensors.as a special purpose hardware this could easily be reduce to a range of 20-30 Euro.Note that although the peak power consumption is 60W, since the system is not transmitting the whole time, the average value is close to 20W.To further reduce the power consumption we use Bluetooth beacons to determine when the user is in the rough proximity and only then switch on the magnetic beacon.

Mobile Device Detection Method
The ID-Code is modulated in the magnetic field using pulse width modulation (PWM).Taking the sampling rate of compass sensors of smart watches and mobile phones into account, the minimum length of a magnetic field pulse should be around 150 ms.Notice that this interval also covers the transient oscillation of the coil.
The modulated values are a multiple of the minimum pulse length The magnetic field strength is sampled using the on board Hall effect based compass sensors of mobile phones and smart watches.In our case these were the Nexus 5 and One Plus One mobile phones (compass sensor AKM AK8963) running under Android 5.0.1 and 4.4.Both phones supported a sampling rate of 17 Hz, for initial test the measurement vector B(x) = (mx, my, mz) was transfered to a computer using Wifi communication.The used smart watch was the LG Watch R with similar performance.For the experiments, the processing sequence (as described in figure 3) was implemented on the watch and the phones.
We transform the measurement vectors Bi into squared vector magnitudes , which automatically weights measurements with higher axis field strength (compared to the earth magnetic field vectors).
The first order derivate of this parameter gives a good indication about the presence of artificial magnetic sources as the activation and deactivation process of the coil results in sharp edges.To suppress signal changes of the environmental magnetic field caused by movements, a median based sliding window is applied to the sensor values.A peak detector finds the exact time when the coil is turned on and off.After detecting when the artificial field is applied, we can use the direction change of the magnetic field vector to determine on which side of the coil the sensor is.Therefore we compute at first the median of the magnetic field vector on each axis during the artificial signal and also in the interval of 100 ms before the signal was applied.Then the difference of these two vectors is calculated.The sign of the largest component of the difference vector, defines the side of the coil.The entire detection runs on the mobile device.

EXPERIMENTS
We evaluated our system in 5 different use cases in semi-naturalistic lab experiments inspired by different application domains (Figure 5).The first two (Figure 5   which a student is sitting while the individual steps involve interaction with devices placed at different regions on the table.The third use case (Figure 5 A)) is devoted to recognizing with a smart watch from which shelve of a cabinet an object is taken.It is relevant for a variety of applications, from task tracking through shopping to the recognition of various activities of daily life.Along the same lines the fourth use case (Figure 5 E)) deals with tracking user's hands when browsing through clothes hanging on a clothing rail.Finally the fifth use case ((Figure 5 C)) shows that with the magnetic beacon embedded in the door or the door frame we can detect which door the user is opening with a key, even if the doors are just next to each other.

Workbench Sitting Location
We attached 4 magnetic beacons below two workbenches (Figure 5  D)) to identify 4 different places next to each others where students can work or perform experiments.Each workbench has the dimension of 220 cm × 120 cm.We split each workbench into two workplaces (each of size 110 × 120 cm).The experiment is motivated by a task tracking scenario where the top level task is determined by the table region at which a student is sitting.While the individual steps involve interaction with devices placed at different regions on the table.Notice that it is a typical hardware workplace which includes machines, metal tools, power supplies and a large anti-static work mat.These influence the AC magnetic field either by reflection, absorption or generate electromagnetic field by itself.
The experiments involved 5 student volunteers.Each subject was asked to sit down at one of the locations and perform a task of their choice for 10 seconds.This was repeated 15 times with a random selection from the four locations.

Interaction with Workbench Regions in a Soldering Task
The workbench (Figure 5 B)) has been equipped with 4 coils, each coil covering a specific object or region which is used in a soldering task.The subjects were asked to solder a surface mounted device (SMD) resistor to a printed circuit board (PCB).This involves 1) placing a PCB on the workbench at location 2, 2) finding a specific resistor in the resistor set placed at position 3, 3) reaching for the soldering iron and changing the tip from big tip to small tip at location 1, and 4) adjust the multimeter to check the resistor value and test if the two contacts are connected at location (taking place at location 4).Five students volunteers wearing a LG Watch R smart watch each performed the above tasks 5 times with no constraints except the above described positions.The activities/positions were x axis y axis z axis  Notice that in the location experiment, there is one event in id 3 where a bluetooth burst disturbed the timing and therefore the length of the artificial magnetic field readings couldn't be correctly be determined.
labeled manually by an observer.For the recognition algorithm the wrist worn smart watch is of high importance as it retrieves the magnetic field information close to the hand.A drawback is that the hand typically moves fast from one location to the other and therefore either magnetic field information is missed or the magnitude trajectory is mutilated resulting in invalid information.

Taking Objects from Cabinet Shelves
A cabinet with 4 shelves (Figure 5 A)) was equipped with 4 magnetic beacons (one for each shelve).Initially we planed to equip the metal shelves of the cabinet but the magnetic field was misshaped and measurements were hardly possible.This scenario also shows the advantage of magnetic fields, it is possible to adjust the applied current so that the magnetic field of a single coil only covers the corresponding bookshelf and does not influence the neighboring compartments.We randomly chose the shelf number and 4 subjects wearing a LG Watch R smart watch were asked to reach for it.Each person reached 20 times for a randomly chosen shelf.

Cloth Rail
We attached two coils to a 120 cm cloth rail (Figure 5 E)) and thus segmented the rail into 4 sections (on the left end right sides of the coils).11 winter jackets were distributed randomly between the 4 sections.5 participants were then asked to browse through the jackets checking the price (attached in the inner side of the jacket).An observer manually annotated when which participant was browsing through which section of the rail.

Opening doors
In this scenario (Figure 5 C)) we attach two coils at two opposing doors that are 1.0 m apart.Within such a tight spacing, standard indoor location algorithms have very often problems in deciding in front of which door a person is standing.In the experiment one door is surrounded by a metal frame, the other door opens into a meeting room which is completely surrounded by metal frames (Resulting in anomalies of the natural environmental earth mag-netic field).Two subjects were asked to open a door 20 times.In each instance the door is randomly chosen.

RESULTS, CONCLUSIONS AND FUTURE WORK
All of the experiments were evaluated on frame by frame and event basis.Frame by frame evaluation refers to every single cycle in which the mobile device tries to decode an ID.Event evaluation was based on the majority of IDs recognized within the window labeled as belonging to a certain location.
As it can be seen in table 1 with three exceptions the frame by frame results are well over 90% and the event based results are 100% correct.The first exception are the areas 2,3, and 4 of the workbench soldering experiment.The errors concern instances where the hand quickly reaches for an object without remaining in the area long enough for the ID to be correctly recognized.If we constrain the evaluation to events that are longer the 3 s then 100% are correct.
The second exception are the first and last region in the clothes rail experiment.The errors occur when the user's hand is closer to the massive metal vertical tail than to the mBeacon.In those cases the ID is correctly recognized, however the algorithm for recognizing which side of the coil the hand is on fails and returns the wrong side.Further improvements in the orientation algorithm are needed to address this problem.The third exception is class 3 in the workbench sitting experiment.Here the error is due to timing problems resulting caused by the task manager on the phone, the magnetic sensor readout is randomly interrupted by the mobile phone scheduler resulting in missing sensor values.
In this paper, we have shown that inexpensive, unobtrusive magnetic beacons allow mobile consumer devices such as smart watches and smart phone to detect proximity to predefined regions of interest within a range that lies between Bluetooth beacons (in general more then 1 sm range) and standard NFC (a few centimeters).Key  limitations are the temporal resolution and the power consumption of the beacons.Both are related to limits of the sampling frequency of the smart phone magnetic sensors (with higher temporal resolution shorter pulses are possible resulting in a lower power consumption ).Future work needs to address intelligent power management of the mBeacons (e.g.synchronization with motion sensors of the mobile devices), more elaborate analysis of the orientation algorithms and investigation of more divers coil setups (e.g. using more windings to reduce the current).

HEALTH CONSIDERATIONS
As the system can be used in real world applications, we also consider the influence of the magnetic field on the environment.According to measurements with the mobile phone, the maximum field strength is 0.0012 T (1.2mT).The guidelines for electro magnetic fields (see [1] for details) limit the magnetic field strength to 800µT for a frequency of 7Hz (the duty cycle is 150 ms for a bit, therefore less than 7Hz is assumed for the field frequency) for general public exposure, occupational exposure is at 4081µT .The magnetic field drops with 1 d 3 , therefore it is sufficient to place the coil in a box to prevent persons from touching the coil but still support magnetic field measurements.Figure 2 (right) depicts the influence of the generated artificial magnetic field of the transmitter coil at a distance of around 30 cm.The superposition results in a field strength of roughly 75µT which is in the safe field strength interval.

Figure 1 :
Figure 1: Schematics of the used coil driving circuit.

Figure 3 :
Figure 3: Processing sequence on the mobile device (smart watch or cellphone).

Figure 2 :
Figure 2: Left: Theoretical field strength trajectory along the norm vector of the coil plain perpendicular to the diagonal intersection with distance d to the plain.Notice the logarithmic scale.It also includes the level of the earth magnetic field strength in Europe.The intersection of earth field strength level and the trajectory is seen as maximum range.(Model values: N=20, a = 0.3m,b=0.4m,I = 5A, frame shaped coil); Middle: Squared magnetic field strength with modulated coil id.The green peaks indicate the edges of the artificial magnetic fields.Right: Superposition of earth magnetic field and artificial magnetic field measured with a mobile phone.

Figure 4 :
Figure 4: Upper left picture: Transmitter coil attached and coiling frame to easily create a coil.Lower left picture: transmitter coil attached to metal door, right picture: Rack with 4 transmitter coils, the current of each coil has been tuned so that the magnetic fields do not influence each other.

Figure 5 :
Figure 5: Different experimental setups.A) Picking equipment from different shelves B) A soldering task with different areas on the desk.C) Distinguishing between different doors.D) Estimating the position of a person at desks E) Finding a specific winter jacket at a cloth rail.

Table 1 :
Results of the different experiments.FbF -Frame by Frame.