Non-obtrusive 3d body tracking for automated mobility assessment in independently living older persons. Results of a pilot trial

INTRODUCTION: With rising age, functional deficit and frequent falls may lead to long-term care admission. Mobility assessment tests can detect fall risk and may induce interventions that prevent a fall. OBJECTIVES: To assess mobility of older persons using real time data and to compare these data with the mobility assessment of physiotherapists. METHODS: 20 older people aged 74±5 (mean ± SD) were monitored over 10 months to investigate the performance of an automated mobility tracker. Physiotherapists performed periodic mobility assessments. Annotated 3d recordings served as ground truth data. RESULTS: High correlation (r=0.684) of annotated and tracked gait speed was found. The mean absolute error is 0.16 m/s. CONCLUSION: 3D mobility trackers can be used to collect long-term mobility data. Since changes in mobility might indicate functional decline, long-term tracking allows to react to changes in mobility. Such a technology may have essential medical and social value.


Introduction
Population ageing [1] leads to an increased number of people with frailty, functional decline, immobility and falls. All these changes are indicators for an increased need in long-term care [2]. Consequently, the number of care beds, health workers as well as costs will increase [2]. For example, in Austria, the estimated average annual growth of care costs between 2015 and 2030 is estimated to lie between 4.4 and 6.2 percent [2]. Apart from economic issues, the demographic change leads to challenges concerning the quality of life in older persons and their caregivers and to impacts on the labour market [3]. Along with the general population, the nursing staff members are also ageing. In Germany, 40.6% of health workers were over the age of 50 years old in 2018 already † and in the US, one third of the nursing staff will be at retirement age within the next 10 to 15 years [4]. In Austria, the number of additional nursing staff needed is estimated to be 80.000 by 2050 [5]. Apart from the shortages in health care staff and the increasing number of people in higher age, almost all older persons prefer to stay at home as long as possible. The American Association of Retired Persons (AARP) reported that almost 80 percent of adults aged 50 and older want to remain in their communities and homes as they age [6]. Older persons however, are at an increased injury risk.
Falls are a major public health issue that can lead to physical injury, psychological harm or both [7,8]. In addition, falls can reduce the ability to live in the community. About 28-35% of persons aged 65 and older fall each year [9,10] and, the number of falls significantly increases with age [11]. The risk of future falls can be detected by a structured fall risk assessment that leads to interventions that reduce potential fall risk factors [12]. Several functional screening tests are available to measure mobility and subsequently identify elevated fall risk [13].
The results of such tests help to identify people at high risk for falling and normally trigger further assessment and ideally interventions.
Well-known assessment methods are the "Timed Up and Go Test (TUG)" where the subject stands up from a chair, walks 3 meters, turns around, walks back and sits down again, and the Gait Speed 3-meter test [14]. The time measured to perform the TUG test is highly associated with increased fall risk. Another validated parameter that correlates with fall risk is habitual gait speed [15]. Schwenk et al. [16] describe gait speed as the parameter with highest performance in distinguishing not only between frail and non-frail persons, but also between frail subgroups. The advantages of such tests are that they do not need a lot of time and can be applied in different settings such as primary care practices and hospital settings. However, a trained person is necessary to provide low inter or intra-rater variability. Another common observation is that older persons "tend to do their best" during clinical assessments, in order to dissimulate subtle functional deficits, while they perform differently when unobserved [17].
In this paper, we present a 10-month pilot field trial study using a non-wearable 3d sensor to automatically monitor body parameters for mobility assessments. This approach, based on an unobserved home-based assessment, is beneficial for reflecting the functional reality. Additionally, older persons can have access to assessments of fall risk despite a shortage of qualified staff. Using this approach is a novel way to monitor gait speed while ensuring user's privacy, minimizing obtrusiveness and thus, increasing user acceptance.

Related Work
In this section, we will review existing approaches to assess gait speed in order to contextualize our work. To date, several different methods have been available to assess gait parameters. One can distinguish between wearable and non-wearable devices [18]. While wearable sensors like watches, belts or shoes are placed at certain parts of the body, non-wearables are installed in a person's environment like floor sensors or cameras mounted on the wall [19] [20]. A wearable device for gait analysis is used by Parvaneh et al. [21] who present a chest-worn accelerometer-based sensor technology to identify frailty status. Based on the number of postural transitions during a day, like sit-to-stand or walk-to-stand, community-dwelling older adults are classified as pre-/non-frail or frail. Walk-to-stand and quick-sitting could be recognized as variables able to identify frailty status. An approach using multiple inertial sensors on different body parts (shanks, thighs, lower back) to discriminate between different frailty categories is described by Schwenk et al. [16]. Gait as well as balance parameters were separately used to identify the frailty status. Gait speed was identified as the most sensitive parameter for determining pre-frailty. Balance parameters did not lead to a clear frailty assessment. While this approach enables frailty assessment within an in-home assessment, supervision by professionals is required.
Although wearable systems have multiple advantages like being portable and lower in costs [22], they are also intrusive since the person wearing has to adjust daily routine to the sensor [23]. Another drawback is that wearable sensors are regarded as obtrusive, meaning that the effort to use and interact with the system might be higher than the user accepts [24]. In contrast with stationary devices, which are mostly equipped with a power cable, regularly charging a wearable device can be a disadvantage too [24]. Alshamaa et al. [25] propose a Doppler Radar sensor for in-home assessment of gait speed. The gait is separated in different zones that are automatically distinguished. The sensor is mounted on a chair, in front of which a person walks forth and back with three different paces. For validation of the radar system a motion capture system is used. The correlation between the reference system and the radar sensor results in 0.9788. Dolatabadi et al. [26] describe a way to bypass the drawbacks of wearable devices while also considering privacy aspects. They present an unobtrusive, machine learning method, where they use the Microsoft Kinect to discriminate between healthy and pathological gait patterns, separated in trunk, upper and lower limb. While a person is walking at different speed, the skeleton is tracked and two different ML approaches used to distinguish between healthy and non-healthy gait patterns. Data was collected in a clinical setting, where two Kinect sensors are installed in the opposite direction. A binary classifier then generates a label for multiple walking sequences of the person. The assignment of the label "healthy gait" or "pathological gait" reached a performance higher than 0.94. For people walking unsupervised misclassification was more likely than in people using walking aids or were supervised. The proposed method aims for applicability in smart homes able to be used with little effort.

EAI Endorsed Transactions on Pervasive Health and Technology
Online First 3d body tracking for automated mobility assessment in independently living older persons. Results of a pilot trial 3 Distinguishing between supervised and non-supervised assessment is another way of differentiating between technology for mobility assessment [27]. Used for passive observation, automated testing enables undistorted results, at habitual speed and without behaviour change of the participants [17]. One approach of an unsupervised gait assessment is the Mixed Reality-based assessment method utilizing a Microsoft HoloLens headset presented by Sun et al. [28]. This approach shall help to guide users through mobility assessment tests without direct clinical oversight. They automatically track the performance of the TUG and STS (Sit-to-Stand) tests and validate the results with inertial sensors. Comparing their approach to standard stopwatch measures, the completion time of the task calculated by the HoloLens highly agrees with the reference values.
Yagi et a. [29] instead use a simple RGB-camera to measure gait speed at home. With a neural network model, persons and their joints are detected. From this information as well as the floor position, step positions are determined, from which other gait parameters such as gait speed and stride length can be calculated. They validate the performance with a motion capture system and conclude that the error is small enough not to be clinically meaningful. Although RGB cameras are cheaper than depth cameras, they reveal a person's identity and can lead to low acceptance by the users if installed long-term. In case the system needs to be installed or switched on regularly for carrying out the assessment test, the user is more likely to feel like being in a test situation and not performing realistically. Furthermore, the effort for the user to set the system up is higher than a one-time installation.
While several authors used in home gait analyses using in depth sensors such as the Microsoft Kinect [26] or even tracked functional parameters such as the time to perform sit-to-stand tests [28], to date there are no data on longitudinal changes of functional parameters using nonintrusive 3D sensor technology.

Methods
In order to overcome the drawbacks of supervised mobility assessments and the need for wearable devices, we present a method to automatically detect gait parameters with an ambient sensor. We focus on the assessment of gait speed since this parameter correlates with frailty [15,18] and is part of standardized mobility assessments.
The development of the presented method and the corresponding user study were part of a project, where an exergame (games controlled by body movements) system was installed in order to analyse its potential influence on overall mobility. The methodology described in this paper aims to present how the automated tracker was developed and tested. In brief, gait and mobility data derived from the automated tracker were compared to standardized functional assessment tests carried out by trained physical therapists. Annotated 3d recordings serve as ground-truth data. In order to assess a person's mobility in a noninvasive way, a 3d sensor automatically measures mobility parameters that are part of standardized assessment methods.

Automated tracking
We used the Orbbec Astra 3d camera for person detection and movement tracking. The sensor was placed on the wall or at the ceiling during the entire time of the field trial, with no interaction from users or professionals required. The objective was to extract gait parameters from the detected movement of a person while carrying out the assessment tests. In a first step, the ground plane was spotted by analysing depth data only as described in [30]. In brief, this was done by classifying grouped points depending on their size as static objects. The first frame represented the background model. Frames were then periodically compared to the background model. The differences were detected as moving voxels. The number of connected pixels, height, density and shape of the object were used to compose feature vectors. These vectors were then classified by a Random Forest classifier which in a next step predicted, if an object was a person or not.
Analysing detected person regions of a frame together with the subsequent frame resulted in person tracks. The person tracks were then assessed by extracting walking patterns, as described by Kampel et al. [31]. In this step, the coordinates of the centre of mass of the tracked object were used as well as its velocity. The movement patterns were then analysed for walking velocity, distance and duration. Since only depth data was processed, no facial recognition was possible and therefore, full anonymity was provided. Moreover, the depth data was only processed locally.

Experimental study
A field trial was done over a period of 10 months in the private homes of 20 generally healthy older people living in Vienna, Austria (05/2018 -03/2019). This trial was carried out in order to test the automated tracking-method in a realistic environment. Participants were selected with help of the respective end-user organisations within the project. About 10 endusers should be aged 65-74 years and the other 10 75 years or older. The amount of 20 study participants was regarded as adequate in terms of assessing the performance of the automated tracker for persons of different height, size and movement patterns.

EAI Endorsed Transactions on Pervasive Health and Technology
Online First For each subject, a trained physical therapist manually conducted GS tests. Therefore, the participant walks a pre-defined distance while the physical therapist measures the time it takes the participant to complete this task. The distance used in this study was 3 metres, although this is not significant since the test could have also been conducted with another distance. Four physical therapists were involved in the trial. They were instructed to apply the gait analyses based on current recommendation and used a digital stopwatch. Each participant was asked to do three repetitions per test in order to screen possible changes in regards to the mobility of the test users. The 3 GS tests were performed between two and five times per participant within a time between 3 and 12 months. The number of measurements per PT (Figure 2) includes tests of users who later dropped out. At the same time the physical therapists carried out the GS tests, the automated tracker measured gait velocity and test duration. To be able to compare the data derived from the manual method with the automated tracker we defined ground truth data. This was again generated by the 3d depth sensor. During the PT assessments, 3d data were permanently recorded. Each recorded sequence was then watched frame by frame and the start as well as the end of an assessment were manually labelled. Thus, the exact time for completing the task could be determined. Together with the fixed walking distance, this led to a highly accurate gait speed value which was used as reference.
The recorded sequence was also used for automatic tracking. Hence, it was possible to directly compare the tracking algorithm performance to the manual annotation and the physiotherapist's measurement (see Figure 1).

Figure 1.
Relationship between a recorded sequence, the annotation and the tracked walks. The sequence contains all recorded tests. The manual annotation labels starting and endpoint. Tracks are detected movements, while walks are the gait tracks that are detected within the tests.
The time measured manually by PT and the results of the automatically tracked time were compared with the annotated ground truth data. We observed the differences between the instructing physical therapists by calculating the Mean Average Error (MAE) and the relative error. The performance of the tracking algorithm was obtained by calculating the error between the annotated and the corresponding tracked data for each available data point.
Habitual GS as a validated parameter to determine people's risk of falling was used for comparison. This measurement can be automatically carried out without the need of a special test set up at home.

Statistical Analysis
Each participant was assigned to an individual anonymous identifier. Gait speed is defined in m/sec. Data are expressed as means and standard deviation. We applied linear regression analysis to assess the associations of manual (PT) with machine based gait data. Data were entered into an Excel spreadsheet and calculated within Excel.

Results
In the beginning of the trial, a total of 20 persons was recruited, 15 females and 5 males with a mean age of 74.1±5.9 years. Table 1 shows the full demographic profile of the initial study population. Seven participants (six women, one man) decided to leave the project. Reasons for leaving the study were scheduled surgery or different expectations. Six new participants could be recruited during the ongoing study to replace the participants who had dropped out. The mean observation period was 9.7 months. Due to drop-outs and the replacement of participants, for a total of 18 users, annotated GS data of at least two time points could be collected. For each participant in the study, an identifier is created, which is used for documenting study data in an anonymized manner ( Figure 5). The mean number of GS assessments was 2.2.

Manual gait speed assessment
Four PT performed a total of 78 gait analyses. Compared to the ground truth, we observed deviations between 0.01s (0.31%) and 3.25s (44.98%) (Figure 2).

Tracking system
The mean average error (MAE) of the GS duration measured by the automated tracker was 0.928s while the MAE of the GS was 0.153m/s. Using linear regression, we found a significant positive correlation of tracked versus annotated (ground truth) assessment of gait speed (Figure 3).

Comparison of manual versus tracked gait speed
The manual measurements of the physical therapists reached a higher accuracy than the automated tracker. The mean deviation in measured GS duration between those two methods is 1.4 seconds. Figure 4 shows a direct comparison between the automated tracker, the physical therapist measurements and the ground truth data concerning the obtained GS duration. The ground truth illustrates the reference value, which enables to make statements about the performance of the manual measurements carried out by physical therapists and the tracker. Additionally, the error of the tracker as well as of the manual measurements are visualized. The tracker and the manual measurements follow the trend of the ground truth data, which allows the tracker to monitor the mobility of older people. The number of the data points in Figure 4 derives from the amount of directly comparable data between all of the three methods. This means that it some cases, only data from one or two measurement methods were available, limiting the comparable amount of data to 59 data points.

Individual Longitudinal Gait Speed Changes
Over the (9 month) period of the trial, regular mobility assessments conducted by the three different methods enabled us to track the changes in the GS tests over time. Six users participated in less than two tests and hence are not listed. Compared with t0, eleven participants show increased gait speed at the end of the trial, while it decreased in seven. Among all participants, the change in gait speed ranged between -25.26% and +47.62%.

Discussion
In our long-term observation comparing tracker-based versus physiotherapist-based mobility assessments we found that the deviation between the measured GS duration and the annotated data varied by physiotherapist.
The mainly positive bias shows that human timing takes longer than the actual value which is most likely due to the reaction time of the physical therapist and of the test person. We also demonstrated that the measurements of physiotherapists were more accurate than the data collected by the tracker. Still, the tracker correlates positively with the annotated ground truth data. Both the Kinect-based method from Ejupi et al. [27] and the Mixed Reality-based approach with the Microsoft HoloLens [28] deliver mobility data in agreement with the supervised assessments, but in contrast to our passive approach, the user is guided either by visual or auditory test instructions. This leads to more effort for the user on the one hand, and to test situations which could lead to unrealistic results on the other hand. Stone et al. carried out similar studies than we did [32] [33], using the Microsoft Kinect sensor to compare the    [34], similar tests have been carried out at senior centres, in a supervised manner. Both [32] and [34] can map the results between both tests, but they cannot determine their validity since ground-truth data is not gathered. Staranowicz et al. [35] assess fall risk of a person by integrating the Kinect on a mobile robot, which follows the participant at home and collects gait parameters. The results were compared to the Vicon motion-capture system. The question of obtrusiveness and user acceptance of this technology needs to be further addressed.
In our longitudinal field test, we could demonstrate that GS varied between individuals and over time. While some persons suffered a significant decline, others improved by almost 50%. The differences in improvement can be explained by an exergame study carried out in parallel to the assessment tests. Participants using the provided games frequently showed on average better results in the GS test than infrequent players. Negative trends can be explained by less movement or a fall occurred during the trial phase. Due to the test setting, however, we were not able to provide physical activity diaries for our participants. In addition, our results could have been influenced by a recruitment bias. Only very motivated persons consented to participate in the study. We conclude that the presented tracking system is able to monitor the mobility of a person via GS. This data can be used to observe a mobility trend, and therefore, health staff can intervene if required. Especially in people with high risk of falling, assessing risk factors such as low gait speed and reacting to them reduces the likelihood of falls [12]. Reacting to changes in mobility allows physiotherapists to save valuable time for patients who actually require medical care.
Additionally, the mobility of a higher number of persons can be observed than with physiotherapists carrying out regular assessment tests in people's homes. The prevention of further mobility decline and falls also decreases medical costs. In 2015, the estimated care costs referred to falls in the US were approximately 50 billion US dollars [36]. Due to the ageing population, the costs are assumed to be rising [36].
Furthermore, the tracking method allows objective and unbiased mobility values. While recording 3d depth data, some walks could not be completely recorded. These limitations are based on the different apartments of the persons, which are partly narrow. Thus, correct system positioning during the installation was required to ensure proper person and GS detection. However, it was essential that the participants could carry out the tests in their own homes to keep the effect of the test situation and the stress to a minimum.
Based on the findings of the study, we highlight the benefit of the automated tracker for people with either decreased mobility, high risk of falling or people who want to monitor their mobility and prevent mobility decline at an early stage. The prevention approach will also help to relieve the healthcare system and enable elderly people to stay at home as long as possible.

Conclusion
A tracking system was presented to automatically monitor the mobility of a person by 3d depth data. In order to show its performance, assessment tests were carried out in a 10-month field trial with 20 older people. The 3d sensor used for the tracker was also used to gain ground truth data, which was then compared to the results measured by physical therapists and the automated tracking system. It has been shown that the presented tracker is able to monitor mobility changes in a passive manner, which allows to prevent falls and to enable target-oriented care work. As a next step, we suggest to investigate how valuable the tracking system is seen by physiotherapists in terms of productivity and support at work. Furthermore, additional gait parameters apart from gait speed will be raised that are useful for long-term monitoring.