Estimation and Tracking of Knee Angle Trajectory using Inertial Sensors and a Smartphone Application

We demonstrate a practical solution to track the 2-D knee angle trajectory of a user in real-time using inertial sensors embedded in a wearable knee-band wirelessly coupled with a mobile device, by mathematically modeling the knee as a hinge joint. We show how the bias of gyroscopes can be automatically computed to eliminate manual calibration and combined with a computationally simple Kalman filter model for tracking the knee-angle on a powerconstrained mobile device. Finally, we visualize the 2-D knee motion trajectory in real-time using an avatar on an Android phone.


INTRODUCTION
Continuous motion trajectory tracking of the human body is an emerging frontier of research that has applications in healthcare, sports medicine, rehabilitation and yoga [1].Advances in miniature and wearable inertial sensors have made it possible to quantify, track and characterize body motion in everyday ambulatory settings.In particular, gait analysis is used to treat individuals with medical conditions affecting their ability to walk [2].In sports biomechanics applications, gait analysis improves limb movements of athletes to optimize and enhance their performance.
There are many advantages of using inertial sensors to track human motion compared to a camera-based optical marker system [3].The miniature size of inertial measurement units (IMUs) enables measurement of full body kinematics by placing them at different joints and limbs, without causing hindrance to routine living.Inertial sensors do not require controlled laboratories like camera-based systems and can be deployed in real-world environments, thereby facilitating routine motion monitoring.Finally, inertial sensors consume low power which enables continuous motion tracking and monitoring for longer durations.
One promising application is continuous tracking of knee-joint motion and trajectory for rehabilitation.Consider a patient who is prescribed a sequence of exercise routines to rehabilitate a diseased knee.Therapy requires constant supervision by a physiotherapist which is not a practical long-term solution and is also infeasible in the home.However, using wearable inertial sensors coupled with a personal mobile device, the patient can track his motion in a home-setting, get continuous feedback on his performance, and regularly benchmark practice against a prescribed exercise regimen without therapist intervention.Such a system can also provide corrective feedback in real-time and can help restore ambulation for patients recovering from fractures and strokes [4].

PRIOR WORK
Prior studies have investigated knee angle tracking using inertial sensors assuming rigid-body motion for thigh and shank under conditions of 2-D and 3-D motion.Favre et.al. [5] integrated angular velocity data obtained from gyroscopes, and corrected the knee-angle estimation based on inclination data obtained from accelerometers gathered during rest or the constant-velocity motion period.Liu et.al [6] proposed a double sensor difference algorithm which requires two accelerometers mounted on a rigid board for one segment.Using data from accelerometers and magnetometers, they perform a 3D orientation study of the lower limb segments.Bennet et.al [7] proposed an extended Kalman filter model to calculate the walking distance and gait parameters by measuring knee angle using gyroscopes and accelerometer.Watanabe et.al. [8] use an error-state complimentary Kalman Filter to combine measurements from accelerometer and gyroscope to calculate an accurate estimate of the knee flexion angle.Sen Qiu et.al. [9] use triaxial accelerometer, magnetometer and angular rate sensors to calculate orientation quaternion of the shank to compute its 3D orientation while walking and transmit this data to an XBUS logger for data aggregation and communication to pc via Bluetooth.Liu et.al [10] propose a gait parameter measurement and a phase detection system.This approach uses data from thigh, shank and foot mounted inertial sensors and an 8 channel data recorder for logging and motion feature extraction algorithms on the PC.All these studies require a manual procedure for gyro bias estimation for input to the Kalman Filter.For example, Bennet et.al [7] used an average of gyroscope readings under static conditions to estimate the bias for an extended Kalman filter model to calculate knee motion parameters.Estimation of bias for a gyroscope is challenging, especially for low-cost sensors.Usually the user must manually calibrate the gyroscope by running static experiments, making the solution obtrusive and unattractive for routine use.Moreover, in a full-body motion-tracking wearable system, the user is instrumented with many IMUs, each with a tri-axial gyrorendering manual bias correction for multiple IMUs infeasible.A usable knee-trajectory tracking system must be autonomous and should not entail any manual calibration requirements.Prior art knee motion tracking solutions are not a good fit for ambulatory and automatic real-time trajectory tracking of body motion on a personal mobile device.They are either computationally demanding or require obtrusive user-calibration of the gyroscopes [9], or require a special data logger device with WiFi for data transmission and post-processing [9,10].These data loggers and aggregators [9,10] are not viable for ambulatory tracking as they are cumbersome, unsuited for real-time feedback and expensive.
Although automated systems for real-time knee motion tracking using a personal mobile device are attractive, stringent constraints need to be met to realize a practical embodiment.These include a user-centric design with no manual calibration for IMU bias, a compact Kalman Filter implementation for power efficiency, real time data transmission and processing, efficient computation on a mobile device (e.g.phone or tablet), and rendering the knee motion trajectory in real-time on an avatar.

CURRENT WORK
In the present work, we demonstrate a user-friendly solution to track 2-D knee motion in real time on a mobile phone using IMUs.Our approach consists of: (i) a gyro bias estimation algorithm that is automatic and requires no user input whatsoever, (ii) a simplified Kalman filter for a mobile implementation that is ~2X faster than the conventional Kalman filter approach and (iii) an android application on a smartphone to track and visualize user's 2-D knee motion in real time.We can thus demonstrate a practical embodiment of a real-time knee-angle tracking solution on a mobile phone using wearable inertial sensors over Bluetooth Low-Energy.

Wearable System Configuration
For knee motion tracking, we capture the movements of the thigh and the shank independently, assuming each as a rigid body element.We instrument each segment, i.e. the thigh and shank, with an IMU as shown in Fig. 1.We also fabricated a conformal band that snugly wraps around the knee to resist motion artifacts.The band houses two IMU nodes as shown in Fig. 1.The thigh sensor node has a microcontroller and a battery that also supplies power to the shank sensor node, which is connected via a flexible interconnect.Each sensor node has an InvenSense® MPU-9150 TM inertial measurement unit that consists of a tri-axis accelerometer, gyroscope, and magnetometer.We set the digital-output accelerometer to provide a full scale range of ±2g.We set the digital-output gyroscope to give a full scale range of ±250 deg/sec.Since we restrict the current study to 2D motion tracking, we do not use the magnetometer.The data from both IMUs is sampled by the microcontroller using the I2C interface every 50ms i.e. the sampling frequency is 20Hz.The microcontroller transmits the measured data to an android tablet/phone in realtime over a Bluetooth low energy (BLE) radio link.We evaluate our knee-motion tracking solution for walking, flexion and cycling activities performed by three users (N=3).We use a mechanical jig with a wooden robotic arm for ground-truth measurements and validation.

Mathematical Model for Knee Angle Estimation
To enable accurate tracking of knee motion, we must describe several parameters that relate to the instantaneous orientation of the user's limbs.An efficient mathematical model that relates these kinematic parameters to the measurements from the inertial sensors is the primary contribution of this work.
We use the instantaneous joint angles as motion parameters.Combining these angles with information about user's segment (thigh and shank) lengths, it is possible to extract rich gait parameters such as step length, stride length etc.In this study we limit our scope to estimating the joint angles using measurements from inertial sensors.However, this task is not trivial as inertial sensors are prone to random and systematic errors which are reflected in the parameters derived from them.Hence, our mathematical model must accurately predict joint angles in the face of this measurement uncertainty.
The random noise of the accelerometers contributes to jitter in the angle estimates and the non-zero bias of the gyroscope leads to integration drift which restricts accurate estimation of angles to barely few seconds.Thus, we need a sensor fusion algorithm, based on an underlying mathematical model for the system dynamics and measurement error, which can mitigate the measurement uncertainty of the accelerometer and gyroscope (i.e.noise and bias), and at the same time harness their complimentary properties (i.e.long-term and short-term stabilities respectively) to advantage so as to produce accurate motion parameters.One contribution of this work is a Kalman filter model for knee-angle estimation in our wearable system configuration.The dynamics of sensors are used to determine suitable process and measurement model equations that relate the unknown state (i.e. the angles) to the measurements.Related studies have explicitly incorporated a bias variable as part of the Kalman state estimation vector because the gyro bias can be dynamically estimated as part of the Kalman filter.When this traditional approach to estimating bias is applied to the knee-angle estimation problem, it introduces a usability problem since gyro bias is calibrated manually a-priori.A second contribution of this work is a technique to dispense with the in-run bias estimation by pre-processing the sensor readings.This leads to a more efficient and usable implementation.
In the following section, we develop a conventional Kalman filter model for knee angle prediction for our wearable system configuration.We then describe a modified Kalman filter formulation which accurately estimates knee angle without including any bias terms in the state vector.This modified filter is simpler to implement, provides good accuracy and is well suited for a mobile implementation.

Conventional Kalman Filter Model with bias variable in state vector
We assume rigid-body motion of the knee and restrict this motion to the plane of walking only i.e. the sagittal plane.We ignore the coronal and transverse plane motions i.e. abduction/adduction and internal/external rotations since they do not significantly affect the knee flexion angle.We model the knee as a two-link revolute, i.e. a hinge joint with flexion as the dominant motion The instantaneous flexion angle between thigh and shank during dynamic motion is calculated as the internal angle ∠ABC (180 −  2 ) in Fig. 2. To estimate  2 , we need to first obtain angles  1 , which represents angle between thigh and vertical, and angle , which represents angle between shank and vertical.The relationship between these angles can be geometrically deduced as below: In a conventional Kalman filter, the thigh angle  1 is first predicted using thigh gyroscope readings (location S1) and corrected by corresponding accelerometer readings.This approach enables one to take advantage of the long term stability of the accelerometer and the short term stability of the gyroscope.
A gyroscope sensor reports instantaneous angular rate of rotation about its sensitive axes.To find the absolute angle in any rotation, the gyro readings are integrated over the duration in which the rotation occurs from an initial reference orientation (2D motion).However, gyroscopes are affected by a bias error i.e. they give a non-zero rate even when stationary.As a result, the knee angles calculated using bias-prone gyroscope measurements are erroneous and increase linearly without bound, a phenomenon referred to as integration drift.Unless the bias is eliminated, the gyroscope cannot estimate knee angles accurately.Since the bias is a systematic error, a commonly used technique to estimate the bias is to sample several measurements over a stationary period and take their average.

Figure 2 Schematic of the knee modeled as a hinge joint
Typically, the gyroscope is left stationary for a short duration, say a 20-second period (i.e.number of samples at 20Hz sampling = 400).The measured samples are averaged to give the bias value as in equation ( 2) below wherein   is the i th sample of angular rate measured by the gyroscope: This static bias estimation process assumes that the bias remains constant over the run of the experiment and does not include drifts in bias caused due to thermal issues, random walk and other bias instability errors.The conventional Kalman Filter recursively corrects the gyro-bias errors by fusing it with an independent estimate of the knee angle derived from accelerometer measurements.The gyro bias is included as a separate variable in the conventional Kalman model and is updated at every time step of the experiment to account for major drifts.The Kalman filter recursively calculates the best estimate of the bias at each time step and computes the optimal trajectory that has the least error.
The inclusion of the bias variable in the state vector is required for high accuracy trajectory tracking seen in many applications such as satellite, missile tracking.Since gyroscope bias values change from run to run, the bias estimation must be done every time before a user begins to track her motion i.e. before the knee angles can begin to be computed.This manual calibration is impractical for everyday use.In section 3.2.2,we demonstrate an automatic bias estimation algorithm that closes this gap.

Calculating thigh angle 𝜃 1
We implement two Kalman Filters, one to estimate the thigh angle  1 and second, to estimate knee angle  2 .We calculate  1 by fusing the thigh angles computed independently from gyroscope and accelerometer sensors.In Fig. 2, the gyro mounted in location S1 provides angular rate 1 and measures the rotation of the thigh about hip joint A. The sensitive axis of the gyro at location S1 is the one that is perpendicular to the plane of rotation i.e. its  axis.By convention we assume counterclockwise rotations as positive.The gyroscope measured angular rate, 1 can be written as a sum of the true angular rate of the thigh 1 and a bias value specific to the gyroscope i.e.  1 (3).
We integrate 1 to calculate  1 as in (4): Substituting the value of 1 from (3) in (4): We implement integration procedure (5) in a discrete-time Kalman Filter [19-20] described later in equations (9-10).To obtain an independent estimate of thigh angle  1 , we use the accelerometer as an inclinometer/tilt sensor [11].Note that the accelerometer can be used as a tilt sensor only under the assumption of slow physical motion of the associated limb i.e. when the net magnitude of motion contributed acceleration is negligible.Referring to Fig. 2, note that  1 and  1 are the normal and tangential accelerations (with reference to thigh) reported by the accelerometer at location S1.These readings are relative to the sensor frame, i.e. the local reference frame that supports the inertial sensor.We calculate the angle  1  , defined as the thigh angle using an inverse tangent function (6): The thigh angle  1  is prone to noise because of the susceptibility of the accelerometer to impact and jitter.To eliminate this noise, we fuse the knee angle estimate from the accelerometer with that of the gyroscope (5) via a Kalman Filter.Consider the generalized state space representation of the Kalman Filter (7): where  is the expectation operator and  is the Kronecker Dirac Delta Function   = 1 when  = , else   = 0.  is the processnoise covariance matrix and  is the measurement noise covariance matrix.The algorithm of the generalized Kalman Filter is shown in Fig. 3 [12].
We formulate the discrete time version of equations (3-5) that relate thigh angles  1 , thigh gyroscope angular rate 1 and bias  1 below in equation (9).The time steps  + 1 and  refer to the current and previous values of time step where  ∈ ℕ :

Figure 3 Kalman Filter algorithm
We rewrite above equations as a state space representation (10): This (eq.10) defines the process model for the Kalman Filter to determine  1 .Separately, we derive a measurement model (see eq. 7) for the knee angle  1  using the accelerometer measurements.Based on (6), we can write: Formulating equation ( 11) using the generic measurement model in (7), we find that: We initialize error covariance matrix (12)  0 =  *  2 where  2 the 2 x 2 identity matrix and  is a large number (1000 for our case).We choose this value empirically so that the Kalman filter iterations start with a high measurement uncertainty and converge towards reduced uncertainty as  gets recursively updated by the algorithm. = 1000 0 0 1000 The off-diagonal terms of the error covariance matrix are zero as the state variables  1 and  1 are uncorrelated.Likewise, the process noise covariance matrix  (13) is a 2 x 2 block diagonal matrix [12-13]: The parameters   and   are r.m.s noise parameters of angle estimate using gyroscope and are empirically determined based on the noise susceptibility of  on gyroscope angular rates.We selected and tuned these parameters for least error in our models [12][13].Likewise, the measurement noise covariance matrix  in the generalized Kalman Filter in Fig. 3 represents the root mean square (r.m.s) of the noise in angles predicted using the accelerometer.Based on the measurement model, the  matrix contains a single element   .Its value is the variance of the angles calculated using raw accelerometer measurements which, in our experiments was found to be to be 10 -3 .

Calculating knee angle 𝜃 2
We likewise estimate  2 by fusing the gyroscope and accelerometer readings via a Kalman Filter.Let 2 be the angular rate reported by the gyroscope on the shank at location S2.In this case, the gyro at location S2 reports the sum of the actual angular rates of the shank 2 and thigh 1.Furthermore, the shank gyro is corrupted by its own bias value  2 .Thus, we can express the sensor reported angular rate as (14): We integrate 2 to calculate  2 as in (15): Substituting the value of 1 from ( 3) in ( 16) we have (17): The discrete time implementation of ( 17) is expressed as: We cast equation ( 19) in the state-space representation to comply with the Kalman filter model ( 7) below: This completes the process model for  2 .We now derive the measurement model for  2 .We do not have a direct estimate of  2 from the accelerometer at location S2 because  2 does not represent the tilt of the shank with respect to the vertical.The tilt of the shank with the vertical is denoted as  (Fig. 2) and is obtained in equation ( 21) from accelerometer readings  2 and  2 at location S2 which represent normal and tangential accelerations (with reference to shank).
Thus, we obtain an independent estimate of angle  2 using  1 and  from the accelerometer.Referring to the Kalman Filter equation ( 7), we can see that: The error covariance matrix, the process covariance matrix and measurement covariance matrix are initialized with the same values as in (12) and (13).
The conventional Kalman filter described above assumes that the bias is known a-priori, and uses it within the Kalman algorithm.This estimation procedure has the disadvantage of being userunfriendly because it requires a manual calibration to determine the bias prior to use in the wearable system.In the next section, we overcome this limitation.

Kalman filter with automatic de-biasing
We formulate an alternate Kalman filter model in which we remove the gyro-bias variable from the state vector in Kalman filter.We automatically compute the gyro bias and subtract this estimate from the gyroscope measurements prior to using them in the Kalman filter.This approach reduces the dimension and computational complexity of Kalman Filter implementation and is a better fit for continuous trajectory tracking using constrained battery-powered device such as a phone.Despite the preprocessing overhead for de-biasing along with a simplified Kalman filter, the overall computational complexity is less than the conventional Kalman method, as we will demonstrate later.
Fig. 4 shows our automatic gyro bias estimation algorithm.It assumes that the bias is dominated by a static component (constant term) and variations due to drift and random walk errors are relatively small.This assumption would be appropriate in practice for non-critical daily tracking applications.We then approximate the bias using the DC component (constant term) of the Fourier transform of the gyroscope measurement under arbitrary activity.When the user initiates an activity, our algorithm uses the first 1000 gyroscope samples to estimate the bias.Every sample of the angular rates acquired after this window of 1000 samples is used for angle estimation in the Kalman filter.
For longer duration activities, we envision that the bias may need to be automatically re-estimated periodically to compensate for any drift errors.Since the bias is computed in real time, we debias the gyro signal by subtracting the DC component from the raw measurements acquired subsequently.We have empirically validated our assumption on the nature of bias by demonstrating that our algorithm is able to predict knee angle trajectories accurately.

Figure 4 FFT based automatic gyroscope de-biasing algorithm
Using the de-biased gyroscope signal, we then formulate a Kalman filter that directly uses a clean gyro signal as input without any bias term in the state vector.The dimension of the state vector is now unity instead of two (in a conventional model), and the corresponding state transition, error covariance and process noise covariance matrices are of size 1x1 instead 2x2.The state estimation equations for the thigh angle are as follows: In this model, the state vector is  =  1 , the state transition matrix is  = 1, the control input matrix is  = , the control input vector is  = 1, the measurement vector is  =  1 and the measurement/observation matrix is  = 1.Similarly, we formulate the shank angle model as follows: The state transition matrix , the control input matrix  and the measurement matrix  are same as for  1 above.The state vector is  =  2  , the control input vector is  = 2 − 1 and the measurement vector is  =  2 .
This simplified Kalman Filter works for the auto de-biasing scheme because the actual bias value is always close to the in-run bias.Since the in-run bias is not updated at every time step in this approach (in contrast to the conventional Kalman Filter formulation, see section 3.2.1), the computed trajectory may not be optimal (in the Kalman sense) and will depend on the extent to which the in-run bias diverges from the actual bias.This approach, however, has that additional advantage that it allows periodic re-computation of the current in-run bias in real-time to correct drifts.If we were to instead use the static bias estimate (eq.2) in this simplified Kalman filter formulation, the resultant angle estimate would progressively drift from reality.Furthermore, the use of static bias is impractical because the bias value cannot be dynamically updated on-the-fly in the middle of an activity.

RESULTS AND DISCUSSIONS
We first examine whether the bias can be approximated by the DC component of the Fourier transform of real-time gyroscope measurements for a user-friendly motion tracking design.Next, we assess the efficacy of our simplified Kalman filter model in comparison to the conventional Kalman filter for walking, flexion and cycling activities.Finally, we demonstrate automatic knee trajectory tracking using an avatar on a mobile phone use our modified Kalman filter algorithm.
To determine if our FFT algorithm is effective for gyro-bias estimation, we use static calibration bias as a reference (eq.2).To assess whether the bias value is sensitive to activity type, we compute the bias for the three activities.To determine the right sampling window size for a reliable and consistent bias estimate, we also assessed the behavior of bias for different window sizes.In Fig. 5, the x-axis is the sample window size and the y-axis is the predicted bias estimate using the FFT method proposed in Section 3.2.2.The orange, blue and purple curves indicate FFT estimated bias values for flexion, walking and cycling activities.
The static bias estimated using the standard averaging method is ~0.81deg/s which is indicated by green horizontal line in Fig. 5.
For a sampling window size greater than ~1000 samples, the bias estimate remains practically unchanged across the three activities.
The bias for a 1000 sample window is within 5% of the static bias value.This result suggests that a good enough estimate of the bias can be calculated in real-time paving the way to a much improved user experience (since no manual calibration is required).At 20Hz sampling frequency, the 1000 sample window size corresponds to 50 seconds of measurement time, implying that the trajectory of the user should be tracked continuously 50s after the activity is initiated to automatically ascertain the gyro-bias.We now evaluate the effectiveness of our simplified Kalman filter formulation by comparing against the conventional Kalman filter model as a baseline.Fig. 6 shows the results of predicted knee angle trajectories for cycling, flexion and walking activities for a user.The graphs in Fig. 6 show the flexion knee angle i.e. (180 −  2 in Fig. 3) against time for the simplified Kalman filter as well as for the convention Kalman filter.The static gyro bias value for the conventional model (refer to eq. 2) are 0.83 deg/s and -0.5 deg/s for the thigh and shank IMUs respectively.
The bias values automatically estimated by the FFT algorithm are 0.94 deg/s and -0.55 deg/s.For flexion and cycling, the angle predicted by our modified Kalman filter has a maximum-error of just 1.8 degrees relative to the conventional Kalman filter.For walking, the maximum error for the predicted angle is 3.7 deg.
For vigorous activities, we expect this error to increase due to higher motion artifact in dynamic movements and increased sensor movement relative to the thigh/shank.For this set of experiments, our modified Kalman Filter model predicts the knee angle trajectory well and is an especially good fit for full-body tracking solutions that use multiple gyroscopes each needing calibration for bias.To quantify the computational benefits of our simplified Kalman filter model, we estimated the execution time in MATLAB (Table 1) and found that the simplified model runs ~2X faster than the conventional model without any significant loss of accuracy even when taking into consideration the de-biasing logic in the simplified Kalman filter algorithm.This demonstrates how knee angle can be tracked in real-time without any user calibration while simultaneously reducing computational load, enhancing battery life on a mobile device.Finally, we demonstrate automatic knee trajectory visualization via an avatar animation on a mobile-phone.Using the motion parameters estimated by the state estimation algorithm, we created an animation of the lower limb of the user in the sagittal plane.Figure 8 shows an avatar that tracks the knee-angle in real-time.
To ensure a smooth rendering of the user's motion, the avatar gets updated every 50ms on the mobile phone.This work has the following limitations: Our mathematical model is based on the assumption that the knee can be modeled as a perfect hinge joint, which, in reality, is not the case.Further, this model assumes that the anatomical axes and the axes of the sensor frame are perfectly aligned, which is not true.We have not quantified the errors resulting from such approximations.This requires the use of an optoelectronic camera based system as an independent ground truth to quantify the accuracy of our algorithms and calibration to account for the misalignment of sensor and anatomical axes.Further, this approach does not readily scale to other joints such as the shoulder because of these assumptions.We have not evaluated the suitability our technique for noisy signals under highly dynamic conditions such as those in sports monitoring applications.

CONCLUSIONS
We have demonstrated a practical embodiment of a real-time knee motion tracking solution on a phone using a smart wearable knee band instrumented inertial sensors.Our approach offers a user friendly alternative to a complex camera based motion tracking system.We achieve this by combining an automatic gyro debiasing technique with a computationally efficient Kalman Filter algorithm.

Figure 1
Figure 1 Wearable System Configuration

Fig. 2
Fig. 2 is a schematic representation of a user's lower limb and its stick model equivalent.In the figure, the line segment A-S1-B-S2-C is the geometric representation of the instantaneous orientation of the user's lower limb.The segments A-S1-B and B-S2-C represent the thigh and shank respectively.Point A represents the fixed hip joint and point B is the patella.Locations S1 and S2 represent the inertial sensor nodes on the thigh and shank respectively.Conceptually, segment S1-B-S2 represents the wearable knee band embedded with the inertial sensors at coordinates S1 and S2.

Figure 5 Figure 6
Figure 5 Sensitivity of estimated bias to sample window size

Figure 5
Figure 5 Snapshots of a user's leg and simultaneous motion tracking by our application.