Machine Learning approaches for automated gait analysis
Keywords: Balance assessment, computer vision, functional assessment, machine learning
Overview of Research
Gait disorders contribute to major health-related consequences including falls, injuries, fear of falling and immobility, and sometimes mortality. Gait analysis is the study of the trajectories of body parts over time and is essential for diagnosis of gait disorders. These analyses help clinicians examine the effectiveness of therapeutic interventions; and help healthcare providers make provisions to support those interventions. However, clinicians are confronted with several issues in the existing gait analysis tools which can be alleviated through involvement of ubiquitous technologies, unobtrusive sensors, and ML approaches. These three strategies are useful aids which make the analysis (1) reachable anywhere and anytime (e.g. a person’s home), (2) accommodate complex movement of body muscles and joints (i.e. high dimensionality, nonlinearity and temporal dependency) and (3) easy to use by both clinician and patients.
The primary objective of this research is therefore to enhance the traditional gait analysis tools through developing an automated gait analysis approach which benefits from being intelligent, portable, cost effective, markerless and which can be easily integrated into individuals' everyday environments.
In general, the gait monitoring and diagnostic tools are potentially appropriate for frequent gait analysis in the home, i.e., without the need to visit a specialized gait clinic. It enables automated capture and analysis of gait for longitudinal monitoring.
Contributions and Novelty
In order to address our primary objective, following contributions were made:
Human Gait Acquisition
Depth sensor was used as the primary source of human gait acquisition given its low-cost, ubiquitous and markerless nature.
- A skeletal tracking application based on the Kinect sensor SDK was developed and used to detect, track, and record the human pose and motion for post-analysis (link).
- A number of pre-processing methods were presented to clean-up and extract useful gait information from the recorded Kinect streams including the spatiotemporal gait parameters and high dimensional hierarchical joint orientations (link).
- An engineering set up of the Kinect sensor in a hospital clinic and home environment is shown and used to collect gait data during routine exams with a sizable number of participants (link).
Human Gait Modelling
Learning models of human motion is a difficult task due to the high dimensionality, non-linear dynamics and stochastic nature of human movement. Machine Learning methods have proven to be a promising approach in handling the complexities of the human gait pattern. Our research capitalizes on state of the art, machine learning approaches for the following applications:
- to track progress of gait recovery (link).
- to diagnose a gait disorder (link).
- to predict the risk of falls based on patterns of gait.
- Gaussian Mixture Model with varying Covariance parameters to track progress of gait recovery (link)
- Dynamical Gaussian Process threshold based model to classify high-dimensional time series gait sequences (link)
- Dynamic Fall risk prediction from gait patterns using Machine Learning
Our machine learning solution discovers quantitative improvement of gait over time using unsupervised Latent Variable modeling. Gaussian Mixture Model with varying Covariance parameters were developed to automatically categorizes multidimensional gait data into different groups based on inherent similarities among subjects. It also generates composite indices equal to the number of groups.
This approach partitions the gait patterns of poststroke individuals into three groups. Three sets of gait features including symmetry ratio of swing time, stance time, and step length, velocity and variability in step length and step time, vary among the generated groups. Our proposed approach ensures that one of the group center is the control (healthy) group center. Moving from the center of this group to the center of other detected groups will therefore deteriorate symmetry, velocity and variability measures while the reverse improves them.
Figure 1. Using the Gaussian Mixture Model with varying Covariance parameters approach, the data were partitioned into three groups. The mixture components are shown as elliptical contours. The markers (circles) in (a) represent spatiotemporal gait observations belonging to training set. The markers vary in color from red to green to blue as the membership index changes from Group 1 to Group 2 and 3 respectively. In (c) the spatiotemporal gait belonging to testset (3 participants) are represented by triangles, diamonds and squares, respectively. For each participant, an arrow is drawn between each pair of spatiotemporal measures with the direction from Time point 1 to Time point 2 and from Time point 2 to Time point 3.
Besides differentiating gait pathology, this approach allows the gait observations to belong to all estimated groups simultaneously with a different degree of membership, which can be used as an outcome measure. This capability makes the composite gait membership index a good candidate as an additional outcome measure in longitudinal gait studies where tracking changes in motor recovery is of interest.
Publication:
Dolatabadi, Elham, et al. "Mixture-model clustering of pathological gait patterns." IEEE journal of biomedical and health informatics 21.5 (2017): 1297-1305 (link).
Some gait abnormalities are obvious, while others are hardly noticeable. Early diagnosis of gait abnormality, specifically in older adult populations, prompt early interventions, which can help prevent dysfunction and subsequent loss of independence. Human gait exhibits complex and rich dynamic behavior. It is therefore useful to explore Machine Learning methods to incorporate dynamics of the full body in order to better classify gait sequences. In visual tracking applications like the Kinect, where the data input involves sequential body joint angles or positions, dynamics (temporal dependence) can provide a powerful model that captures the essential structure of the sequential data in the presence of noise and bias. In our research we presented a machine learning approach to learn and discriminate normal gait and various abnormal gait patterns through a complex mechanism based on the dynamics of human movement. It can be used in any environments to detect abnormality in a gait pattern in a timely manner through observing the way an individual walks in front of the sensor.
Our approach was inspired by a Gaussian Process Latent Variable Model (GPLVM) and Gaussian Process Dynamical model (GPDM). This approach enables mapping the high dimensional gait sequences (i.e. a sequence of all joint angles during a gait cycle) to a lower dimensional (latent) space. In this lower dimensional space, the trajectory of all body joints during a gait cycle is mapped to lie near a nonlinear manifold. The structure of the low dimensional manifold changes as the type or quality of walking varies among people. This property enables visualization of high dimensional sequential gait to better graphically observe the difference between pathological gait and a model learned from healthy gait.
Figure 2. The 2-D nonlinear manifold learned by the dynamical GPLVM applied on gait sequences pertaining (a) healthy participant, (b) a participant with acquired brain injuries, and (c) stroke participants.
Due to the probabilistic formulation of this approach, it was adopted for classification where the difference between classes is in terms of their dynamics and not based on the distances. Moreover, when dealing with pathological gait, the probabilistic model should be built solely from normal examples since abnormal motions are highly variant. At test time, an unknown gait sequence is classified by computing its likelihood under dynamical Gaussian Process threshold based model trained on normal walking and comparing it to a chosen threshold.
Publication
Dolatabadi, Elham, Babak Taati, and Alex Mihailidis. "An Automated Classification of Pathological Gait Using Unobtrusive Sensing Technology." IEEE Transactions on Neural Systems and Rehabilitation Engineering 25.12 (2017): 2336-2346 (link).
Please see the AMBIENT page
Research Team
Elham Dolatabadi, University of Toronto
Alex Mihailidis, University of Toronto
Babak Taati, Toronto Rehabilitation Institute
