Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury – a comparison of an image extraction and classification system to expert scoring

Ashleigh Mobbs; Michelle Kahn; Gavin Williams; Benjamin F. Mentiplay; Yong-Hao Pua; Ross A. Clark

doi:10.1186/s12984-024-01406-w

Back

Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury – a comparison of an image extraction and classification system to expert scoring

Journal article

Open access

Peer reviewed

Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury – a comparison of an image extraction and classification system to expert scoring

Ashleigh Mobbs, Michelle Kahn, Gavin Williams, Benjamin F. Mentiplay, Yong-Hao Pua and Ross A. Clark

Journal of Neuroengineering and Rehabilitation, Vol.21, pp.1-11

2024

DOI: https://doi.org/10.1186/s12984-024-01406-w

PMCID: PMC11264460

PMID: 39039594

Files and links (1)

pdf

s12984-024-01406-w1.50 MBDownload View

Published Version Open Access CC BY V4.0

Abstract

Background Walking impairment is a common disability post acquired brain injury (ABI), with visually evident arm movement abnormality identified as negatively impacting a multitude of psychological factors. The International Classification of Functioning, Disability and Health (ICF) qualifiers scale has been used to subjectively assess arm movement abnormality, showing strong intra-rater and test-retest reliability, however, only moderate inter-rater reliability. This impacts clinical utility, limiting its use as a measurement tool. To both automate the analysis and overcome these errors, the primary aim of this study was to evaluate the ability of a novel two-level machine learning model to assess arm movement abnormality during walking in people with ABI. Methods Frontal plane gait videos were used to train four networks with 50%, 75%, 90%, and 100% of participants (ABI: n = 42, healthy controls: n = 34) to automatically identify anatomical landmarks using DeepLabCut™ and calculate two-dimensional kinematic joint angles. Assessment scores from three experienced neurorehabilitation clinicians were used with these joint angles to train random forest networks with nested cross-validation to predict assessor scores for all videos. Agreement between unseen participant (i.e. test group participants that were not used to train the model) predictions and each individual assessor’s scores were compared using quadratic weighted kappa. One sample t-tests (to determine over/underprediction against clinician ratings) and one-way ANOVA (to determine differences between networks) were applied to the four networks. Results The machine learning predictions have similar agreement to experienced human assessors, with no statistically significant (p < 0.05) difference for any match contingency. There was no statistically significant difference between the predictions from the four networks (F = 0.119; p = 0.949). The four networks did however under-predict scores with small effect sizes (p range = 0.007 to 0.040; Cohen’s d range = 0.156 to 0.217). Conclusions This study demonstrated that machine learning can perform similarly to experienced clinicians when subjectively assessing arm movement abnormality in people with ABI. The relatively small sample size may have resulted in under-prediction of some scores, albeit with small effect sizes. Studies with larger sample sizes that objectively and automatically assess dynamic movement in both local and telerehabilitation assessments, for example using smartphones and edge-based machine learning, to reduce measurement error and healthcare access inequality are needed.

Details

Title: Machine learning for automating subjective clinical assessment of gait impairment in people with acquired brain injury – a comparison of an image extraction and classification system to expert scoring
Authors: Ashleigh Mobbs - University of the Sunshine Coast, Queensland, School of Business and Creative Industries
Michelle Kahn - Epworth Hospital
Gavin Williams - Epworth Hospital
Benjamin F. Mentiplay - La Trobe University
Yong-Hao Pua - Singapore General Hospital
Ross A. Clark (Corresponding Author) - University of the Sunshine Coast, Queensland, School of Health - Sports & Exercise Science
Publication details: Journal of Neuroengineering and Rehabilitation, Vol.21, pp.1-11
Publisher: BioMed Central Ltd.
Date published: 2024
DOI: 10.1186/s12984-024-01406-w
ISSN: 1743-0003
PMID: 39039594; PMC11264460
Copyright note: This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Data Availability: Raw data for our dataset are not publicly available to preserve individuals’ privacy under the European General Data Protection Regulation. They are however available upon request in an anonymised and supervised form.
Organisation Unit: School of Business and Creative Industries; Healthy Ageing Research Cluster; School of Health - Sports & Exercise Science
Language: English
Record Identifier: 991048859802621
Output Type: Journal article

Metrics

3 File views/ downloads

56 Record Views