Gait-to-Gait Emotional Human-Robot Interaction Utilizing Trajectories-Aware and Skeleton-Graph-Aware Spatial-Temporal Transformer

Chenghao Li; Kah Phooi Seng; Li-Minn Ang

doi:10.3390/s25030734

Back

Gait-to-Gait Emotional Human-Robot Interaction Utilizing Trajectories-Aware and Skeleton-Graph-Aware Spatial-Temporal Transformer

Journal article

Open access

Peer reviewed

Gait-to-Gait Emotional Human-Robot Interaction Utilizing Trajectories-Aware and Skeleton-Graph-Aware Spatial-Temporal Transformer

Chenghao Li, Kah Phooi Seng and Li-Minn Ang

Sensors, Vol.25(3), pp.1-22

2025

DOI: https://doi.org/10.3390/s25030734

PMCID: PMC11820152

PMID: 39943373

Files and links (1)

pdf

sensors-25-00734-v33.49 MBDownload View

Published Version Open Access CC BY V4.0

Abstract

social robot

human-robot interaction

emotion classification

The emotional response of robotics is crucial for promoting the socially intelligent level of human-robot interaction (HRI). The development of machine learning has extensively stimulated research on emotional recognition for robots. Our research focuses on emotional gaits, a type of simple modality that stores a series of joint coordinates and is easy for humanoid robots to execute. However, a limited amount of research investigates emotional HRI systems based on gaits, indicating an existing gap in human emotion gait recognition and robotic emotional gait response. To address this challenge, we propose a Gait-to-Gait Emotional HRI system, emphasizing the development of an innovative emotion classification model. In our system, the humanoid robot NAO can recognize emotions from human gaits through our Trajectories-Aware and Skeleton-Graph-Aware Spatial-Temporal Transformer (TS-ST) and respond with pre-set emotional gaits that reflect the same emotion as the human presented. Our TS-ST outperforms the current state-of-the-art human-gait emotion recognition model applied to robots on the Emotion-Gait dataset.

Details

Title: Gait-to-Gait Emotional Human-Robot Interaction Utilizing Trajectories-Aware and Skeleton-Graph-Aware Spatial-Temporal Transformer
Authors: Chenghao Li - Xi’an Jiaotong-Liverpool University
Kah Phooi Seng (Corresponding Author) - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
Li-Minn Ang - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
Publication details: Sensors, Vol.25(3), pp.1-22
Publisher: MDPI AG
Date published: 2025
DOI: 10.3390/s25030734
ISSN: 1424-8220
PMID: 39943373; PMC11820152
Copyright note: Copyright: © 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Data Availability: The data presented in this study are openly available in (https://go.umd.edu/emotion-gait) accessed from 28 October 2019.
Organisation Unit: School of Science, Technology and Engineering
Language: English
Record Identifier: 991104946002621
Output Type: Journal article

Metrics

5 File views/ downloads

26 Record Views

1 Times Cited - Web of Science

InCites Highlights

These are selected metrics from InCites Benchmarking & Analytics tool, related to this output

Collaboration types: Domestic collaboration; International collaboration
Web Of Science research areas: Chemistry, Analytical; Engineering, Electrical & Electronic; Instruments & Instrumentation