Files and links (1)
Collaborative_AI_Dysarthric_Speech_Recognition_System_with_Data_Augmentation_using_Generative_Adversarial_Neural_Network2.04 MB
Published VersionCC BY V4.0, Open Access
Journal article
Collaborative AI Dysarthric Speech Recognition System with Data Augmentation using Generative Adversarial Neural Network
IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol.33, pp.2097-2111
2025
PMID: 40372851
Abstract
This paper proposes a novel collaborative dysarthric speech recognition system designed to convert dysarthric speech into non-dysarthric speech to enhance the robustness of automatic speech recognition (ASR) systems fine-tuned for dysarthric speech. The system employs an innovative three-stage data augmentation framework: The first stage collaboratively augments the training dataset by generating static data and high-quality synthetic speech samples using a natural text-to-speech model (Tacotron2). The second stage applies a tempo perturbation technique that simulates the natural variation of speech rhythms by adjusting the playback tempo to improve the model’s adaptability to varying speech speeds. The third stage integrates the Inception-ResNet module with a temporal masking strategy using an enhanced CycleGAN-based conversion model to efficiently map conformal and non-conformal phonological features while preserving the overall speech structure and resolving temporal irregularities. Experiments conducted on the UASpeech corpus demonstrate a significant reduction in the word error rate (WER) compared to the baseline approach. Specifically, the three-stage data enhancement process achieves a reduction in the WER for the fine-tuned Wav2Vec2-XLSR and Whisper-Tiny models by 9.81% and 6.56%, respectively, with an average WER of 13.58% for the best performing system. These results highlight the effectiveness of the collaborative framework in improving the accuracy and naturalness of speech recognition for dysarthria, thereby offering individuals with dysarthria a more natural and intelligible communication experience.
Details
- Title
- Collaborative AI Dysarthric Speech Recognition System with Data Augmentation using Generative Adversarial Neural Network
- Authors
- Yibo He - Xi’an Jiaotong-Liverpool UniversityKah Phooi Seng - University of the Sunshine Coast, Queensland, School of Science, Technology and EngineeringLi Minn Ang - University of the Sunshine Coast, Queensland, School of Science, Technology and Engineering
- Publication details
- IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol.33, pp.2097-2111
- Publisher
- Institute of Electrical and Electronics Engineers
- Date published
- 2025
- DOI
- 10.1109/TNSRE.2025.3570383
- ISSN
- 1558-0210; 1534-4320
- PMID
- 40372851
- Copyright note
- This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
- Organisation Unit
- School of Science, Technology and Engineering; Engage Research Lab
- Language
- English
- Record Identifier
- 991129981802621
- Output Type
- Journal article
Metrics
13 Record Views
InCites Highlights
These are selected metrics from InCites Benchmarking & Analytics tool, related to this output
- Collaboration types
- Domestic collaboration
- International collaboration
- Web Of Science research areas
- Engineering, Biomedical
- Rehabilitation
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites