Uncertainty-aware few-shot learning for silent speech decoding

초록

Silent speech decoding enables communication for individuals with speech impairments by leveraging Electromyographic (EMG) signals from facial muscle movements. However, existing deep learning models struggle with handling uncertainty, leading to poor generalization, particularly with limited and varied data. This study aims to develop an uncertainty-aware few-shot learning approach to enhance robustness, generalization, and rapid adaptation for silent speech decoding. We propose an innovative method involving three core components: (i) creation of a Dynamic MultiAccent dataset that integrates multiple accents and diverse scenarios to introduce realistic variability, (ii) enhancement of the transduction model with skip connections to improve feature propagation, and (iii) incorporation of uncertainty-aware modeling via variational inference within the vocoder to effectively address variability and uncertainty. Evaluations demonstrate that our enhanced transduction model achieves superior performance compared to the state-of-the-art methods, reducing word error rate (WER) by 2.1% on the public dataset. Furthermore, on our dataset with limited samples, the proposed uncertainty-aware optimization approach significantly improved error calibration metrics, reducing Expected Calibration Error (ECE) by 2.07%, Maximum Calibration Error (MCE) by 2.69%, and achieving 3.67% reductions in WER compared to our model without uncertainty-aware optimization. The proposed approach effectively mitigates uncertainties, enhancing the reliability of silent speech decoding in clinical settings. This study contributes to enabling seamless and reliable assistive communication for speech-disabled individuals, overcoming current challenges in silent speech decoding.

키워드