English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Individuals with hearing impairments face challenges in their ability tocomprehend speech, particularly in noisy environments. The aim of this study isto explore the effectiveness of audio-visual speech enhancement (AVSE) inenhancing the intelligibility of vocoded speech in cochlear implant (CI)simulations. Notably, the study focuses on a challenged scenario where there islimited availability of training data for the AVSE task. To address thisproblem, we propose a novel deep neural network framework termedSelf-Supervised Learning-based AVSE (SSL-AVSE). The proposed SSL-AVSE combinesvisual cues, such as lip and mouth movements, from the target speakers withcorresponding audio signals. The contextually combined audio and visual dataare then fed into a Transformer-based SSL AV-HuBERT model to extract features,which are further processed using a BLSTM-based SE model. The resultsdemonstrate several key findings. Firstly, SSL-AVSE successfully overcomes theissue of limited data by leveraging the AV-HuBERT model. Secondly, byfine-tuning the AV-HuBERT model parameters for the target SE task, significantperformance improvements are achieved. Specifically, there is a notableenhancement in PESQ (Perceptual Evaluation of Speech Quality) from 1.43 to 1.67and in STOI (Short-Time Objective Intelligibility) from 0.70 to 0.74.Furthermore, the performance of the SSL-AVSE was evaluated using CI vocodedspeech to assess the intelligibility for CI users. Comparative experimentaloutcomes reveal that in the presence of dynamic noises encountered during humanconversations, SSL-AVSE exhibits a substantial improvement. The NCM (NormalCorrelation Matrix) values indicate an increase of 26.5% to 87.2% compared tothe noisy baseline.

Audio-Visual Speech Enhancement Using Self-supervised Learning to Improve Speech Intelligibility in Cochlear Implant Simulations