English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-supervised speech pre-training enables deep neural network models tocapture meaningful and disentangled factors from raw waveform signals. Thelearned universal speech representations can then be used across numerousdownstream tasks. These representations, however, are sensitive to distributionshifts caused by environmental factors, such as noise and/or roomreverberation. Their large sizes, in turn, make them unfeasible for edgeapplications. In this work, we propose a knowledge distillation methodologytermed RobustDistiller which compresses universal representations while makingthem more robust against environmental artifacts via a multi-task learningobjective. The proposed layer-wise distillation recipe is evaluated on top ofthree well-established universal representations, as well as with threedownstream tasks. Experimental results show the proposed methodology applied ontop of the WavLM Base+ teacher model outperforming all other benchmarks acrossnoise types and levels, as well as reverberation times. Oftentimes, theobtained results with the student model (24M parameters) achieved resultsinline with those of the teacher model (95M).

RobustDistiller: Compressing Universal Speech Representations for Enhanced Environment Robustness