English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Model architectures such as wav2vec 2.0 and HuBERT have been proposed tolearn speech representations from audio waveforms in a self-supervised manner.When they are combined with downstream tasks such as keyword spotting andspeaker verification, they provide state-of-the-art performance. However, thesemodels use a large number of parameters, the smallest version of which has 95million parameters. This constitutes a challenge for edge AI devicedeployments. In this paper, we investigate the application of knowledgedistillation to speech representation learning (SRL) models followed by jointfine-tuning with multiple downstream voice-activated tasks. In our experimentson two such tasks, our approach results in nearly 75% reduction in model sizewhile suffering only 0.1% accuracy and 0.9% equal error rate degradationcompared to the full-size model. In addition, we show that fine-tuning the SRLmodels results in a significant performance boost compared to using frozen SRLmodels.

Application of Knowledge Distillation to Multi-task Speech Representation Learning