English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Wav2vec 2.0 (W2V2) has shown impressive performance in automatic speechrecognition (ASR). However, the large model size and the non-streamingarchitecture make it hard to be used under low-resource or streaming scenarios.In this work, we propose a two-stage knowledge distillation method to solvethese two problems: the first step is to make the big and non-streaming teachermodel smaller, and the second step is to make it streaming. Specially, we adoptthe MSE loss for the distillation of hidden layers and the modified LF-MMI lossfor the distillation of the prediction layer. Experiments are conducted onGigaspeech, Librispeech, and an in-house dataset. The results show that thedistilled student model (DistillW2V2) we finally get is 8x faster and 12xsmaller than the original teacher model. For the 480ms latency setup, theDistillW2V2's relative word error rate (WER) degradation varies from 9% to23.4% on test sets, which reveals a promising way to extend the W2V2'sapplication scope.

DistillW2V2: A Small and Streaming Wav2vec 2.0 Based ASR Model