English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large-scale self-supervised pre-trained speech encoders outperformconventional approaches in speech recognition and translation tasks. Due to thehigh cost of developing these large models, building new encoders for new tasksand deploying them to on-device applications are infeasible. Prior studiespropose model compression methods to address this issue, but those works focuson smaller models and less realistic tasks. Thus, we propose ContrastiveLayer-to-layer Distillation (CoLLD), a novel knowledge distillation method tocompress pre-trained speech encoders by leveraging masked prediction andcontrastive learning to train student models to copy the behavior of a largeteacher model. CoLLD outperforms prior methods and closes the gap between smalland large models on multilingual speech-to-text translation and recognitionbenchmarks.

CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoders