English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Self-supervised learning (SSL) has led to great strides in speech processing.However, the resources needed to train these models has become prohibitivelylarge as they continue to scale. Currently, only a few groups with substantialresources are capable of creating SSL models, which harms reproducibility. Inthis work, we optimize HuBERT SSL to fit in academic constraints. We reproduceHuBERT independently from the original implementation, with no performanceloss. Our code and training optimizations make SSL feasible with only 8 GPUs,instead of the 32 used in the original work. We also explore a semi-supervisedroute, using an ASR model to skip the first pre-training iteration. Within oneiteration of pre-training, our models improve over HuBERT on several tasks.Furthermore, our HuBERT Large variant requires only 8 GPUs, achieving similarperformance to the original trained on 128. As our contribution to thecommunity, all models, configurations, and code are made open-source in ESPnet.

Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute