English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Self-supervised learning (SSL) has proven vital in speech and audio-relatedapplications. The paradigm trains a general model on unlabeled data that canlater be used to solve specific downstream tasks. This type of model is costlyto train as it requires manipulating long input sequences that can only behandled by powerful centralised servers. Surprisingly, despite many attempts toincrease training efficiency through model compression, the effects oftruncating input sequence lengths to reduce computation have not been studied.In this paper, we provide the first empirical study of SSL pre-training fordifferent specified sequence lengths and link this to various downstream tasks.We find that training on short sequences can dramatically reduce resource costswhile retaining a satisfactory performance for all tasks. This simple one-linechange would promote the migration of SSL training from data centres touser-end edge devices for more realistic and personalised applications.

Match to Win: Analysing Sequences Lengths for Efficient Self-supervised Learning in Speech and Audio