English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The massive growth of self-supervised learning (SSL) has been witnessed inlanguage, vision, speech, and audio domains over the past few years. Whilediscrete label prediction is widely adopted for other modalities, thestate-of-the-art audio SSL models still employ reconstruction loss forpre-training. Compared with reconstruction loss, semantic-rich discrete labelprediction encourages the SSL model to abstract the high-level audio semanticsand discard the redundant details as in human perception. However, asemantic-rich acoustic tokenizer for general audio pre-training is usually notstraightforward to obtain, due to the continuous property of audio andunavailable phoneme sequences like speech. To tackle this challenge, we proposeBEATs, an iterative audio pre-training framework to learn Bidirectional Encoderrepresentation from Audio Transformers, where an acoustic tokenizer and anaudio SSL model are optimized by iterations. In the first iteration, we userandom projection as the acoustic tokenizer to train an audio SSL model in amask and label prediction manner. Then, we train an acoustic tokenizer for thenext iteration by distilling the semantic knowledge from the pre-trained orfine-tuned audio SSL model. The iteration is repeated with the hope of mutualpromotion of the acoustic tokenizer and audio SSL model. The experimentalresults demonstrate our acoustic tokenizers can generate discrete labels withrich audio semantics and our audio SSL models achieve state-of-the-art resultsacross various audio classification benchmarks, even outperforming previousmodels that use more training data and model parameters significantly.Specifically, we set a new state-of-the-art mAP 50.6% on AudioSet-2M foraudio-only models without using any external data, and 98.1% accuracy onESC-50. The code and pre-trained models are available at https://aka.ms/beats.

BEATs: Audio Pre-Training with Acoustic Tokenizers