English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The success of supervised deep learning methods is largely due to theirability to learn relevant features from raw data. Deep Neural Networks (DNNs)trained on large-scale datasets are capable of capturing a diverse set offeatures, and learning a representation that can generalize onto unseen tasksand datasets that are from the same domain. Hence, these models can be used aspowerful feature extractors, in combination with shallower models asclassifiers, for smaller tasks and datasets where the amount of training datais insufficient for learning an end-to-end model from scratch. During the pastyears, Convolutional Neural Networks (CNNs) have largely been the method ofchoice for audio processing. However, recently attention-based transformermodels have demonstrated great potential in supervised settings, outperformingCNNs. In this work, we investigate the use of audio transformers trained onlarge-scale datasets to learn general-purpose representations. We study how thedifferent setups in these audio transformers affect the quality of theirembeddings. We experiment with the models' time resolution, extracted embeddinglevel, and receptive fields in order to see how they affect performance on avariety of tasks and datasets, following the HEAR 2021 NeurIPS challengeevaluation setup. Our results show that representations extracted by audiotransformers outperform CNN representations. Furthermore, we will show thattransformers trained on Audioset can be extremely effective representationextractors for a wide range of downstream tasks.

Learning General Audio Representations with Large-Scale Training of Patchout Audio Transformers