English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Automated Audio Captioning (AAC) involves generating natural languagedescriptions of audio content, using encoder-decoder architectures. An audioencoder produces audio embeddings fed to a decoder, usually a Transformerdecoder, for caption generation. In this work, we describe our model, whichnovelty, compared to existing models, lies in the use of a ConvNeXtarchitecture as audio encoder, adapted from the vision domain to audioclassification. This model, called CNext-trans, achieved state-of-the-artscores on the AudioCaps (AC) dataset and performed competitively on Clotho(CL), while using four to forty times fewer parameters than existing models. Weexamine potential biases in the AC dataset due to its origin from AudioSet byinvestigating unbiased encoder's impact on performance. Using the well-knownPANN's CNN14, for instance, as an unbiased encoder, we observed a 1.7% absolutereduction in SPIDEr score (where higher scores indicate better performance). Toimprove cross-dataset performance, we conducted experiments by combiningmultiple AAC datasets (AC, CL, MACS, WavCaps) for training. Although thisstrategy enhanced overall model performance across datasets, it still fellshort compared to models trained specifically on a single target dataset,indicating the absence of a one-size-fits-all model. To mitigate performancegaps between datasets, we introduced a Task Embedding (TE) token, allowing themodel to identify the source dataset for each input sample. We provide insightsinto the impact of these TEs on both the form (words) and content (sound eventtypes) of the generated captions. The resulting model, named CoNeTTE, anunbiased CNext-trans model enriched with dataset-specific Task Embeddings,achieved SPIDEr scores of 44.1% and 30.5% on AC and CL, respectively. Codeavailable: https://github.com/Labbeti/conette-audio-captioning.

CoNeTTE: An efficient Audio Captioning system leveraging multiple datasets with Task Embedding