English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Automated audio captioning (AAC) aims to generate informative descriptionsfor various sounds from nature and/or human activities. In recent years, AAChas quickly attracted research interest, with state-of-the-art systems nowrelying on a sequence-to-sequence (seq2seq) backbone powered by strong modelssuch as Transformers. Following the macro-trend of applied machine learningresearch, in this work, we strive to improve the performance of seq2seq AACmodels by extensively leveraging pretrained models and large language models(LLMs). Specifically, we utilize BEATs to extract fine-grained audio features.Then, we employ Instructor LLM to fetch text embeddings of captions, and infusetheir language-modality knowledge into BEATs audio features via an auxiliaryInfoNCE loss function. Moreover, we propose a novel data augmentation methodthat uses ChatGPT to produce caption mix-ups (i.e., grammatical and compactcombinations of two captions) which, together with the corresponding audiomixtures, increase not only the amount but also the complexity and diversity oftraining data. During inference, we propose to employ nucleus sampling and ahybrid reranking algorithm, which has not been explored in AAC research.Combining our efforts, our model achieves a new state-of-the-art 32.6 SPIDEr-FLscore on the Clotho evaluation split, and wins the 2023 DCASE AAC challenge.

Improving Audio Captioning Models with Fine-grained Audio Features, Text Embedding Supervision, and LLM Mix-up Augmentation