English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

We tackle the task of environmental event classification by drawinginspiration from the transformer neural network architecture used in machinetranslation. We modify this attention-based feedforward structure in such a waythat allows the resulting model to use audio as well as video to compute soundevent predictions. We perform extensive experiments with these adaptedtransformers on an audiovisual data set, obtained by appending relevant visualinformation to an existing large-scale weakly labeled audio collection. Theemployed multi-label data contains clip-level annotation indicating thepresence or absence of 17 classes of environmental sounds, and does not includetemporal information. We show that the proposed modified transformers stronglyimprove upon previously introduced models and in fact achieve state-of-the-artresults. We also make a compelling case for devoting more attention to researchin multimodal audiovisual classification by proving the usefulness of visualinformation for the task at hand,namely audio event recognition. In addition,we visualize internal attention patterns of the audiovisual transformers and indoing so demonstrate their potential for performing multimodal synchronization.

Audiovisual Transformer Architectures for Large-Scale Classification and Synchronization of Weakly Labeled Audio Events