English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Machine learning advances in the last decade have relied significantly onlarge-scale datasets that continue to grow in size. Increasingly, thosedatasets also contain different data modalities. However, large multi-modaldatasets are hard to annotate, and annotations may contain biases that we areoften unaware of. Deep-net-based classifiers, in turn, are prone to exploitthose biases and to find shortcuts. To study and quantify this concern, weintroduce the perceptual score, a metric that assesses the degree to which amodel relies on the different subsets of the input features, i.e., modalities.Using the perceptual score, we find a surprisingly consistent trend across fourpopular datasets: recent, more accurate state-of-the-art multi-modal models forvisual question-answering or visual dialog tend to perceive the visual dataless than their predecessors. This trend is concerning as answers are henceincreasingly inferred from textual cues only. Using the perceptual score alsohelps to analyze model biases by decomposing the score into data subsetcontributions. We hope to spur a discussion on the perceptiveness ofmulti-modal models and also hope to encourage the community working onmulti-modal classifiers to start quantifying perceptiveness via the proposedperceptual score.

Perceptual Score: What Data Modalities Does Your Model Perceive?