English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Recent work has shown that speech paired with images can be used to learnsemantically meaningful speech representations even without any textualsupervision. In real-world low-resource settings, however, we often have accessto some transcribed speech. We study whether and how visual grounding is usefulin the presence of varying amounts of textual supervision. In particular, weconsider the task of semantic speech retrieval in a low-resource setting. Weuse a previously studied data set and task, where models are trained on imageswith spoken captions and evaluated on human judgments of semantic relevance. Wepropose a multitask learning approach to leverage both visual and textualmodalities, with visual supervision in the form of keyword probabilities froman external tagger. We find that visual grounding is helpful even in thepresence of textual supervision, and we analyze this effect over a range ofsizes of transcribed data sets. With ~5 hours of transcribed speech, we obtain23% higher average precision when also using visual supervision.

On the Contributions of Visual and Textual Supervision in Low-Resource Semantic Speech Retrieval