English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Through solving pretext tasks, self-supervised learning (SSL) leveragesunlabeled data to extract useful latent representations replacing traditionalinput features in the downstream task. A common pretext task consists inpretraining a SSL model on pseudo-labels derived from the original signal. Thistechnique is particularly relevant for speech data where various meaningfulsignal processing features may serve as pseudo-labels. However, the process ofselecting pseudo-labels, for speech or other types of data, remains mostlyunexplored and currently relies on observing the results on the finaldownstream task. Nevertheless, this methodology is not sustainable at scale dueto substantial computational (hence carbon) costs. Thus, this paper introducesa practical and theoretical framework to select relevant pseudo-labels withrespect to a given downstream task. More precisely, we propose a functionalestimator of the pseudo-label utility grounded in the conditional independencetheory, which does not require any training. The experiments conducted onspeaker recognition and automatic speech recognition validate our estimator,showing a significant correlation between the performance observed on thedownstream task and the utility estimates obtained with our approach,facilitating the prospection of relevant pseudo-labels for self-supervisedspeech representation learning.

Conditional independence for pretext task selection in Self-supervised speech representation learning