English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Mutual information (MI) is an information-theoretic measure of dependencybetween two random variables. Several methods to estimate MI, from samples oftwo random variables with unknown underlying probability distributions havebeen proposed in the literature. Recent methods realize parametric probabilitydistributions or critic as a neural network to approximate unknown densityratios. The approximated density ratios are used to estimate differentvariational lower bounds of MI. While these methods provide reliable estimationwhen the true MI is low, they produce high variance estimates in cases of highMI. We argue that the high variance characteristic is due to the uncontrolledcomplexity of the critic's hypothesis space. In support of this argument, weuse the data-driven Rademacher complexity of the hypothesis space associatedwith the critic's architecture to analyse generalization error bound ofvariational lower bound estimates of MI. In the proposed work, we show that itis possible to negate the high variance characteristics of these estimators byconstraining the critic's hypothesis space to Reproducing Hilbert Kernel Space(RKHS), which corresponds to a kernel learned using Automated Spectral KernelLearning (ASKL). By analysing the aforementioned generalization error bounds,we augment the overall optimisation objective with effective regularisationterm. We empirically demonstrate the efficacy of this regularization inenforcing proper bias variance tradeoff on four variational lower bounds,namely NWJ, MINE, JS and SMILE.

Reducing the Variance of Variational Estimates of Mutual Information by Limiting the Critic's Hypothesis Space to RKHS