English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Information theoretic quantities play an important role in various settingsin machine learning, including causality testing, structure inference ingraphical models, time-series problems, feature selection as well as inproviding privacy guarantees. A key quantity of interest is the mutualinformation and generalizations thereof, including conditional mutualinformation, multivariate mutual information, total correlation and directedinformation. While the aforementioned information quantities are well definedin arbitrary probability spaces, existing estimators add or subtract entropies(we term them $\Sigma H$ methods). These methods work only in purely discretespace or purely continuous case since entropy (or differential entropy) is welldefined only in that regime.  In this paper, we define a general graph divergence measure ($\mathbb{GDM}$),as a measure of incompatibility between the observed distribution and a givengraphical model structure. This generalizes the aforementioned informationmeasures and we construct a novel estimator via a coupling trick that directlyestimates these multivariate information measures using the Radon-Nikodymderivative. These estimators are proven to be consistent in a general settingwhich includes several cases where the existing estimators fail, thus providingthe only known estimators for the following settings: (1) the data has somediscrete and some continuous-valued components (2) some (or all) of thecomponents themselves are discrete-continuous mixtures (3) the data isreal-valued but does not have a joint density on the entire space, rather issupported on a low-dimensional manifold. We show that our proposed estimatorssignificantly outperform known estimators on synthetic and real datasets.

Estimators for Multivariate Information Measures in General Probability Spaces