English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The Information Bottleneck (IB) objective uses information theory toformulate a task-performance versus robustness trade-off. It has beensuccessfully applied in the standard discriminative classification setting. Wepose the question whether the IB can also be used to train generativelikelihood models such as normalizing flows. Since normalizing flows useinvertible network architectures (INNs), they are information-preserving byconstruction. This seems contradictory to the idea of a bottleneck. In thiswork, firstly, we develop the theory and methodology of IB-INNs, a class ofconditional normalizing flows where INNs are trained using the IB objective:Introducing a small amount of {\em controlled} information loss allows for anasymptotically exact formulation of the IB, while keeping the INN's generativecapabilities intact. Secondly, we investigate the properties of these modelsexperimentally, specifically used as generative classifiers. This model classoffers advantages such as improved uncertainty quantification andout-of-distribution detection, but traditional generative classifier solutionssuffer considerably in classification accuracy. We find the trade-off parameterin the IB controls a mix of generative capabilities and accuracy close tostandard classifiers. Empirically, our uncertainty estimates in this mixedregime compare favourably to conventional generative and discriminativeclassifiers.

Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification