English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep generative models trained by maximum likelihood remain very popularmethods for reasoning about data probabilistically. However, it has beenobserved that they can assign higher likelihoods to out-of-distribution (OOD)data than in-distribution data, thus calling into question the meaning of theselikelihood values. In this work we provide a novel perspective on thisphenomenon, decomposing the average likelihood into a KL divergence term and anentropy term. We argue that the latter can explain the curious OOD behaviourmentioned above, suppressing likelihood values on datasets with higher entropy.Although our idea is simple, we have not seen it explored yet in theliterature. This analysis provides further explanation for the success of OODdetection methods based on likelihood ratios, as the problematic entropy termcancels out in expectation. Finally, we discuss how this observation relates torecent success in OOD detection with manifold-supported models, for which theabove decomposition does not hold directly.

Entropic Issues in Likelihood-Based OOD Detection