English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Developing and deploying machine learning models safely depends on theability to characterize and compare their abilities to generalize to newenvironments. Although recent work has proposed a variety of methods that candirectly predict or theoretically bound the generalization capacity of a model,they rely on strong assumptions such as matching train/test distributions andaccess to model gradients. In order to characterize generalization when theseassumptions are not satisfied, we propose neighborhood invariance, a measure ofa classifier's output invariance in a local transformation neighborhood.Specifically, we sample a set of transformations and given an input test point,calculate the invariance as the largest fraction of transformed pointsclassified into the same class. Crucially, our measure is simple to calculate,does not depend on the test point's true label, makes no assumptions about thedata distribution or model, and can be applied even in out-of-domain (OOD)settings where existing methods cannot, requiring only selecting a set ofappropriate data transformations. In experiments on robustness benchmarks inimage classification, sentiment analysis, and natural language inference, wedemonstrate a strong and robust correlation between our neighborhood invariancemeasure and actual OOD generalization on over 4,600 models evaluated on over100 unique train/test domain pairs.

Predicting Out-of-Domain Generalization with Neighborhood Invariance