English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Understanding the performance of machine learning models across diverse datadistributions is critically important for reliable applications. Motivated bythis, there is a growing focus on curating benchmark datasets that capturedistribution shifts. While valuable, the existing benchmarks are limited inthat many of them only contain a small number of shifts and they lacksystematic annotation about what is different across different shifts. Wepresent MetaShift--a collection of 12,868 sets of natural images across 410classes--to address this challenge. We leverage the natural heterogeneity ofVisual Genome and its annotations to construct MetaShift. The key constructionidea is to cluster images using its metadata, which provides context for eachimage (e.g. "cats with cars" or "cats in bathroom") that represent distinctdata distributions. MetaShift has two important benefits: first, it containsorders of magnitude more natural data shifts than previously available. Second,it provides explicit explanations of what is unique about each of its data setsand a distance score that measures the amount of distribution shift between anytwo of its data sets. We demonstrate the utility of MetaShift in benchmarkingseveral recent proposals for training models to be robust to data shifts. Wefind that the simple empirical risk minimization performs the best when shiftsare moderate and no method had a systematic advantage for large shifts. We alsoshow how MetaShift can help to visualize conflicts between data subsets duringmodel training.

MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflicts