English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Overparameterized neural networks can be highly accurate on average on ani.i.d. test set yet consistently fail on atypical groups of the data (e.g., bylearning spurious correlations that hold on average but not in such groups).Distributionally robust optimization (DRO) allows us to learn models thatinstead minimize the worst-case training loss over a set of pre-defined groups.However, we find that naively applying group DRO to overparameterized neuralnetworks fails: these models can perfectly fit the training data, and any modelwith vanishing average training loss also already has vanishing worst-casetraining loss. Instead, the poor worst-case performance arises from poorgeneralization on some groups. By coupling group DRO models with increasedregularization---a stronger-than-typical L2 penalty or early stopping---weachieve substantially higher worst-group accuracies, with 10-40 percentagepoint improvements on a natural language inference task and two image tasks,while maintaining high average accuracies. Our results suggest thatregularization is important for worst-group generalization in theoverparameterized regime, even if it is not needed for average generalization.Finally, we introduce a stochastic optimization algorithm, with convergenceguarantees, to efficiently train group DRO models.

Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization