English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Empirical risk minimization (ERM) is known in practice to be non-robust todistributional shift where the training and the test distributions aredifferent. A suite of approaches, such as importance weighting, and variants ofdistributionally robust optimization (DRO), have been proposed to solve thisproblem. But a line of recent work has empirically shown that these approachesdo not significantly improve over ERM in real applications with distributionshift. The goal of this work is to obtain a comprehensive theoreticalunderstanding of this intriguing phenomenon. We first posit the class ofGeneralized Reweighting (GRW) algorithms, as a broad category of approachesthat iteratively update model parameters based on iterative reweighting of thetraining samples. We show that when overparameterized models are trained underGRW, the resulting models are close to that obtained by ERM. We also show thatadding small regularization which does not greatly affect the empiricaltraining accuracy does not help. Together, our results show that a broadcategory of what we term GRW approaches are not able to achievedistributionally robust generalization. Our work thus has the followingsobering takeaway: to make progress towards distributionally robustgeneralization, we either have to develop non-GRW approaches, or perhaps devisenovel classification/regression loss functions that are adapted to the class ofGRW approaches.

Understanding Why Generalized Reweighting Does Not Improve Over ERM