English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Foundation models encode rich representations that can be adapted to adesired task by fine-tuning on task-specific data. However, fine-tuning a modelon one particular data distribution often compromises the model's originalperformance on other distributions. Current methods for robust fine-tuningutilize hand-crafted regularization techniques to constrain the fine-tuningprocess towards the base foundation model. Yet, it is hard to precisely specifywhat characteristics of the foundation model to retain during fine-tuning, asthis depends on how the pre-training, fine-tuning, and evaluation datadistributions relate to each other. We propose AutoFT, a data-driven approachfor guiding foundation model fine-tuning. AutoFT optimizes fine-tuninghyperparameters to maximize performance on a small out-of-distribution (OOD)validation set. To guide fine-tuning in a granular way, AutoFT searches ahighly expressive hyperparameter space that includes weight coefficients formany different losses, in addition to learning rate and weight decay values. Weevaluate AutoFT on nine natural distribution shifts which include domain shiftsand subpopulation shifts. Our experiments show that AutoFT significantlyimproves generalization to new OOD data, outperforming existing robustfine-tuning methods. Notably, AutoFT achieves new state-of-the-art performanceon the WILDS-iWildCam and WILDS-FMoW benchmarks, outperforming the previousbest methods by $6.0\%$ and $1.5\%$, respectively.

AutoFT: Robust Fine-Tuning by Optimizing Hyperparameters on OOD Data