English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Despite the massive success of fine-tuning Pre-trained Language Models(PLMs), they remain susceptible to out-of-distribution input. Datasetcartography is a simple yet effective dual-model approach that improves therobustness of fine-tuned PLMs. It involves fine-tuning a model on the originaltraining set (i.e. reference model), selecting a subset of important traininginstances based on the training dynamics, and fine-tuning again only on theseselected examples (i.e. main model). However, this approach requiresfine-tuning the same model twice, which is computationally expensive for largePLMs. In this paper, we show that (1) training dynamics are highly transferableacross model sizes and pre-training methods, and that (2) fine-tuning mainmodels using these selected training instances achieves higher trainingefficiency than empirical risk minimization (ERM). Building on theseobservations, we propose a novel fine-tuning approach: Fine-Tuning bytransFerring Training dynamics (FTFT). Compared with dataset cartography, FTFTuses more efficient reference models and aggressive early stopping. FTFTachieves robustness improvements over ERM while lowering the training cost byup to $\sim 50\%$.

FTFT: Efficient and Robust Fine-Tuning by Transferring Training Dynamics