English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Current pre-trained language models rely on large datasets for achievingstate-of-the-art performance. However, past research has shown that not allexamples in a dataset are equally important during training. In fact, it issometimes possible to prune a considerable fraction of the training set whilemaintaining the test performance. Established on standard vision benchmarks,two gradient-based scoring metrics for finding important examples are GraNd andits estimated version, EL2N. In this work, we employ these two metrics for thefirst time in NLP. We demonstrate that these metrics need to be computed afterat least one epoch of fine-tuning and they are not reliable in early steps.Furthermore, we show that by pruning a small portion of the examples with thehighest GraNd/EL2N scores, we can not only preserve the test accuracy, but alsosurpass it. This paper details adjustments and implementation choices whichenable GraNd and EL2N to be applied to NLP.

BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning