English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

With the increase in availability of large pre-trained language models (LMs)in Natural Language Processing (NLP), it becomes critical to assess their fitfor a specific target task a priori - as fine-tuning the entire space ofavailable LMs is computationally prohibitive and unsustainable. However,encoder transferability estimation has received little to no attention in NLP.In this paper, we propose to generate quantitative evidence to predict whichLM, out of a pool of models, will perform best on a target task without havingto fine-tune all candidates. We provide a comprehensive study on LM ranking for10 NLP tasks spanning the two fundamental problem types of classification andstructured prediction. We adopt the state-of-the-art Logarithm of MaximumEvidence (LogME) measure from Computer Vision (CV) and find that it positivelycorrelates with final LM performance in 94% of the setups. In the first studyof its kind, we further compare transferability measures with the de factostandard of human practitioner ranking, finding that evidence from quantitativemetrics is more robust than pure intuition and can help identify unexpected LMcandidates.

Evidence > Intuition: Transferability Estimation for Encoder Selection