English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In order to equip NLP systems with selective prediction capability, severaltask-specific approaches have been proposed. However, which approaches workbest across tasks or even if they consistently outperform the simplest baseline'MaxProb' remains to be explored. To this end, we systematically study'selective prediction' in a large-scale setup of 17 datasets across several NLPtasks. Through comprehensive experiments under in-domain (IID), out-of-domain(OOD), and adversarial (ADV) settings, we show that despite leveragingadditional resources (held-out data/computation), none of the existingapproaches consistently and considerably outperforms MaxProb in all threesettings. Furthermore, their performance does not translate well across tasks.For instance, Monte-Carlo Dropout outperforms all other approaches on DuplicateDetection datasets but does not fare well on NLI datasets, especially in theOOD setting. Thus, we recommend that future selective prediction approachesshould be evaluated across tasks and settings for reliable estimation of theircapabilities.

Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings