English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite recent explosion of interests in in-context learning, the underlyingmechanism and the precise impact of the quality of demonstrations remainelusive. Intuitively, ground-truth labels should have as much impact inin-context learning (ICL) as supervised learning, but recent work reported thatthe input-label correspondence is significantly less important than previouslythought. Intrigued by this counter-intuitive observation, we re-examine theimportance of ground-truth labels in in-context learning. With the introductionof two novel metrics, namely Label-Correctness Sensitivity and Ground-truthLabel Effect Ratio (GLER), we were able to conduct quantifiable analysis on theimpact of ground-truth label demonstrations. Through extensive analyses, wefind that the correct input-label mappings can have varying impacts on thedownstream in-context learning performances, depending on the experimentalconfiguration. Through additional studies, we identify key components, such asthe verbosity of prompt templates and the language model size, as thecontrolling factor to achieve more noise-resilient ICL.

Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations