English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many recent approaches to natural language tasks are built on the remarkableabilities of large language models. Large language models can performin-context learning, where they learn a new task from a few taskdemonstrations, without any parameter updates. This work examines theimplications of in-context learning for the creation of datasets for newnatural language tasks. Departing from recent in-context learning methods, weformulate an annotation-efficient, two-step framework: selective annotationthat chooses a pool of examples to annotate from unlabeled data in advance,followed by prompt retrieval that retrieves task examples from the annotatedpool at test time. Based on this framework, we propose an unsupervised,graph-based selective annotation method, voke-k, to select diverse,representative examples to annotate. Extensive experiments on 10 datasets(covering classification, commonsense reasoning, dialogue, and text/codegeneration) demonstrate that our selective annotation method improves the taskperformance by a large margin. On average, vote-k achieves a 12.9%/11.4%relative gain under an annotation budget of 18/100, as compared to randomlyselecting examples to annotate. Compared to state-of-the-art supervisedfinetuning approaches, it yields similar performance with 10-100x lessannotation cost across 10 tasks. We further analyze the effectiveness of ourframework in various scenarios: language models with varying sizes, alternativeselective annotation methods, and cases where there is a test data domainshift. We hope that our studies will serve as a basis for data annotations aslarge language models are increasingly applied to new tasks. Our code isavailable at https://github.com/HKUNLP/icl-selective-annotation.

Selective Annotation Makes Language Models Better Few-Shot Learners