English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recent years, pre-trained large language models (LLMs) have demonstratedremarkable efficiency in achieving an inference-time few-shot learningcapability known as in-context learning. However, existing literature hashighlighted the sensitivity of this capability to the selection of few-shotdemonstrations. Current understandings of the underlying mechanisms by whichthis capability arises from regular language model pretraining objectivesremain disconnected from the real-world LLMs. This study aims to examine thein-context learning phenomenon through a Bayesian lens, viewing real-world LLMsas latent variable models. On this premise, we propose an algorithm to selectoptimal demonstrations from a set of annotated data with a small LM, and thendirectly generalize the selected demonstrations to larger LMs. We demonstratesignificant improvement over baselines, averaged over eight GPT models on eightreal-world text classification datasets. We also demonstrate the real-worldusefulness of our algorithm on GSM8K, a math word problem dataset. Ourempirical findings support our hypothesis that LLMs implicitly infer a latentvariable containing task information.

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning