English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The widespread adoption of large language models (LLMs) makes it important torecognize their strengths and limitations. We argue that in order to develop aholistic understanding of these systems we need to consider the problem thatthey were trained to solve: next-word prediction over Internet text. Byrecognizing the pressures that this task exerts we can make predictions aboutthe strategies that LLMs will adopt, allowing us to reason about when they willsucceed or fail. This approach - which we call the teleological approach -leads us to identify three factors that we hypothesize will influence LLMaccuracy: the probability of the task to be performed, the probability of thetarget output, and the probability of the provided input. We predict that LLMswill achieve higher accuracy when these probabilities are high than when theyare low - even in deterministic settings where probability should not matter.To test our predictions, we evaluate two LLMs (GPT-3.5 and GPT-4) on eleventasks, and we find robust evidence that LLMs are influenced by probability inthe ways that we have hypothesized. In many cases, the experiments revealsurprising failure modes. For instance, GPT-4's accuracy at decoding a simplecipher is 51% when the output is a high-probability word sequence but only 13%when it is low-probability. These results show that AI practitioners should becareful about using LLMs in low-probability situations. More broadly, weconclude that we should not evaluate LLMs as if they are humans but shouldinstead treat them as a distinct type of system - one that has been shaped byits own particular set of pressures.

Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve