English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite rapid adoption and deployment of large language models (LLMs), theinternal computations of these models remain opaque and poorly understood. Inthis work, we seek to understand how high-level human-interpretable featuresare represented within the internal neuron activations of LLMs. We train$k$-sparse linear classifiers (probes) on these internal activations to predictthe presence of features in the input; by varying the value of $k$ we study thesparsity of learned representations and how this varies with model scale. With$k=1$, we localize individual neurons which are highly relevant for aparticular feature, and perform a number of case studies to illustrate generalproperties of LLMs. In particular, we show that early layers make use of sparsecombinations of neurons to represent many features in superposition, thatmiddle layers have seemingly dedicated neurons to represent higher-levelcontextual features, and that increasing scale causes representational sparsityto increase on average, but there are multiple types of scaling dynamics. Inall, we probe for over 100 unique features comprising 10 different categoriesin 7 different models spanning 70 million to 6.9 billion parameters.

Finding Neurons in a Haystack: Case Studies with Sparse Probing