English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Advances in Large Language Models (LLMs) have led to remarkable capabilities,yet their inner mechanisms remain largely unknown. To understand these models,we need to unravel the functions of individual neurons and their contributionto the network. This paper introduces a novel automated approach designed toscale interpretability techniques across a vast array of neurons within LLMs,to make them more interpretable and ultimately safe. Conventional methodsrequire examination of examples with strong neuron activation and manualidentification of patterns to decipher the concepts a neuron responds to. Wepropose Neuron to Graph (N2G), an innovative tool that automatically extracts aneuron's behaviour from the dataset it was trained on and translates it into aninterpretable graph. N2G uses truncation and saliency methods to emphasise onlythe most pertinent tokens to a neuron while enriching dataset examples withdiverse samples to better encompass the full spectrum of neuron behaviour.These graphs can be visualised to aid researchers' manual interpretation, andcan generate token activations on text for automatic validation by comparisonwith the neuron's ground truth activations, which we use to show that the modelis better at predicting neuron activation than two baseline methods. We alsodemonstrate how the generated graph representations can be flexibly used tofacilitate further automation of interpretability research, by searching forneurons with particular properties, or programmatically comparing neurons toeach other to identify similar neurons. Our method easily scales to build graphrepresentations for all neurons in a 6-layer Transformer model using a singleTesla T4 GPU, allowing for wide usability. We release the code and instructionsfor use at https://github.com/alexjfoote/Neuron2Graph.

Neuron to Graph: Interpreting Language Model Neurons at Scale