Advanced · Safety

Mechanistic Interpretability

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A research program that reverse-engineers neural networks into human-understandable circuits, features, and algorithms.

Technical Definition

A research program that reverse-engineers neural networks into human-understandable circuits, features, and algorithms.

How it works

Pioneered by Anthropic, Neel Nanda, and the Distill 'Circuits' line, mech interp treats networks as compiled programs and tries to recover their source. Techniques include activation patching, path patching, logit lens, sparse autoencoders for feature decomposition, and toy-model studies of superposition. Goals range from finding deception circuits to enabling formal alignment guarantees.

Related Concepts

• AI Alignment — The research field aimed at making AI systems pursue the goals their developers and users actually intend.
• Sparse Autoencoder (SAE) — An overcomplete autoencoder with an L1 penalty used in interpretability to decompose neural activations into monosemantic features.
• Activation Patching — An interpretability technique that swaps activations from one forward pass into another to causally localize where a behavior lives in a model.