1 episodes
Paper: Sparse Autoencoders for Interpretability
Superposition lets networks store more sparse features than they have dimensions, which makes neurons polysemantic. The module reproduces the toy model of superposition (features learned grow from 5 to 12 in five dimensions as sparsity rises), then builds sparse autoencoders in NumPy with hand-derived gradients: L1 versus TopK, shrinkage, dead latents, transpose initialization and an AuxK loss. Recovery is scored against planted ground truth (mean max cosine), and the GPU lab trains a TopK SAE on the learner's own GPT residual stream with feature dashboards.
Progress — loading…
Objectives
Language
- Explain polysemantic neurons and why language's many rare concepts favour superposition
- Build and read feature dashboards; know why top activations are correlational
Math & data structures
- Dimensions per feature; why sparsity and a nonlinearity are both needed
- SAE objectives (L1 vs TopK), soft-threshold shrinkage, gradients with a bias on two paths
- Evaluation: mean max cosine, normalized MSE, L0, dead fraction
Code
- nlpcore/interp/sae.py: toy model, planted data, SAE training (L1/TopK/AuxK), recovery, shrinkage, dashboards
- Break-it: the L1 rescaling loophole without decoder renormalization; dead latents without transpose init
- PyTorch port trained on a GPT residual stream (GPU tier)
Episodes
- Sparse Autoencoders: Finding Features16 min · 28 cues
Superposition, dictionary learning, L1 vs TopK SAEs, feature interpretation.
Exercises
| Exercise | Tier | Ladder | Time |
|---|---|---|---|
| A TopK sparse autoencoder from scratch that recovers planted features | browser | Predict · Scratch · Compose · Probe a claim | 75 min |
| Break it: the L1 cheat without decoder normalization, and dead latents without transpose init | browser | Break it · Harden · Probe a claim | 60 min |
| Port the TopK SAE to PyTorch and train it on your GPT's residual stream | gpu | Port · Compose · Probe a claim | 120 min |
Labs
- Sparse Autoencoders: Finding Features browser F35.01 lab: superposition in a toy model, then a sparse autoencoder that recovers planted features. Data is synthetic (planted directions) so recovery can be scored against ground truth; a small real-English probe at the end applies the same SAE to character-context vectors. Exercises: F35.ex01-sae-from-scratch, F35.ex02-sae-break-it, F35.ex03-sae-on-your-gpt-torch (GPU). · 90 min
Library contributions
Files in nlpcore this module teaches you to write.
Sources
- Bricken et al. Towards Monosemanticity (Anthropic, transformer-circuits.pub, Oct 2023)
- This course, Interpretability (probes, logit lens, toy SAE)
- Cunningham et al. Sparse Autoencoders Find Highly Interpretable Features in Language Models (2023) · arXiv:2309.08600
- Elhage, Hume, Olsson, … , Wattenberg, Olah. Toy Models of Superposition (21 Sep 2022) · arXiv:2209.10652
- Gao, Dupré la Tour, Tillman, Goh, Troll, Radford, Sutskever, Leike, Wu. Scaling and evaluating sparse autoencoders (6 Jun 2024) · arXiv:2406.04093