Skip to content
Roadmap0/111
stage S108h

Mechanistic Interpretability & Alignment Research

Use TransformerLens to reproduce one Logit Lens and one Activation Patching experiment locally. Distinguish correlational activations from causal mechanisms, and document why head patching does not replace application security.

Stage detail

When: Weeks 14 (8-hour bounded investigation). This is a research specialization, so the required outcome is a reproducible experiment and a careful explanation of limits---not research-level mastery.

  • Learn the purpose of activation patching, logit lens, circuit discovery, and TransformerLens.
  • Understand the difference between activation correlations and evidence of a causal mechanism.
  • Explore reward hacking and sycophancy as research/evaluation questions.
  • If interested, reproduce one small published experiment on an open-weight model and document limitations.

Do not make this a core milestone. Finding attention heads responsible for "deception" and patching them to neutralize it is not a reliable general-purpose engineering recipe. These behaviors are difficult to define and attribute; interventions can have unintended effects.