stage S108h
Mechanistic Interpretability & Alignment Research
Use TransformerLens to reproduce one Logit Lens and one Activation Patching experiment locally. Distinguish correlational activations from causal mechanisms, and document why head patching does not replace application security.
Stage detail
When: Weeks 14 (8-hour bounded investigation). This is a research specialization, so the required outcome is a reproducible experiment and a careful explanation of limits---not research-level mastery.
- Learn the purpose of activation patching, logit lens, circuit discovery, and TransformerLens.
- Understand the difference between activation correlations and evidence of a causal mechanism.
- Explore reward hacking and sycophancy as research/evaluation questions.
- If interested, reproduce one small published experiment on an open-weight model and document limitations.
Do not make this a core milestone. Finding attention heads responsible for "deception" and patching them to neutralize it is not a reliable general-purpose engineering recipe. These behaviors are difficult to define and attribute; interventions can have unintended effects.
Study days for this stage
6- 66Set up a small open model and TransformerLens-compatible…W11S10R34
- 67Reproduce a documented logit-lens experimentW12S10R34
- 68Activation Patching ExperimentW12S10R34 R45
- 69Study alignment, sycophancy, refusal robustness, and behavioral…W12S10R31 R42
- 70Write a short reproducibility reportW12S10R34 R42
- 111Consolidated Security PortfolioW19S10S12D1D2D3D4D5D6D7D8D9D10D11D12R28 R31 R34 R35 R42