Activation Patching Experiment

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R34 — TransformerLens Activation Patching & Exploratory Analysis

   R45 — NIST AI 100-2

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Reproduce a published Activation Patching (causal tracing) experiment on a clean vs. corrupted prompt pair (e.g., Indirect Object Identification). Record the causal intervention, control condition, logit difference metric, and alternative explanations.

   Study introductory circuit analysis and feature attribution. Analyze how causal claims depend heavily on prompt template, corrupt baseline choice, and task definition.

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   No separate task text for this block on day 68 — the sourced work sits in the block above. Keep the time box.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   Reproducible activation-patching notebook + critique of single-head causal claims.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R34TransformerLens Docs↗ transformerlensorg.github.io

  HookedTransformer, residual stream, ActivationCache, Logit Lens, and Activation Patching tutorials.

• R45NIST AI 100-2 (Adversarial ML)↗ doi.org

  Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (evasion, poisoning, privacy, abuse).

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsNIST CSRC · reference
  AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations | CSRC

  Sets out what counts as evidence for an adversarial claim versus a correlational observation, the standard you apply when stating what an activation-patching result does and does not establish.

  R45checked 2026-11-11
• articleLessWrong · ~25 min read
  How to use and interpret activation patching — LessWrong

  Explains clean versus corrupted prompt pairs, noising versus denoising direction, and how corrupt baseline choice and prompt template change conclusions, which is exactly what the IOI replication must document.

  R34checked 2026-11-11
• videoMartinHanderPhD · 31 min
  Activation Patching & Feature Steering: How to Edit an AI's Thoughts

  Implements patching of activations between runs with a logit-difference metric and controls, giving you the mechanics to reproduce a causal-tracing experiment on your clean/corrupted pair.

  R34checked 2026-11-11
• videoNeel Nanda · 38 min
  A Walkthrough of Automated Circuit Discovery w/ Arthur Conmy Part 1/3

  Walks circuit analysis and edge attribution over patching results, showing how a circuit claim is assembled from many causal interventions and where alternative explanations creep in.

  R34checked 2026-11-11
• repoGitHub · replication code + notebooks
  GitHub - canrager/indirect_object_identification: Replicating the results of the paper "Interpretability in the Wild"

  Provides a full IOI replication with patching notebooks and metric definitions, the known-good reference for your clean-versus-corrupted prompt pair and logit-difference measurement.

  R34checked 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.