Study alignment, sycophancy, refusal robustness, and behavioral…

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R31 — NIST AI RMF measurement

   R42 — NIST GenAI Profile

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Study alignment concepts: sycophancy, reward hacking, refusal robustness, and behavioral vs. mechanistic evaluations.

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   Compare a behavioral evaluation test (prompting for sycophancy/refusal) against internal activation observations; document why you must not claim a production "sycophancy-neutralization patch" from simple head ablation.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   Comparative analysis note distinguishing behavioral eval evidence from mechanistic evidence.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R31NIST AI RMF 1.0↗ nist.gov

  Govern, Map, Measure, and Manage functions; risk ownership, measurement, and lifecycle documentation.

• R42NIST AI RMF GenAI Profile↗ doi.org

  NIST AI 600-1: generative AI risks and suggested actions for confabulation, privacy, information integrity, and value chain.

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsNIST · reference
  AI Risk Management Framework | NIST

  Supplies the Measure and Manage functions that tell you how to treat behavioral versus mechanistic evidence as different measurement instruments with different confidence.

  R31checked 2026-11-11
• videoAnthropic · 6 min
  What is sycophancy in AI models?

  Defines sycophancy as agreement pressure from human feedback and shows how it appears in outputs, the behavioral failure mode you contrast with mechanistic evidence inside the model.

  R42checked 2026-11-11
• videoRobert Miles AI Safety · 10 min
  9 Examples of Specification Gaming

  Catalogs reward-hacking behaviors where the model satisfies the metric instead of the intent, giving concrete cases for why behavioral evals can pass while the objective is still gamed.

  R42checked 2026-11-11
• articlearXiv (Anthropic) · ~20 min read
  [2310.13548] Towards Understanding Sycophancy in Language Models

  Measures how RLHF models change answers when a user pushes back across math, policy and factual tasks, the empirical basis for treating refusal robustness as a behavioral eval separate from internals.

  R42checked 2026-11-11
• articlearXiv (Anthropic) · ~25 min read
  [2406.10162] Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models

  Shows reward tampering emerging from specification gaming curricula, the escalation path from sycophancy to tampering that explains why behavioral and mechanistic evaluations are both needed.

  R31R42checked 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.