Incident Playbooks & Tabletop Exercise
NIST AI RMF Full Mapping
Jump & context
Week 17: Runaway Agent Drill, Incident Playbooks & Comprehensive AI Governance Mapping
The five hours
- 01
Concepts & system design
60 min· read · map the mechanismR31 — NIST AI RMF
R27 — OpenTelemetry incident telemetry
R42 — NIST AI 600-1 GenAI Profile
Read the assigned sections; map the mechanism, threat model and trade-offs.
- 02
Project implementation
120 min· build · integrate · debugConsolidate the 4 incident drills into formal runbooks (detection, containment, eradication, recovery, privacy-safe evidence retention, and post-mortem). Validate
P1'sC8diagnostic assistant against the runbooks.Map
P2andP3across all four NIST AI RMF functions (Govern,Map,Measure,Manage) and the NIST GenAI Profile, linking each control to concrete code and test artifacts in your repositories.Build, integrate, debug and improve the increment in the active project.
- 03
Security lab
60 min· attack your own systemNo separate task text for this block on day 98 — the sourced work sits in the block above. Keep the time box.
Test the abuse case, failure path, authorization bypass or resource limit.
- 04
Verification & evidence
30 min· test · record evidenceComplete AI Incident Playbook + NIST AI RMF & GenAI Profile evidence matrix.
Add or run regression tests; record the 9-point evidence record.
- 05
Deliberate practice & Socratic review
30 min· explain aloud · update gapsAnswer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?
Re-explain the hardest concept aloud, inspect one edge case, update the gap log.
- total5 h
Read today · inside the concepts block
R27 R31 R42Traces, spans, context propagation, semantic conventions, and redacting sensitive attributes.
Govern, Map, Measure, and Manage functions; risk ownership, measurement, and lifecycle documentation.
NIST AI 600-1: generative AI risks and suggested actions for confabulation, privacy, information integrity, and value chain.
Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.
Free resources for this day
5 curated- docsOpenTelemetryTraces
Defines spans, parent-child relationships and context propagation, the structure that lets one incident timeline stitch agent steps to tool side effects across services.
R27checked 2026-11-11 - videoBlack Hills Information Security · 59:04BHIS | How to Use Backdoors & Breaches to do Tabletop Exercises and Learn Cybersecurity
Demonstrates running a facilitated tabletop with an inject deck, the format for rehearsing your four drills and finding runbook gaps before a real incident.
R31checked 2026-11-11 - videoLightstep is now ServiceNow Cloud Observability · 9:40Context Propagation makes OpenTelemetry awesome
Explains how trace context travels across process and service boundaries, the mechanism that keeps an agent's evidence chain unbroken from request to tool call.
R27checked 2026-11-11 - articleOpenTelemetryContext propagation
Details inject and extract with traceparent so you can verify correlation survives queue handoffs and retries, where incident timelines usually break.
R27checked 2026-11-11 - bookGoogle SRE Book (O'Reilly, free online)Postmortem Culture: Learning from Failure
Covers blameless postmortems and what a useful one records, the template for the post-mortem section of each runbook and for retaining evidence without storing sensitive data.
R31checked 2026-11-11
Evidence record · 9-point standard
A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.
Day notes
emptyThe two source formulations
verbatim · nothing summarisedProject & Stage Target
All Projects (C8) | S11 (Governance & Incident Playbooks)
Focus 1 (Incident Playbooks & Tabletop Exercise — Orig. Day 126)
Consolidate the 4 incident drills into formal runbooks (detection, containment, eradication, recovery, privacy-safe evidence retention, and post-mortem). Validate P1's C8 diagnostic assistant against the runbooks.
Focus 2 (NIST AI RMF Full Mapping — Orig. Day 127)
Map P2 and P3 across all four NIST AI RMF functions (Govern, Map, Measure, Manage) and the NIST GenAI Profile, linking each control to concrete code and test artifacts in your repositories.
Verification & Evidence
Complete AI Incident Playbook + NIST AI RMF & GenAI Profile evidence matrix.
Read Today (60m Concepts Block)
R31 (NIST AI RMF), R27 (OpenTelemetry incident telemetry), R42 (NIST AI 600-1 GenAI Profile).
---