Enterprise Audit & Retention Policy
Model & Prompt Change-Management Policy
Jump & context
Week 17: Runaway Agent Drill, Incident Playbooks & Comprehensive AI Governance Mapping
The five hours
- 01
Concepts & system design
60 min· read · map the mechanismR27 — OpenTelemetry
R38 — RFC 8785
R31 — NIST AI RMF
R42 — NIST GenAI Profile
Read the assigned sections; map the mechanism, threat model and trade-offs.
- 02
Project implementation
120 min· build · integrate · debugFinalize the cross-project Audit Policy around
D11: mandatory event schemas, RFC 8785 canonicalization, hash-chain verification schedule, PII redaction, retention TTLs, and investigator access controls.Define strict change-control gates: any change to a model checkpoint, provider, system prompt, tool schema, or RAG dataset requires
D12provenance verification, a passingD8security/eval run, owner sign-off, and a tested rollback target.Build, integrate, debug and improve the increment in the active project.
- 03
Security lab
60 min· attack your own systemNo separate task text for this block on day 101 — the sourced work sits in the block above. Keep the time box.
Test the abuse case, failure path, authorization bypass or resource limit.
- 04
Verification & evidence
30 min· test · record evidencePublished Audit Policy (
D11) and AI Change-Management Policy (D8/D12).Add or run regression tests; record the 9-point evidence record.
- 05
Deliberate practice & Socratic review
30 min· explain aloud · update gapsAnswer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?
Re-explain the hardest concept aloud, inspect one edge case, update the gap log.
- total5 h
Read today · inside the concepts block
R27 R31 R38 R42Traces, spans, context propagation, semantic conventions, and redacting sensitive attributes.
Govern, Map, Measure, and Manage functions; risk ownership, measurement, and lifecycle documentation.
JSON Canonicalization Scheme: deterministic key sorting and number formatting required before cryptographic hashing.
NIST AI 600-1: generative AI risks and suggested actions for confabulation, privacy, information integrity, and value chain.
Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.
Free resources for this day
5 curated- docsRFC Editor / IETF · ~20 pagesRFC 8785: JSON Canonicalization Scheme (JCS)
Defines deterministic key sorting and ECMAScript number serialization, the property that makes a ledger entry hash to the same value on every machine — the precondition for any hash-chain verification of D11.
R38moved 2026-11-11 - videoLightstep is now ServiceNow Cloud Observability · 9 minContext Propagation makes OpenTelemetry awesome
Explains how trace and span context is injected and extracted across service boundaries, which is what lets a single ledger event be correlated to the full retrieval, rerank, and generation path.
R27checked 2026-11-11 - videoDevOps & AI Toolkit · 22 minDistributed Tracing Explained: OpenTelemetry & Jaeger Tutorial
Covers auto versus manual instrumentation, span parent-child structure, semantic conventions, and sampling — the mechanics behind the span attributes you will redact before they reach the audit store.
R27checked 2026-11-11 - articleDmitrii Zatona · 20 min readTamper-Evident Audit Logs: Hash Chains, Merkle Trees, and External Anchoring
Separates what a hash chain actually proves from what Merkle checkpoints and external anchoring add, including the truncation and trusted-head limits that decide your hash-chain verification schedule.
R31R38checked 2026-11-11 - repoGitHub (pernebye) · Small MIT repo + testshash-chain-ledger
A dependency-free SHA-256 hash chain over canonical JSON with verifyChain, including the truncation and authorship caveats — a directly reusable reference for the D11 append-and-verify path.
R38checked 2026-11-11
Evidence record · 9-point standard
A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.
Day notes
emptyThe two source formulations
verbatim · nothing summarisedProject & Deliverable Target
All Projects | `D8` (PyRIT Gate), `D11` (Trajectory Ledger), `D12` (SBOM Scanner) | S11, M8
Focus 1 (Enterprise Audit & Retention Policy — Orig. Day 130)
Finalize the cross-project Audit Policy around D11: mandatory event schemas, RFC 8785 canonicalization, hash-chain verification schedule, PII redaction, retention TTLs, and investigator access controls.
Focus 2 (Model & Prompt Change-Management Policy — Orig. Day 131)
Define strict change-control gates: any change to a model checkpoint, provider, system prompt, tool schema, or RAG dataset requires D12 provenance verification, a passing D8 security/eval run, owner sign-off, and a tested rollback target.
Verification & Evidence
Published Audit Policy (D11) and AI Change-Management Policy (D8/D12).
Read Today (60m Concepts Block)
R27 (OpenTelemetry), R38 (RFC 8785), R31 (NIST AI RMF), R42 (NIST GenAI Profile).
---