Skip to content
Roadmap0/111
not startedstudy day 5 of 6 · covers original days 130 & 131

Enterprise Audit & Retention Policy

Model & Prompt Change-Management Policy

P1 AssistantP2 RAG platformP3 Agent systemS11M8D8D11D12
not started0/5 blocks
0/5
Jump & context

Week 17: Runaway Agent Drill, Incident Playbooks & Comprehensive AI Governance Mapping

The five hours

  1. 01

    Concepts & system design

    60 min

    R27 — OpenTelemetry

    R38 — RFC 8785

    R31 — NIST AI RMF

    R42 — NIST GenAI Profile

    Read the assigned sections; map the mechanism, threat model and trade-offs.

  2. 02

    Project implementation

    120 min

    Finalize the cross-project Audit Policy around D11: mandatory event schemas, RFC 8785 canonicalization, hash-chain verification schedule, PII redaction, retention TTLs, and investigator access controls.

    Define strict change-control gates: any change to a model checkpoint, provider, system prompt, tool schema, or RAG dataset requires D12 provenance verification, a passing D8 security/eval run, owner sign-off, and a tested rollback target.

    Build, integrate, debug and improve the increment in the active project.

  3. 03

    Security lab

    60 min

    No separate task text for this block on day 101 — the sourced work sits in the block above. Keep the time box.

    Test the abuse case, failure path, authorization bypass or resource limit.

  4. 04

    Verification & evidence

    30 min

    Published Audit Policy (D11) and AI Change-Management Policy (D8/D12).

    Add or run regression tests; record the 9-point evidence record.

  5. 05

    Deliberate practice & Socratic review

    30 min

    Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

    Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

  6. total5 h

Read today · inside the concepts block

R27 R31 R38 R42

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

Free resources for this day

5 curated
  • docsRFC Editor / IETF · ~20 pages
    RFC 8785: JSON Canonicalization Scheme (JCS)

    Defines deterministic key sorting and ECMAScript number serialization, the property that makes a ledger entry hash to the same value on every machine — the precondition for any hash-chain verification of D11.

    R38moved 2026-11-11
  • videoLightstep is now ServiceNow Cloud Observability · 9 min
    Context Propagation makes OpenTelemetry awesome

    Explains how trace and span context is injected and extracted across service boundaries, which is what lets a single ledger event be correlated to the full retrieval, rerank, and generation path.

    R27checked 2026-11-11
  • videoDevOps & AI Toolkit · 22 min
    Distributed Tracing Explained: OpenTelemetry & Jaeger Tutorial

    Covers auto versus manual instrumentation, span parent-child structure, semantic conventions, and sampling — the mechanics behind the span attributes you will redact before they reach the audit store.

    R27checked 2026-11-11
  • articleDmitrii Zatona · 20 min read
    Tamper-Evident Audit Logs: Hash Chains, Merkle Trees, and External Anchoring

    Separates what a hash chain actually proves from what Merkle checkpoints and external anchoring add, including the truncation and trusted-head limits that decide your hash-chain verification schedule.

    R31R38checked 2026-11-11
  • repoGitHub (pernebye) · Small MIT repo + tests
    hash-chain-ledger

    A dependency-free SHA-256 hash chain over canonical JSON with verifyChain, including the truncation and authorship caveats — a directly reusable reference for the D11 append-and-verify path.

    R38checked 2026-11-11

Evidence record · 9-point standard

0/9

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.

Day notes

empty

The two source formulations

verbatim · nothing summarised

Project & Deliverable Target

All Projects | `D8` (PyRIT Gate), `D11` (Trajectory Ledger), `D12` (SBOM Scanner) | S11, M8

Focus 1 (Enterprise Audit & Retention Policy — Orig. Day 130)

Finalize the cross-project Audit Policy around D11: mandatory event schemas, RFC 8785 canonicalization, hash-chain verification schedule, PII redaction, retention TTLs, and investigator access controls.

Focus 2 (Model & Prompt Change-Management Policy — Orig. Day 131)

Define strict change-control gates: any change to a model checkpoint, provider, system prompt, tool schema, or RAG dataset requires D12 provenance verification, a passing D8 security/eval run, owner sign-off, and a tested rollback target.

Verification & Evidence

Published Audit Policy (D11) and AI Change-Management Policy (D8/D12).

Read Today (60m Concepts Block)

R27 (OpenTelemetry), R38 (RFC 8785), R31 (NIST AI RMF), R42 (NIST GenAI Profile).

---