PyRIT / Garak Scheduled Fuzzing

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R28 — PyRIT orchestrators

   R29 — Garak documentation

   R31 — NIST AI RMF Manage/Measure

   R42 — NIST GenAI Profile

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Configure PyRIT and/or Garak in a controlled local environment. Keep the fast deterministic subset on PRs, and wire broader multi-turn/fuzzing campaigns into a scheduled nightly or pre-release workflow (D8).

   Execute a full release drill: introduce a failing security regression, verify D8 blocks the release, execute a documented rollback of a model/prompt change, and archive build-specific evidence.

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   No separate task text for this block on day 51 — the sourced work sits in the block above. Keep the time box.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   D8 PR vs. nightly pipeline config + recorded release-block and rollback drill log.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R28Microsoft PyRIT↗ github.com

  Python Risk Identification Tool: targets, orchestrators, converters, scorers, and multi-turn attack automation.

• R29Garak Documentation↗ garak.ai

  LLM vulnerability scanner architecture: probes, generators, detectors, and buffs (authorized targets only).

• R31NIST AI RMF 1.0↗ nist.gov

  Govern, Map, Measure, and Manage functions; risk ownership, measurement, and lifecycle documentation.

• R42NIST AI RMF GenAI Profile↗ doi.org

  NIST AI 600-1: generative AI risks and suggested actions for confabulation, privacy, information integrity, and value chain.

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsNIST
  NIST AI Risk Management Framework

  The Govern/Map/Measure/Manage structure that the nightly fuzzing campaign must report into, so scan results become measured risk rather than an unowned test log.

  R31checked 2026-11-11
• videoNull:404 Cyber Security
  PyRIT Setup Workshop: AI Red Teaming on Windows

  A full local PyRIT install-and-run workshop including a live prompt-injection demo, which is the controlled environment the scheduled campaign needs.

  R28checked 2026-11-11
• videoEmbrace The Red
  LLM Vulnerability Scanning with garak. Tutorial: Test your own chat bots!

  Targets your own REST-based LLM app rather than a base model, matching the release drill where garak must exercise the deployed endpoint, not a raw checkpoint.

  R29checked 2026-11-11
• articlenhimg.org
  garak Guide: LLM Vulnerability Scanning, Probes and CI

  Covers scoped probe runs, JSONL/HTML reports, OWASP tags, and CI usage — the mechanics for keeping a fast subset on PRs and a broad set on a schedule.

  R29checked 2026-11-11
• labgarak docs
  Your first scan

  Guided first scan against a small local model with probe listing and result interpretation, the smallest reproducible prerequisite for a nightly campaign.

  R29checked 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.