stage S86h
Automated Red Teaming & Fuzzing (PyRIT / Garak)
Run a compact, deterministic 20–50 test security regression suite on every PR (`D8`), and schedule larger PyRIT/Garak fuzzing campaigns nightly or pre-release. Gate releases on severity and reproducibility, not arbitrary refusal percentages.
Stage detail
When: Small fixed tests Weeks 4--6; CI gates Week 8.
- Adversarial test design, provenance, reproducibility, severity, and false positives.
- Explore PyRIT and Garak after you have a working target and baseline tests.
- Generate variations for injection, tool misuse, data leakage, malformed inputs, and resource exhaustion.
- Add deterministic security regressions to CI. Run expensive fuzz campaigns nightly or before release rather than making every PR run 1,000 model calls.
- Define thresholds from a measured baseline; a universal "jailbreak rate below 1%" is meaningless without a defined dataset, task, model, and severity.
Practice: run 20--50 deterministic tests on each PR; schedule larger batches nightly or pre-release. Track attack success, impact severity, benign task success, false positives, latency, and cost.
Do not learn yet: large-scale fuzzing before deterministic tests, or noisy/expensive CI gates that teams routinely bypass.
Study days for this stage
5- 50Build a compact security regression suite for prompt injectionW9S8M8D8R04 R28 R29
- 51PyRIT / Garak Scheduled FuzzingW9S8M8D8R28 R29 R31 R42
- 89Create a unified attack taxonomy across projectsW15S1S8D1D2D3D4D5D6D7D8D9D10D11D12R05 R39 R41
- 90Run direct, indirect, multi-turn, encoded, split-payload, and…W15S2S8D1D8R04 R05 R28
- 93Prioritize findings by impact and exploitabilityW16S8S11D8D11D12R31 R42