Model Extraction Threat Model

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R04 — OWASP LLM10: Model Extraction/Theft

   R05 — MITRE ATLAS model extraction

   R45 — NIST AI 100-2

   R03 — OWASP API rate limiting

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Threat-model query-based model extraction/distillation, logit/logprob harvesting, systematic hyperparameter sweeps, and watermarking limitations.

   Run a controlled extraction-style query sweep against your own local endpoint. Implement per-API-key quotas, logprob stripping, and query-pattern anomaly alerts. *(Never probe third-party APIs).*

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   No separate task text for this block on day 54 — the sourced work sits in the block above. Keep the time box.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   Extraction threat model + anomaly detection and rate-limit test report on local endpoint.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R03OWASP API Security Top 10↗ owasp.org

  BOLA (API1), Broken Auth (API2), Unrestricted Resource Consumption (API4), SSRF (API7), Misconfiguration (API8).

• R04OWASP Top 10 for LLM Apps↗ genai.owasp.org

  Prompt injection, sensitive info disclosure, improper output handling, excessive agency, vector weaknesses, unbounded consumption.

• R05MITRE ATLAS↗ atlas.mitre.org

  ML attack tactics/techniques: prompt injection, data poisoning, model extraction, and exfiltration.

• R45NIST AI 100-2 (Adversarial ML)↗ doi.org

  Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (evasion, poisoning, privacy, abuse).

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsMITRE
  MITRE ATLAS

  The tactic/technique matrix that names exfiltration via AI inference API, giving you the technique IDs to cite for query-based extraction in the threat model.

  R05checked 2026-11-11
• videoAiSecurityDIR
  Model Extraction Attacks: How Attackers Steal AI | AiSecurityDIR

  Explains functional extraction from API queries and the economics of cloning a valuable model for a few thousand dollars, the business case for quota and logprob stripping.

  R05checked 2026-11-11
• videoSecureTechIn
  AI Security Lesson 38: Model Inversion Explained for Teams

  Walks reconstruction of private training records from confidence outputs, showing why per-request logits and probabilities must not be returned to untrusted callers.

  R45checked 2026-11-11
• articlePraetorian
  Stealing AI Models Through the API: A Practical Model Extraction Attack

  A measured extraction run reaching high replica fidelity on ~1,000 queries, giving you realistic query volumes and sweep shapes for your own authorized local test.

  R03checked 2026-11-11
• labGitHub
  aipromptengineering

  Offline lab exercises covering model extraction alongside poisoning and adversarial examples, mapped to NIST AI RMF and ATLAS, for the controlled local sweep.

  R45checked 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.