Study membership inference

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R17 — NIST Privacy Framework

   R45 — NIST AI 100-2 privacy attacks: membership inference & model inversion

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Study model memorization, training-data regurgitation, membership inference (MI), model inversion, and embedding inversion risks.

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   Run a controlled membership-inference demonstration on a local toy/small model; measure loss/confidence threshold signals and document the false-positive rate and threat-model assumptions.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   Reproducible membership-inference experiment script and threat-model analysis.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R17NIST Privacy Framework↗ nist.gov

  Identify-P, Govern-P, Control-P, Communicate-P, and Protect-P lifecycle controls.

• R45NIST AI 100-2 (Adversarial ML)↗ doi.org

  Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (evasion, poisoning, privacy, abuse).

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsNIST CSRC
  AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations

  Defines membership inference, model inversion, and data-extraction privacy attacks with agreed terminology and mitigation limits, the vocabulary you need to write the threat model.

  R45checked 2026-11-11
• videoBirhanu Eshete
  Lecture 5: Membership Inference Attacks

  A lecture walkthrough of shadow-model membership inference and the confidence/overfitting signal it exploits, which is the attack you must reason about for user-supplied training data.

  R45checked 2026-11-11
• videoUCL Centre for Artificial Intelligence
  Membership and Property Inference Attacks against Machine Learning Models

  Extends membership inference to property inference, showing how aggregate model behaviour leaks dataset-level attributes rather than only individual records.

  R45checked 2026-11-11
• articleIronCore Labs
  There and Back Again: An Embedding Attack Journey

  Demonstrates practical reversal of text embeddings back into sentences, making the case that a vector store is not a one-way hash and must be treated as sensitive data.

  R45checked 2026-11-11
• repoJina AI (GitHub)
  embedding-inversion-demo

  Runnable reconstruction of original text from an embedding vector alone, the hands-on check that shows how much of a chunk survives embedding before you store it.

  R45checked 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.