Advanced Privacy-Preserving ML (DP/MI)
Understand model memorization, membership inference, model inversion, and embedding leakage. Run a controlled membership-inference demo and an Opacus Differential Privacy (`DP-SGD`) experiment to measure $(\epsilon, \delta)$ vs. utility trade-offs. Verify end-to-end deletion across all stores.
Module detail
Best time to learn: After the original PII proxy and model extraction/privacy stages. This module distinguishes application-level privacy controls from model-level information leakage.
Prerequisites - PII classification and masking. - Basic model training or fine-tuning concepts. - Statistics and evaluation metrics. - Familiarity with membership inference and model inversion concepts.
### Threat model
Sensitive information can leak from training data, embeddings, model outputs, logs, caches, and shared infrastructure. Removing obvious PII from prompts does not establish that the model has not memorized sensitive examples.
Study: - Memorization and training-data regurgitation. - Membership inference. - Model inversion and reconstruction risks. - Embedding inversion and semantic leakage. - Privacy risks in fine-tuning and retrieval. - Differential privacy: adjacency, clipping, noise, privacy budget, epsilon/delta, and utility trade-offs. - Federated learning threat model: client poisoning, update leakage, and secure aggregation concepts. - Machine unlearning: deletion claims, retraining, and verification limitations. - Cache and cross-tenant information leakage.
### Labs
- Privacy threat model (1h): Identify sensitive data at ingestion, storage, embedding, prompt construction, model output, logs, and caches.
- Membership inference demonstration (2h): Use a small controlled dataset and toy or published demonstration. Explain the attack signal and false-positive limitations.
- Embedding privacy (1h): Examine what metadata and semantic information can be inferred from stored embeddings; test access control and minimization.
- Differential privacy experiment (2h): Run a small DP example or documented library tutorial. Compare privacy parameters, utility, and assumptions.
- Deletion and cache test (2h): Delete a synthetic user's data and verify primary storage, vector index, caches, and logs follow the declared deletion policy.
### Project integration
- Project 1: Redact sensitive telemetry and define conversation/memory retention.
- Project 2: Enforce source ACLs before retrieval; test embedding-store isolation and deletion propagation.
- Training/fine-tuning workflow: Record whether data is suitable for training, how consent/provenance is represented, and how deletion requests are handled.
### Measurable completion criteria
- You can distinguish PII masking, differential privacy, access control, and machine unlearning.
- A small privacy attack demonstration includes assumptions and limitations.
- Deletion tests cover derived data stores and caches.
- You can explain the privacy/utility trade-off of a DP configuration.
- No claim of "privacy guaranteed" is based solely on masking or a single experiment.
Do not overreach: Do not claim production-grade DP or federated-learning expertise from one tutorial. Understand the threat model, assumptions, and measured trade-offs.
------------------------------------------------------------------------