Server-Verified Citations & Abstention

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R14 — Ragas faithfulness & context precision

   R15 — BEIR retrieval evaluation

   R05 — MITRE ATLAS poisoning

   R45 — NIST AI 100-2 Adversarial ML taxonomy

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Implement D5 citation verification using server-owned document_id/chunk_id mappings and abstention when evidence score is below threshold. *Lab:* Test citation spoofing and fabricated model URLs.

   Study index poisoning, metadata tampering, and stale permissions. Insert a labeled synthetic poisoned document into a disposable corpus and measure retrieval rank and answer distortion.

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   No separate task text for this block on day 21 — the sourced work sits in the block above. Keep the time box.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   D5 citation provenance verifier + poisoned-document influence test report.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R05MITRE ATLAS↗ atlas.mitre.org

  ML attack tactics/techniques: prompt injection, data poisoning, model extraction, and exfiltration.

• R14Ragas Documentation↗ docs.ragas.io

  RAG evaluation metrics: context precision, context recall, faithfulness, and answer relevance.

• R15BEIR Benchmark↗ github.com

  Retrieval evaluation methodology (nDCG@k, Recall@k) and why a single metric is insufficient.

• R45NIST AI 100-2 (Adversarial ML)↗ doi.org

  Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (evasion, poisoning, privacy, abuse).

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsRagas Documentation · metric reference
  List of available metrics

  Defines context precision, context recall, faithfulness and response relevancy separately, the split you need to score abstention and citation accuracy without one blended number.

  R14checked 2026-11-11
• docsNIST (AI 100-2e2025) · technical report
  A Taxonomy and Terminology of Attacks and Mitigations

  Organises poisoning and evasion by attacker goal and life-cycle stage, giving the vocabulary to classify index poisoning and metadata tampering as specific AML categories.

  R45checked 2026-11-11
• videoLogical Lenses · 12 min
  RAG Evaluation: Precision, Recall, Faithfulness, RAGAS Explained Clearly

  Breaks down why similarity metrics like BLEU/ROUGE fail and how faithfulness as claim-support is computed, the mechanism behind grounding checks in citation verification.

  R14checked 2026-11-11
• videoMervin Praison · 9 min
  RAGAS: How to Evaluate a RAG Application Like a Pro for Beginners

  Shows running Ragas metrics over a pipeline end-to-end, a practical template for scoring your synthetic poisoned-corpus experiment before and after insertion.

  R14checked 2026-11-11
• repoBEIR (GitHub) · benchmark code + datasets
  beir-cellar/beir: A Heterogeneous Benchmark for Information Retrieval

  Provides nDCG@k and Recall@k implementations across heterogeneous datasets, so retrieval rank shifts from a poisoned document are measured with a standard metric rather than eyeballing.

  R15checked 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.