Skip to content
Roadmap0/111
not startedstudy day 6 of 6 · covers original day 15

Implement token counting

P1 AssistantM5
not started0/5 blocks
0/5
Jump & context

Week 2: Prompt Injection, Crescendo Fuzzing, Streaming, Persistence & Cost Controls

The five hours

  1. 01

    Concepts & system design

    60 min

    R04 — OWASP LLM10: Unbounded Consumption

    R03 — OWASP API4: Unrestricted Resource Consumption

    Read the assigned sections; map the mechanism, threat model and trade-offs.

  2. 02

    Project implementation

    120 min

    Implement strict input token counting, history truncation, max_tokens output caps, and per-request context budgeting.

    Build, integrate, debug and improve the increment in the active project.

  3. 03

    Security lab

    60 min

    Study denial-of-wallet and cost amplification; test oversized inputs and rapid context-stuffing requests.

    Test the abuse case, failure path, authorization bypass or resource limit.

  4. 04

    Verification & evidence

    30 min

    Per-request token budget design and rejection tests for oversized payloads.

    Add or run regression tests; record the 9-point evidence record.

  5. 05

    Deliberate practice & Socratic review

    30 min

    Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

    Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

  6. total5 h

Read today · inside the concepts block

R03 R04

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

Free resources for this day

5 curated
  • docsOWASP API Security Top 10
    API4:2023 Unrestricted Resource Consumption

    Frames missing max_tokens, missing history caps, and per-request cost as an API-level denial-of-service and economic abuse risk, with the control set you implement as a context budget.

    R03M5moved 2026-11-11
  • videoIBM Technology · 11:31
    What is a Context Window? Unlocking LLM Secrets

    Explains the token window as a fixed budget shared by system prompt, history, retrieved content, and reserved output, which is the arithmetic behind choosing a truncation point.

    M5checked 2026-11-11
  • videoAI Engineer · 16:17
    How we solved Context Management in Agents — Sally-Ann Delucia

    Describes production strategies for keeping history inside the window (truncate, summarize, offload) and the failure modes of each, which is the tradeoff your truncation policy encodes.

    M5S7checked 2026-11-11
  • articleLangChain Blog
    Context Engineering

    Taxonomizes context strategies (write, select, compress, isolate) and explains compaction and summarization, giving you named options instead of an ad hoc character cut.

    M5moved 2026-11-11
  • labOpenAI Cookbook
    How to count tokens with Tiktoken

    Runnable code that counts message-array tokens including per-message overhead, which is the measurement you need before enforcing a max input budget.

    M5R03moved 2026-11-11

Evidence record · 9-point standard

0/9

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.

Day notes

empty

The two source formulations

verbatim · nothing summarised

Project & Stage Target

P1 | M5 (DoS & Economic Abuse)

Core Implementation

Implement strict input token counting, history truncation, max_tokens output caps, and per-request context budgeting.

Security Lab

Study denial-of-wallet and cost amplification; test oversized inputs and rapid context-stuffing requests.

Verification & Evidence

Per-request token budget design and rejection tests for oversized payloads.

Read Today (60m Concepts Block)

R04 (OWASP LLM10: Unbounded Consumption), R03 (OWASP API4: Unrestricted Resource Consumption).

---