Study inference gateway versus model-server architecture

1. 01

   Concepts & system design

   60 min· read · map the mechanismtimer

   R26 — vLLM production deployment & API server architecture

   R25 — Docker networking & daemon security

   Read the assigned sections; map the mechanism, threat model and trade-offs.

2. 02

   Project implementation

   120 min· build · integrate · debugtimer

   Study inference gateway vs. model-server architecture (e.g., vLLM / Ollama internal server behind an API gateway). Map the serving topology: API gateway, model runtime, vector DB, background workers, telemetry collector, and internal Docker networks.

   Build, integrate, debug and improve the increment in the active project.

3. 03

   Security lab

   60 min· attack your own systemtimer

   Verify that the raw model runtime and vector DB ports are placed on an internal-only Docker network and cannot be reached directly from the host/public network.

   Test the abuse case, failure path, authorization bypass or resource limit.

4. 04

   Verification & evidence

   30 min· test · record evidencetimer

   Serving network topology diagram and network-isolation verification script.

   Add or run regression tests; record the 9-point evidence record.

5. 05

   Deliberate practice & Socratic review

   30 min· explain aloud · update gapstimer

   Answer the 8 Socratic questions without notes: What? Why? How? What if? Why not? How can I prove it? Where can it break? What have I learned?

   Re-explain the hardest concept aloud, inspect one edge case, update the gap log.

6. total5 h

• R25Docker Security Docs↗ docs.docker.com

  Daemon surface, Linux namespaces/capabilities, rootless/non-root containers, read-only FS, and secrets.

• R26vLLM Documentation↗ docs.vllm.ai

  Serving architecture, OpenAI-compatible server, authentication, concurrency flags, and production deployment.

Read only the named sections needed for today's task. Reading sits inside the 60-minute concepts block; it is not extra time.

• docsDocker Docs · ~25 min read
  Docker Engine security

  Explains namespaces, capability sets and the daemon surface, which is what you have to reason about when the model runtime and gateway share one Docker host.

  R25checked 2026-11-11
• videoNeuralNine · ~15 min
  vLLM: Easily Deploying & Serving LLMs

  Starts a vLLM model server and separates the engine from the HTTP front end, the concrete instance of model runtime sitting behind a gateway.

  R26checked 2026-11-11
• videoFahd Mirza · ~10 min
  Serve Any Hugging Face Model with vLLM: Hands-on Tutorial

  Hands-on serving of an arbitrary model through vLLM's OpenAI-compatible endpoint, useful for confirming which port and path the gateway must proxy to.

  R26checked 2026-11-11
• articleLiteLLM · ~10 min read
  What Is an AI Gateway? Why Your Enterprise LLM Infrastructure Needs One

  Draws the line between an LLM proxy, an LLM gateway and an MCP gateway, which is the vocabulary you need to map gateway vs model-server responsibilities in the topology.

  R26moved 2026-11-11
• docsvLLM · Reference
  serve - vLLM

  Reference for the serving flags that define the runtime's behaviour — API server count, host/port binding, API key auth and concurrency knobs — so you can place it behind the gateway deliberately.

  R26moved 2026-11-11

A lab is complete only when it produces evidence. Installing PyRIT, Garak, Presidio or TransformerLens is not completion. Saved on this device only.