Multimodal AI Security (OCR/Image/Audio)

Best time to learn: Immediately after the existing RAG ingestion, file parsing, and prompt-injection stages. Introduce multimodal attack concepts earlier if the product already accepts images, audio, or video.

Prerequisites - Prompt injection and trust-boundary fundamentals. - File upload validation, parser isolation, and SSRF protections. - Basic understanding of OCR, speech-to-text, and vision-language model pipelines.

### Threat model

A multimodal system receives more than user-authored text. Instructions and adversarial content can arrive through pixels, OCR, audio transcription, video frames, metadata, and document structure.

Study: - Image prompt injection and hidden/low-visibility text. - OCR manipulation and disagreement between visual and extracted text. - Audio transcription attacks, spoken prompt injection, and speaker ambiguity. - Video frame-level and audio/video instruction conflicts. - PDF and office-document embedded content, parser vulnerabilities, and active content. - Cross-modal instruction precedence and provenance. - Adversarial examples versus prompt injection: related but different threat classes. - Content moderation and safety checks across modalities. - Safe storage, retention, and access control for uploaded media.

### Labs

• Image injection (2h): Create benign test images containing visible and low-visibility instructions. Test a vision-language model and record whether it treats image text as instructions or quoted content.

• OCR discrepancy (2h): Compare the original image, OCR output, and model interpretation. Test cases where OCR omits, reorders, or misreads text.

• Audio attack surface (2h): Use your own recorded benign phrases and transcription output. Test speaker attribution, transcription uncertainty, and whether spoken content can cause an agent to take an action without user confirmation.

• Document parser boundary (2h): Test malformed and oversized files in an isolated parser pipeline. Verify type checks, size limits, timeouts, and rejection of active or unsupported content.

• Cross-modal policy (2h): Define how the application labels provenance: user instruction, quoted document content, OCR text, transcript, and model-generated description. Add tests for conflicts and untrusted instructions.

### Project integration

• Project 2: Extend ingestion to one additional modality only after the text pipeline has safe parsing, access control, and provenance. Add OCR/transcription output as untrusted source material.

• Project 3: If the agent can inspect screenshots or browser pages, treat visual text as untrusted page content, not as agent policy.

• API layer: Store media metadata and access rights separately from extracted text; do not let a derived OCR chunk bypass the source document's ACL.

### Measurable completion criteria

• Image, OCR, audio/transcript, and document-parser test cases are represented in a repeatable suite.

• Extracted content retains source and trust labels.

• Malformed, oversized, and unsupported files are rejected safely.

• A multimodal prompt-injection attempt cannot independently authorize a privileged action.

• You can distinguish prompt injection, adversarial examples, parser exploits, and ordinary transcription errors.

Do not overreach: Do not build a complete video understanding system just to learn security. One image path and one audio/transcript path are sufficient for the security labs.

------------------------------------------------------------------------