Securing the AI Attack Surface

Prompt injection — untrusted content carrying instructions that hijack the model — has no perfect defense today, and that is the single most important fact for a security team. You can't filter your way to safety; you contain damage through layered controls and least privilege. Input defense is the first layer: shields, jailbreak screening, and treating every input the model consumes as potentially hostile.

An AI system is only as trustworthy as what went into it. Data poisoning (corrupting what the model learns or retrieves) and supply-chain compromise (a tampered model or dependency) are attacks with no analog in a traditional app, invisible unless you've built provenance to detect them. Know where every model came from, verify integrity, vet dependencies, and control who can write to your corpora.

Because injection can't be fully prevented, security comes down to what a compromised model can actually do — a question of access, not prompts. An agent inherits the blast radius of every credential and tool you hand it, and can act thousands of times a minute. It should never hold permissions its accountable human owner doesn't; scope every tool to the task. This ties straight to the autonomy ladder in Part 4.

Output is an attack surface in its own right: it can leak PII or secrets from context, emit harmful content, or — most subtly — produce output a downstream system executes without sanitizing (insecure output handling). Treat every output as untrusted until filtered. Content safety and PII redaction on the way out, plus strict handling of anything a database, browser, or shell will act on.

Because you can't prevent every attack, you must see and stop one. Adversarial red-teaming finds weaknesses before attackers do; monitoring surfaces anomalous prompts and actions; and a rehearsed, AI-specific incident-response plan turns an incident from existential to managed. Red-teaming especially deserves a standing cadence — AI systems fail in ways functional testing never finds.