We commit to develop the processes and infrastructure required to secure our AI systems across their lifecycle, and to treat the safety of their actions as part of that boundary, enabling Safety by Security.
Security spans the full lifecycle of an AI system: training data can be poisoned, model artifacts and their dependencies compromised through the supply chain, and deployed models probed through their own interface. Prompt injection turns any input a model processes into a potential instruction channel, jailbreaks defeat behavioural safeguards, and an agent connected to tools converts a successful injection into real actions with real permissions. This is where security and safety meet: a compromised agent is not only a breached system but an unsafe one, and controls should match both the system’s exposure and the harm its actions could cause.
Teams should assume that capable adversaries will probe their models and tools, and provision agents accordingly: least privilege, per-task credentials, and no standing access beyond need. The institute’s MLSecOps Top 10 covers the pipeline-level vulnerabilities; the attack vectors specific to agentic systems are catalogued in the OWASP agentic security work the institute co-authored. Together with the OWASP Top 10 for LLM applications, these should anchor a threat review before any AI system reaches production.
01 — WHERE IT FAILS
Where it fails
Most AI security failures are prepared in advance by the team’s own defaults, long before an adversary arrives.
- Agents granted standing permissions beyond what their task requires.
- Models, data and tools adopted without vetting or integrity checks.
- Agent credentials shared across users, tasks and environments.
02 — PRACTICAL CONTROLS
Practical controls
Controls should match the system’s exposure and the harm its actions could cause.
- Threat-model against the MLSecOps Top 10, the OWASP agentic attack vectors and the OWASP Top 10 for LLM applications.
- Enforce trust boundaries that separate model instructions from external content.
- Validate layered safeguards through repeatable red-teaming and monitoring.