The Agentic AI Threat Landscape
That one single trick that can spread through an AI's brain, tools, and memory, causing it to autonomously take harmful actions across your entire system.
Agentic AI systems refer to systems that plan, use tools, and have a degree of autonomous action. This means that the risk has evolved from more than a LLM hallucinating your soufflé recipe but acting unexpectedly, producing unanticipated side-effects, and possibly creating a cascade of failures across multiple systems. To begin to understand this risk we need to study what the threat landscape looks like and how it differs from traditional systems or even more banal LLMs.
The Expanding Threat Landscape
We use the term “threat landscape” as a metaphor to describe the complete map of all possible perils a system or organization may face. It is the current environment of risks, including who might attack you, how they might do it, and what new dangers have appeared recently. This could include the natural environment the system resides in (bugs, natural disasters, hackers, incompetent employees), enemies (insiders, competitors, cybercriminals, the Russians etc), weapons (phishing emails, malware, Microsoft, prompt injection), and weaknesses (unpatched software, weak passwords, etc).
In the world of Agentic AI the natural environment or terrain has changes. Instead of having unpatched software you now need to worry about an AI Agent being tricked into doing something that is malicious to your organization or customers. This could cause malware to be propagated other agents, a database to be deleted, or customers become exposed to fraud.
With Agentic AI Systems you are not defending a single system, but instead, you are defending a chain of systems that have have a level of autonomy and have multiple input points. Any breach in one component can propagate throughout the entire architecture. This is non-trivial and having a level of humility and respect for the problem space is essential.
Major Implications
There are some major implications that we need to consider:
There are many more trust boundaries in interactions between agents, tools, and data sources.
Patching one layer is insufficient of other layers accept malicious calls.
Finding vulnerabilities is more difficult as there may be different effects at different layers.
Defence in depth is fundamental. Security controls must be enforce from infrastructure to promptm
What “Compositional” Means
When you attack surface is “compositional” it means it is made up of many parts. You are not defending a single system, you are defending a chain of systems that can amplify off of each other’s weaknesses. A breach or flaw in any one components can propagate through the entire architecture.
This means that you need a layered approach to your security. Defence in depth is essential.
