The most useful agents are often the most permission-hungry. They read internal information, invoke tools, execute code and communicate with external systems. That combination creates a new security problem: untrusted natural-language content can influence software that holds real authority.
Traditional cybersecurity principles remain essential, but they must be adapted for agents. A secure design assumes that model instructions can be manipulated, tools can be misused and downstream content may be hostile. It limits the consequences through identity, isolation, deterministic controls and evidence-rich monitoring rather than relying on the model to police itself.
This course applies zero-trust thinking across the agent lifecycle. Participants will threat-model realistic systems, implement practical controls and test whether those controls continue to work under adversarial conditions.
Learning Outcomes
Upon completion of this course, participants will be able to:
- Map the attack surface of an enterprise agent and its connected tools
- Explain prompt injection, tool poisoning, excessive agency and memory attacks
- Design least-privilege permissions for users, agents, tools and data
- Isolate agent execution with sandbox, network and filesystem controls
- Implement deterministic guardrails for consequential actions
- Secure MCP servers and other agent integration points
- Test agent systems using threat modelling and adversarial scenarios
- Design auditable detection, response and recovery processes
Course Outline
Understanding the Agent Attack Surface
- Trust boundaries across users, models, context, tools and outputs
- Differences between application vulnerabilities and model manipulation
- Direct and indirect prompt injection
- Tool poisoning, malicious descriptions and compromised integrations
- Memory poisoning and cross-session data leakage
- Excessive agency and unsafe autonomous side effects
Threat Modelling Agentic Systems
- Identifying assets, actors, entry points and consequential actions
- Mapping data and authority through multi-step workflows
- Applying misuse cases and abuse stories
- Assessing autonomy, reversibility and business impact
- Modelling threats across agent-to-agent delegation
- Prioritising controls according to credible risk
Identity and Least Privilege
- Giving agents distinct, managed identities
- Propagating user context without expanding authority
- Short-lived credentials and just-in-time access
- Separating read, propose, approve and execute permissions
- Scoping access by tool, resource, parameter and environment
- Preventing confused-deputy and privilege-escalation paths
Sandboxed Execution
- Isolating shell, browser and code-execution capabilities
- Filesystem, process and resource restrictions
- Network egress controls and destination allow-lists
- Disposable environments for untrusted work
- Protecting secrets from prompts, tools and logs
- Balancing developer utility with enforceable boundaries
Deterministic Guardrails and Approvals
- Why model instructions are not security controls
- Validating structured inputs and outputs outside the model
- Policy engines for prohibited actions and sensitive data
- Transaction limits, rate limits and loop controls
- Human approval for destructive, financial and external actions
- Designing safe failure and refusal behaviour
Prompt Injection and Untrusted Content
- Separating instructions from retrieved or user-provided data
- Content provenance and trust labelling
- Limiting tools available during untrusted-content processing
- Detecting data-exfiltration and cross-domain requests
- Testing multi-modal and stored injection scenarios
- Designing systems that remain safe when detection fails
Securing MCP and Agent Integrations
- Threat boundaries in clients, servers, tools and registries
- Secure remote transport and server authentication
- OAuth and OpenID Connect considerations for MCP
- Tool allow-listing, description integrity and server approval
- Preventing command injection and server-side request forgery
- Audit and telemetry requirements for tool calls
Security Testing and Red Teaming
- Building adversarial scenarios from the threat model
- Testing authorisation independently from model behaviour
- Simulating malicious tools, documents and agent peers
- Verifying sandbox escape and network-control assumptions
- Automating regression tests for discovered weaknesses
- Defining evidence for production approval
Monitoring and Incident Response
- Logging identities, decisions, approvals and side effects
- Detecting anomalous tool use and authorisation failures
- Revoking agent access and isolating compromised integrations
- Preserving evidence without retaining unnecessary sensitive data
- Responding to poisoned memory or knowledge sources
- Post-incident hardening and security control validation
Practical Capstone
- Threat-model a tool-using enterprise agent
- Implement least-privilege identity and tool permissions
- Run the agent inside a constrained execution environment
- Add deterministic policies and risk-based approval
- Conduct an adversarial test against the complete workflow
- Present the security architecture and residual risks
Intended audience
This course is designed for software and AI engineers, application security professionals, security architects, platform engineers and technical risk specialists who design or assess tool-using agents. It is particularly relevant to regulated organisations and teams handling sensitive code, financial data or operational systems.
Prerequisites
Those attending this course should meet the following:
- Understanding of application and API security fundamentals
- Familiarity with authentication, authorisation and least privilege
- Basic knowledge of LLM applications, tool calling and retrieval
- Experience reading code or reviewing system architectures