IT

Prerequisites

Participants should have:

• A basic understanding of AI and LLM concepts
• Familiarity with basic scripting or programming (e.g., Python)
• A foundational knowledge of cybersecurity threats and controls

Target audience

This course is ideal for:

• Security professionals securing LLM or AI-based applications
• Developers and engineers integrating LLMs into enterprise systems
• System architects, DevSecOps teams, and product managers
• Prompt engineers and AI researchers interested in system hardening

Mastering LLM Integration Security: Offensive & Defensive Tactics Course Content

Prompt engineering

• Fundamentals of writing secure, context-aware prompts
• Few-shot prompting and use of delimiters
• Prompt clarity and techniques to reduce injection risk

Prompt injection

• Overview of prompt injection vectors (direct and indirect)
• Practical exploitation scenarios and impacts
• Detection, mitigation, and secure design strategies

Lab activities:

• The Math Professor (direct injection)
• RAG-based data poisoning via indirect injection

ReACT LLM agent prompt injection

• Introduction to the Reasoning-Action-Observation (RAO) model
• Vulnerabilities in frameworks such as LangChain
• Agent behaviour manipulation and plugin exploitation

Lab activities:

• The Bank scenario using GPT-based agents

Insecure output handling

• AI output misuse leading to privilege escalation or code execution
• Front-end exploitation via summarisation and rendering

Lab activities:

• Injection via document summarisation
• Network analysis and arbitrary code execution
• Internal data leaks through stock bot interactions

Training data poisoning

• Poisoning training or fine-tuning datasets to alter LLM behaviour
• Attack simulation and defence strategies

Lab activities:

• Adversarial poisoning
• Injection of incorrect factual data

Supply chain vulnerabilities

• Security gaps in third-party plugin, model, or framework usage
• Dependency risk, plugin sandboxing, and deployment hygiene

Sensitive information disclosure

• How LLMs can inadvertently leak personal or proprietary data
• Overfitting, filtering failures, and context misinterpretation

Lab activities:

• Incomplete filtering and memory retention
• Overfitting and hallucinated disclosure
• Misclassification scenarios

Insecure plugin design

• Misconfigured plugins leading to execution or access control flaws
• Securing LangChain plugins and sanitising file operations

Lab activities:

• Exploiting the LangChain run method
• File system access manipulation

Excessive agency in LLM systems

• Over-privileged agents and unintended capability exposure
• Agent hallucination, plugin misuse, and permission escalation

Lab activities:

• Medical records manipulation
• File system agent abuse and command execution

Overreliance in LLMs

• Cognitive, technical, and organisational risks of AI overdependence
• Legal liabilities, compliance gaps, and mitigation frameworks

Exams and assessments

This course does not include formal certification. Participants will complete multiple hands-on labs simulating attacker tactics and securing LLM implementations. These labs are designed to assess comprehension, critical thinking, and applied technical skill.

Hands-on learning

This course includes:

• Over 10 scenario-based labs hosted in a cloud-accessible platform
• 30-day extended access to all lab environments
• Realistic LLM threat simulations: injection, escalation, data manipulation
• Post-course access to instructor guidance for continued learning