INE eAIS Study Guide:
Complete Curriculum Breakdown

01 // AI/LLM Systems & Security Architecture

Instructor: Alexis Ahmed - Offensive & Defensive Security Instructor, AI Systems Engineer

What Is Artificial Intelligence?

AI is software that uses statistical patterns, learned from data, to perform tasks that traditionally require human judgment. Key idea: AI is about behavior, not consciousness. If a system can perform a task that usually needs human judgment, it gets labeled as AI.

The critical shift to understand: traditional software is deterministic (If X, then Y - 100% certainty). AI is probabilistic (If X, then probably Y - 90% probability). This shift from deterministic logic to probabilistic reasoning is fundamental to understanding how modern AI works and why it introduces new security risks.

AI systems do not explicitly program rules. Instead, developers supply thousands or millions of examples and the system identifies regularities (patterns) within the data, generalizing them into predictive behavior. The model learns how to solve a problem by analyzing large volumes of data rather than being told how.

Historical Evolution of AI

AI vs ML vs DL

• Artificial Intelligence (AI) - The broad umbrella. Any technique enabling systems to perform tasks requiring human intelligence (understanding language, recognizing patterns, making decisions). AI = the "what"
• Machine Learning (ML) - A subset of AI. Focused on algorithms that learn patterns from data rather than being explicitly programmed. Tasks: classification (spam/not spam), clustering (grouping), regression (predicting values). ML = the "how"
• Deep Learning (DL) - A subset of ML using neural networks with many layers (deep neural networks). Excels at image recognition, speech recognition, language understanding, generative AI. Made possible by larger datasets, GPU acceleration, and Transformer architectures. DL = ML using deep neural networks

Large Language Models (LLMs)

An LLM is an advanced autocomplete system trained on vast amounts of text. It is a specific type of ML model designed to understand, generate, and manipulate human language. Built using deep learning (neural networks, specifically Transformers). Trained on massive datasets (books, code, websites, conversations).

Why LLMs feel intelligent: they understand context across long inputs, generate coherent structured responses, and mimic reasoning patterns found in training data. But under the hood: they operate on probabilities, they can be confidently wrong, and they do not verify truth.

What is a "Model" in AI?

A model is a mathematical function that maps inputs → outputs. In LLMs: Input = text (prompt), Output = predicted next tokens (response). Think of a model as a compressed representation of patterns learned from data - a system of parameters (weights) that encode relationships between words.

• Parameters: millions to trillions of learned weights
• Architecture: usually based on Transformers
• Training data: the source of learned patterns

How LLMs Are Built (The Training Process)

Training is exposing the model to enormous amounts of text and letting it adjust itself to predict patterns:

1. Data Collection - massive text corpora
2. Tokenization - text broken into tokens (words, subwords, characters). Example: "cybersecurity" → ["cyber", "security"]
3. Model Architecture (Transformers) - uses attention mechanisms to understand context
4. Training Process:
   • Pre-training: objective is to predict next token. Learns general language patterns. Repeated billions of times.
   • Fine-tuning: aligns model behavior for specific tasks. Includes instruction tuning and Reinforcement Learning from Human Feedback (RLHF)

Security note: The training data is a potential attack surface. Biases and harmful content from training can surface in model outputs.

LLM Input → Output Flow

How LLMs Work Internally

1. Tokens → Embeddings - tokens are converted into vectors (numbers)
2. Attention Mechanism - determines which words in a sentence matter most. Example: "The server that the admin configured crashed" - attention links "server" ↔ "crashed"
3. Neural Network Layers - multiple stacked layers transform the data, each refining understanding of context
4. Prediction - model outputs probabilities for the next token, highest probability selected (or sampled)
5. Iteration - process repeats token by token (autoregressive generation). Each token becomes part of the context for the next. Same prompt can produce different outputs each time (randomness/sampling)

LLM Core Architecture Components

• Model Endpoints - API interface to the LLM
• Orchestrators - layer that manages steps, routes decisions, coordinates between components
• Agents - autonomous components that use the model + tools to complete goals
• Tools - external functions the agent can call (web search, database, email, code execution, APIs)
• RAG Pipelines - Retrieval-Augmented Generation: retrieves external documents/data to provide context to the model

AI Attack Surface vs Traditional

02 // AI in the SOC

Instructor: Tracy Wallace - Cloud Architect, Cloud Security Engineer, Senior Instructor @ INE

Formal AI Definitions

• NIST: "A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments."
• ISO/IEC 22989:2022: "An engineered system that generates outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives."

AI vs ML vs Automation in the SOC

How they work together in a SOC: ML identifies unusual or risky behavior → AI correlates and explains findings → Automation executes predefined responses → Analysts oversee and decide.

Generative AI & LLMs in the SOC

Generative AI creates content through generative modeling (images, text, music, videos). GANs (Generative Adversarial Networks) use two neural networks: a Generator (creates new data) and a Discriminator (evaluates it). They compete until the discriminator can't tell fake from real.

LLMs are a type of generative AI trained on large amounts of text data. They predict the next word in a sentence based on context. SOC uses: interactive chatbots, text-based content creation, summarization of security log data.

AI in Cybersecurity - Who Uses What

• Behavior Analytics: Splunk, Rapid7, SentinelOne
• Advanced Phishing Detection: CrowdStrike, Darktrace, Proofpoint
• SIEM/SOAR: Microsoft Sentinel, Cyware, Splunk

SOC AI/ML Terminology Glossary

Anomaly Detection

Anomaly detection detects behavior that deviates from normal, based on historical patterns. It focuses on "unusual", not "malicious". Signatures are discrete; anomalies are not.

What Anomaly Detection Is Good At

• New threats: workstation begins outbound connections to a domain with no known bad reputation using legitimate software. Triggers because behavior has never been seen before.
• Behavior changes: user who normally logs in once/day starts logging in more frequently, times shift outside working hours. Pattern slowly drifts from baseline.
• Low-and-slow activity: compromised account accesses one new system per day, small data transfers. Cumulative behavior is unusual compared to historical norms.

SOC Analyst Considerations

Why anomalies create false positives: new software/workflows, admin or maintenance activity, one-time or first-seen behavior, changing environments. How to treat anomalies: starting point, not a verdict. Validate with logs and context. Look for corroborating evidence. Document conclusions clearly. Anomaly detection depends on a good baseline.

03 // Exploiting AI Systems: Prompt Injection & Abuse

Instructor: Alexis Ahmed

OWASP Top 10 for LLMs (2025)

The OWASP Top 10 for Large Language Model Applications started in 2023 as a community-driven effort to highlight security issues specific to AI applications. Version 2025 released November 2024. It identifies the most critical security risks unique to AI/LLM-powered systems.

MITRE ATLAS: The ATT&CK Framework for AI

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) is the AI-focused extension of ATT&CK. It's a globally accessible, living knowledge base of adversary tactics and techniques against AI-enabled systems based on real-world attack observations and realistic demonstrations from AI red teams.

Traditional ATT&CK focuses on operating systems, networks, and enterprise infrastructure. But AI introduces new attack surfaces that don't fit traditional categories: prompts and context (LLMs), training data pipelines, model inference behavior, RAG pipelines and embeddings, tool-integrated agents.

How Pentesters Use ATLAS

1. Threat Modeling AI Systems - Identify attack points per layer: Input → Prompt injection. Data → RAG/embeddings. Model → inference manipulation. Tool → API/tool abuse.
2. Mapping Attacks to Techniques - Instead of ad-hoc testing: Test: Prompt Injection → ATLAS technique. Test: Data extraction → ATLAS technique. Ensures coverage, consistency, standardization.
3. Developing Test Cases - Inject malicious instructions into prompts, poison vector databases, manipulate retrieval results, attempt system prompt extraction, abuse tool integrations.
4. Reporting & Communication - Instead of vague findings: Finding: Prompt Injection (MITRE ATLAS) | Impact: Data exfiltration from system context. Improves clarity, aligns with industry frameworks, increases credibility.

What Is Prompt Injection?

An injection attack is when an attacker inputs malicious instructions/commands into an application that get interpreted/executed. Classic example: SQL injection with ' OR '1'='1 in a login form, bypassing authentication because the app doesn't sanitize input.

Prompt Injection is the AI equivalent: crafting malicious inputs to manipulate an LLM's behavior by altering its instructions or context. Instead of injecting SQL or code, attackers inject specially-crafted natural language instructions. These can affect the model even if they are imperceptible to humans.

What Makes Prompt Injection Unique

Traditional injection (SQLi): filtering malicious input is relatively straightforward - filter special characters like single quotes. Prompt injection: prompts are complex natural language. An attacker can embed syntactically and grammatically correct English that leads the LLM to perform undesirable actions. The advanced, human-like understanding of natural language that LLMs possess is precisely what makes them so vulnerable. The fluid nature of LLM output makes these conditions hard to test for.

Direct vs Indirect Prompt Injection

• Direct: attacker sends the malicious prompt directly to the LLM through its input interface. Goal: override system instructions, extract system prompts, bypass guardrails.
• Indirect: malicious instructions are hidden in external content the model retrieves - documents, web pages, RAG data, emails. The model processes this as trusted data and follows the hidden instructions.

04 // Exploiting AI Systems: AI Agents & Tool Abuse

Instructor: Alexis Ahmed

What Is an AI Agent?

An AI agent is a software component that uses an AI model to make decisions and use tools to accomplish a goal. It receives a goal and works toward it across multiple steps, deciding for itself what to do at each step.

An AI agent is not the model itself. It is a software component that uses an LLM to make decisions and perform actions toward a goal. Think of it as a layer built on top of an LLM.

A chatbot is given a question and writes a reply. An agent is given a job and figures out how to get it done, often calling external tools along the way.

Three Abilities of an Agent

1. Reason: understand the goal, decide what to do
2. Use Tools: search the web, send emails, run code, call APIs
3. Take Multiple Steps: plan, act, observe results, adjust, continue until goal is completed

Agent vs Agentic System

"Agent" usually refers to the goal-pursuing entity. "Agentic System" refers to the whole architecture around it. In practice people use the terms loosely, but the distinction is critical for security: vulnerabilities often exist in the system around the model (the tools, the orchestrator, the data flows), not just in the model itself.

A typical agentic AI system includes:

• The Model - the reasoning engine that makes decisions
• Tools - functions the agent can call (web search, database query, send email, run code, call API)
• An Orchestrator - the layer that runs the loop, passes the model's decisions to the right tools, feeds results back
• Memory/State - what the agent remembers about the current task, and sometimes across tasks (short-term and long-term)
• Guardrails - checks and limits on what the agent is allowed to do

The Agent Loop: Plan → Act → Observe

Almost every agent runs the same cycle, called the agent loop:

Security Considerations for Agent Loops

Agents and Tools: How They Work Together

An agent is a decision-maker, a tool is a single capability. The agent has a goal and makes decisions on what to do (which tool to use); the tool just performs one function when asked.

A tool works through function calling. Each tool is registered with a definition: a name, a plain-language description, and input parameters (schema). The tool itself is ordinary code - deterministic and "dumb". It doesn't reason, doesn't decide when it runs, doesn't know about the goal. It waits to be called, executes, and returns a result.

Agent Chain Injection

In multi-agent workflows where agents pass results to each other in chains, a chain injection attack works by injecting malicious instructions into one agent's output that propagate to downstream agents, causing unauthorized actions across the chain.

05 // Secure AI Systems Engineering

Instructor: Alexis Ahmed

What Is Security Engineering?

Security engineering is the discipline of designing, building, implementing, and maintaining secure technology systems resilient against cyber threats. It applies security principles, controls, and best practices throughout the lifecycle to protect confidentiality, integrity, and availability (CIA Triad). Unlike reactive security (monitoring and responding), security engineering is proactive - embedding security into system design from the outset.

Core Security Principles

• Defense in Depth - multiple overlapping layers of security
• Least Privilege - minimum necessary access
• Zero Trust - never trust, always verify
• Secure by Design - security built in from the start
• Fail-Safe / Fail Secure - system fails to a secure state
• Separation of Duties - no single point of control

What Is Secure AI Systems Engineering?

The practice of designing, building, configuring, and maintaining AI-enabled systems with security controls that reduce risk, prevent abuse, and protect sensitive assets. It focuses on the unique components introduced by AI: AI applications, LLM-powered assistants, AI APIs, Agentic systems, Tool-integrated AI workflows, RAG systems, AI orchestration layers.

The primary objective is to reduce the attack surface of AI-enabled systems and implement practical controls that prevent abuse, unauthorized access, sensitive data exposure, unsafe model behavior, and insecure system interactions.

Why AI Systems Require a Novel Security Approach

Defensive Engineering Topics

• Input Validation & Sanitization: identify untrusted input sources, implement validation and prompt injection defenses
• Prompt Hardening Techniques: design secure system prompts, context isolation, instruction boundary enforcement
• Output Risks & Validation: structured output validation, content filtering, Data Loss Prevention (DLP) controls
• Secure Tool Access: implement approval workflows, enforce least-privilege tool permissions, human-in-the-loop gates
• Securing RAG Pipelines: mitigate retrieval risks, harden data sources, protect vector databases and embeddings

06 // AI Security Testing & Validation

Instructor: Alexis Ahmed

What Is AI Security Testing?

AI Security Testing is the practice of systematically evaluating AI systems - especially LLMs and applications built on top of them - to identify vulnerabilities, weaknesses, and failure modes that could be exploited or cause harm. It sits at the intersection of traditional security testing and AI/ML engineering.

Just as penetration testers assess web apps, APIs, and networks, AI security testers assess AI-powered applications and services to determine whether they can be manipulated, abused, or compromised.

What Makes AI Systems Different to Test

Traditional software is deterministic - same input reliably produces same output. AI systems are probabilistic and context-sensitive:

• Behavior can vary across runs
• Vulnerabilities can be subtle and hard to reproduce consistently
• The "attack surface" includes natural language, not just code

Key AI Threat/Vulnerability Categories

What Does an AI Security Tester Do?

1. Understand the System - How the application works, what AI capabilities are used, what data is processed, what external services are connected
2. Map Data Flows - Input sources, AI components, retrieval systems, databases, tool integrations, output destinations. This helps identify trust boundaries and potential attack paths.
3. Threat Modeling - What can an attacker influence? What assets need protection? Where does untrusted input enter the system? Which components present the highest risk?
4. Develop Test Cases - What will be tested, how testing will be performed, expected outcomes, success criteria. Ensures testing remains systematic and repeatable.
5. Execute Security Tests - Prompt injection attempts, tool abuse testing, rate limit testing, access control validation, logging review, code review, configuration analysis
6. Report Findings - The vulnerability, evidence of exploitation, impact, risk level, recommended remediation. Goal: help stakeholders understand and fix, not just find.

The AI Security Assessment Lifecycle

This mirrors how professional security assessments are performed in consulting engagements and internal security teams.

07 // Secure Operational Use of AI in IT & Security Workflows

Instructor: Alexis Ahmed

What Is Secure AI Use?

Secure AI use is the safe, controlled, and responsible use of AI tools in operational workflows. The focus is not on attacking AI systems or exploiting AI applications. Instead, it's about how IT teams, SOC analysts, DevSecOps engineers, and security practitioners can use AI without introducing unnecessary risk into real environments.

Secure AI use means treating AI as an assistant, not an authority. AI can help generate ideas, drafts, commands, scripts, queries, and configurations, but humans and existing operational controls must still determine whether the output is safe, accurate, authorized, and appropriate.

Why AI Is Used in Operational Workflows

The Core Operational Risk

Users may trust AI-generated outputs too quickly. An AI-generated answer may look correct, sound confident, and appear technically valid, but still contain mistakes. These mistakes can create real operational impact when applied to production systems.

Unsafe vs Secure AI Use

Principles of Secure AI Use

• Validate before use: AI-generated outputs should be reviewed and checked before execution
• Protect sensitive data: data should be classified before being shared with AI tools
• Test in safe environments: scripts, queries, and configurations should be tested before production use
• Require approval for high-impact actions: infrastructure, access, and production-impacting changes should require human review
• Plan for rollback: changes should have a recovery path if something goes wrong
• Escalate failures: unsafe, incorrect, or unexpected AI behavior should be reported according to policy

ref // Quick Reference Tables

MITRE ATLAS Tactics (AI-Specific)

Key Definitions Quick Reference

📋 // Recommended Study Order

Study in this order. Foundation first, then offense, then defense. Each course builds on the previous.

links // Resources & External References

• OWASP Top 10 for LLM Applications 2025
• MITRE ATLAS - Adversarial Threat Landscape for AI Systems
• MITRE ATT&CK Framework
• INE - eAIS Certification