Cybersecurity for AI and Agentic Systems 培训

Module 1: Prompt Injection and Jailbreaks

• Live demonstrations against RAG systems, assistants, and tool-using applications
• System prompt extraction and leakage
• Data exfiltration through model responses
• Tool abuse in agentic workflows
• Direct, indirect, multi-turn, and cross-agent injection
• Why developers miss these risks, and immediate mitigations

Labs: Prompt-hacking-and-defenses: perform real prompt injection attacks and implement layered defenses. Social-Engineering-and-AI: explore how users and attackers manipulate AI systems through natural language. FreeEed: attack a live RAG-based eDiscovery system, extract sensitive information, and test its guardrails.

Module 2: AI Threat Modeling

• Why traditional threat modeling is not enough for AI
• Threat modeling frameworks for AI systems
• Attack surface analysis across ingestion, indexing, querying, export
• Making threat modeling part of delivery
• Manager responsibility for the model

Labs: Threat-Modeling-for-AI: build a full threat model for a GenAI and RAG system. FreeEed case study: model threats across ingestion, indexing, AI querying, and export workflows.

Module 3: OWASP Top 10 for LLM Applications

• Deep dive into the OWASP Top 10 for LLM applications
• Mapping risks to real architectures
• Enforcement strategies and engineering controls
• Turning categories into checklists

Labs: GenAI-Top-10: map vulnerabilities in a real system to OWASP categories. FreeEed: identify OWASP Top 10 risks in its architecture and workflows.

Module 4: Agentic AI Top Risks and Failure Modes

• Agentic risk categories
• AI scheming and self-preservation concerns
• Multi-agent risks
• Management controls for autonomous and semi-autonomous systems

Lab: OpenClawSec: analyze and secure an agentic AI system with tool access.

Module 5: GenAI Security Architecture

• LLM application security architecture
• Secure design patterns
• Making architecture non-optional in delivery
• Segmentation of services, data, and model access

Labs: Introduction to AI security: build a baseline secure GenAI application. FreeEed: review and improve a live RAG, service, and API architecture.

Module 6: AI Guardrails and Safety Systems

• Guardrail architecture patterns
• Content safety and moderation
• Implementation frameworks
• Making guardrails stick in operations

Labs: Guardrails: implement guardrails and evaluate their effectiveness. FreeEed: add and tune guardrails for real legal and document workflows.

Module 7: Securing Agentic AI Systems

• Secure agent architecture
• Zero trust for AI agents
• Scoped tool access and approval paths
• The developer reality of shipping agents
• Reviewing an agent design for excessive agency

Module 8: Secure AI Data Pipelines and MLOps

• Data pipeline security across ingestion, OCR, indexing, storage
• Model security
• MLOps security
• Organizational controls across the lifecycle

Labs: PenAI: simulate attacks on AI pipelines and their defenses. FreeEed: analyze ingestion, OCR, indexing, and storage for vulnerabilities.

Module 9: API Security for AI Systems

• REST API security for model endpoints
• AI-specific API attacks
• Operational protections: quotas, rate limits, monitoring
• Authentication and delegated authorization for agents

Module 10: Advanced Defense and Red Teaming

• Defense in depth architecture
• Adversarial testing and automated red teaming
• Making security continuous

Labs: HarmBench: automated red teaming and benchmarking of AI defenses. FreeEed: end-to-end red team attack simulation on a realistic AI system.

Module 11: AI Security Governance and Compliance

• The regulatory landscape
• Governance frameworks and internal controls
• A manager playbook for enforcement
• Monitoring, auditing, and accountability

Module 12: Future Trends and Emerging Threats

• Multimodal attack vectors
• Autonomous AI malware
• AI-powered security operations
• Long-term agent risks

Final Module: Case Studies and Best Practices

• Enterprise multi-agent RAG architectures
• OWASP and MITRE ATLAS mapping
• Guardrails implementation strategies
• Incident response simulation
• Manager-led security transformation and getting feature-driven teams to adopt secure behavior
• An end-to-end security blueprint

Case work: Incident response simulation, OWASP and MITRE ATLAS mapping, and an end-to-end security blueprint for the case system.