The Developer's Playbook for Large Language Model Security

Table of contents

1. Preface
   1. Who Should Read This Book
   2. Why I Wrote This Book
   3. Navigating This Book
      1. Section 1: Laying the Foundation (Chapters 1–3)
      2. Section 2: Risks, Vulnerabilities, and Remediations (Chapters 4–9)
      3. Section 3: Building a Security Process and Preparing for the Future (Chapters 10–12)
   4. Conventions Used in This Book
   5. O’Reilly Online Learning
   6. How to Contact Us
   7. Acknowledgments
2. 1. Chatbots Breaking Bad
   1. Let’s Talk About Tay
   2. Tay’s Rapid Decline
   3. Why Did Tay Break Bad?
   4. It’s a Hard Problem
3. 2. The OWASP Top 10 for LLM Applications
   1. About OWASP
   2. The Top 10 for LLM Applications Project
      1. Project Execution
      2. Reception
      3. Keys to Success
   3. This Book and the Top 10 List
4. 3. Architectures and Trust Boundaries
   1. AI, Neural Networks, and Large Language Models: What’s the Difference?
   2. The Transformer Revolution: Origins, Impact, and the LLM Connection
      1. Origins of the Transformer
      2. Transformer Architecture’s Impact on AI
   3. Types of LLM-Based Applications
   4. LLM Application Architecture
      1. Trust Boundaries
      2. The Model
      3. User Interaction
      4. Training Data
      5. Access to Live External Data Sources
      6. Access to Internal Services
   5. Conclusion
5. 4. Prompt Injection
   1. Examples of Prompt Injection Attacks
      1. Forceful Suggestion
      2. Reverse Psychology
      3. Misdirection
      4. Universal and Automated Adversarial Prompting
   2. The Impacts of Prompt Injection
   3. Direct Versus Indirect Prompt Injection
      1. Direct Prompt Injection
      2. Indirect Prompt Injection
      3. Key Differences
   4. Mitigating Prompt Injection
      1. Rate Limiting
      2. Rule-Based Input Filtering
      3. Filtering with a Special-Purpose LLM
      4. Adding Prompt Structure
      5. Adversarial Training
      6. Pessimistic Trust Boundary Definition
   5. Conclusion
6. 5. Can Your LLM Know Too Much?
   1. Real-World Examples
      1. Lee Luda
      2. GitHub Copilot and OpenAI’s Codex
   2. Knowledge Acquisition Methods
   3. Model Training
      1. Foundation Model Training
      2. Security Considerations for Foundation Models
      3. Model Fine-Tuning
      4. Training Risks
   4. Retrieval-Augmented Generation
      1. Direct Web Access
      2. Accessing a Database
   5. Learning from User Interaction
   6. Conclusion
7. 6. Do Language Models Dream of Electric Sheep?
   1. Why Do LLMs Hallucinate?
   2. Types of Hallucinations
   3. Examples
      1. Imaginary Legal Precedents
      2. Airline Chatbot Lawsuit
      3. Unintentional Character Assassination
      4. Open Source Package Hallucinations
   4. Who’s Responsible?
   5. Mitigation Best Practices
      1. Expanded Domain-Specific Knowledge
      2. Chain of Thought Prompting for Increased Accuracy
      3. Feedback Loops: The Power of User Input in Mitigating Risks
      4. Clear Communication of Intended Use and Limitations
      5. User Education: Empowering Users Through Knowledge
   6. Conclusion
8. 7. Trust No One
   1. Zero Trust Decoded
   2. Why Be So Paranoid?
   3. Implementing a Zero Trust Architecture for Your LLM
      1. Watch for Excessive Agency
      2. Securing Your Output Handling
   4. Building Your Output Filter
      1. Looking for PII with Regex
      2. Evaluating for Toxicity
      3. Linking Your Filters to Your LLM
      4. Sanitize for Safety
   5. Conclusion
9. 8. Don’t Lose Your Wallet
   1. DoS Attacks
      1. Volume-Based Attacks
      2. Protocol Attacks
      3. Application Layer Attacks
      4. An Epic DoS Attack: Dyn
   2. Model DoS Attacks Targeting LLMs
      1. Scarce Resource Attacks
      2. Context Window Exhaustion
      3. Unpredictable User Input
   3. DoW Attacks
   4. Model Cloning
   5. Mitigation Strategies
      1. Domain-Specific Guardrails
      2. Input Validation and Sanitization
      3. Robust Rate Limiting
      4. Resource Use Capping
      5. Monitoring and Alerts
      6. Financial Thresholds and Alerts
   6. Conclusion
10. 9. Find the Weakest Link
    1. Supply Chain Basics
       1. Software Supply Chain Security
       2. The Equifax Breach
       3. The SolarWinds Hack
       4. The Log4Shell Vulnerability
    2. Understanding the LLM Supply Chain
       1. Open Source Model Risk
       2. Training Data Poisoning
       3. Accidentally Unsafe Training Data
       4. Unsafe Plug-ins
    3. Creating Artifacts to Track Your Supply Chain
       1. Importance of SBOMs
       2. Model Cards
       3. Model Cards Versus SBOMs
       4. CycloneDX: The SBOM Standard
       5. The Rise of the ML-BOM
       6. Building a Sample ML-BOM
    4. The Future of LLM Supply Chain Security
       1. Digital Signing and Watermarking
       2. Vulnerability Classifications and Databases
    5. Conclusion
11. 10. Learning from Future History
    1. Reviewing the OWASP Top 10 for LLM Apps
    2. Case Studies
       1. Independence Day: A Celebrated Security Disaster
       2. 2001: A Space Odyssey of Security Flaws
    3. Conclusion
12. 11. Trust the Process
    1. The Evolution of DevSecOps
       1. MLOps
       2. LLMOps
    2. Building Security into LLMOps
    3. Security in the LLM Development Process
       1. Securing Your CI/CD
       2. LLM-Specific Security Testing Tools
       3. Managing Your Supply Chain
    4. Protect Your App with Guardrails
       1. The Role of Guardrails in an LLM Security Strategy
       2. Open Source Versus Commercial Guardrail Solutions
       3. Mixing Custom and Packaged Guardrails
    5. Monitoring Your App
       1. Logging Every Prompt and Response
       2. Centralized Log and Event Management
       3. User and Entity Behavior Analytics
    6. Build Your AI Red Team
       1. Advantages of AI Red Teaming
       2. Red Teams Versus Pen Tests
       3. Tools and Approaches
    7. Continuous Improvement
       1. Establishing and Tuning Guardrails
       2. Managing Data Access and Quality
       3. Leveraging RLHF for Alignment and Security
    8. Conclusion
13. 12. A Practical Framework for Responsible AI Security
    1. Power
       1. GPUs
       2. Cloud
       3. Open Source
       4. Multimodal
       5. Autonomous Agents
    2. Responsibility
       1. The RAISE Framework
       2. The RAISE Checklist
    3. Conclusion
14. Index
15. About the Author