The Developer's Playbook for Large Language Model Security

The Developer's Playbook for Large Language Model Security

by Steve Wilson
Released September 2024
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098162207

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Large language models (LLMs) are not just shaping the trajectory of AI, they're also unveiling a new era of security challenges. This practical book takes you straight to the heart of these threats. Author Steve Wilson, chief product officer at Exabeam, focuses exclusively on LLMs, eschewing generalized AI security to delve into the unique characteristics and vulnerabilities inherent in these models.

Complete with collective wisdom gained from the creation of the OWASP Top 10 for LLMs list—a feat accomplished by more than 400 industry experts—this guide delivers real-world guidance and practical strategies to help developers and security teams grapple with the realities of LLM applications. Whether you're architecting a new application or adding AI features to an existing one, this book is your go-to resource for mastering the security landscape of the next frontier in AI.

You'll learn:

  • Why LLMs present unique security challenges
  • How to navigate the many risk conditions associated with using LLM technology
  • The threat landscape pertaining to LLMs and the critical trust boundaries that must be maintained
  • How to identify the top risks and vulnerabilities associated with LLMs
  • Methods for deploying defenses to protect against attacks on top vulnerabilities
  • Ways to actively manage critical trust boundaries on your systems to ensure secure execution and risk minimization

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Who Should Read This Book
    2. Why I Wrote This Book
    3. Navigating This Book
      1. Section 1: Laying the Foundation (Chapters 1–3)
      2. Section 2: Risks, Vulnerabilities, and Remediations (Chapters 4–9)
      3. Section 3: Building a Security Process and Preparing for the Future (Chapters 10–12)
    4. Conventions Used in This Book
    5. Using Code Examples
    6. O’Reilly Online Learning
    7. How to Contact Us
    8. Acknowledgments
  2. 1. Chatbots Breaking Bad
    1. Let’s Talk About Tay
    2. Tay’s Rapid Decline
    3. Why Did Tay Break Bad?
    4. It’s a Hard Problem
  3. 2. The OWASP Top 10 for LLM Applications
    1. About OWASP
    2. The Top 10 for LLM Applications Project
      1. Project Execution
      2. Reception
      3. Keys to Success
    3. This Book and the Top 10 List
  4. 3. Architectures and Trust Boundaries
    1. AI, Neural Networks, and Large Language Models: What’s the Difference?
    2. The Transformer Revolution: Origins, Impact, and the LLM Connection
      1. Origins of the Transformer
      2. Transformer Architecture’s Impact on AI
    3. Types of LLM-based Applications
    4. LLM Application Architecture
      1. Trust Boundaries
      2. The Model
      3. User Interaction
      4. Training Data
      5. Access to Live External Data Sources
      6. Access to Internal Services
    5. Conclusion
  5. 4. Prompt Injection
    1. Examples of Prompt Injection Attacks
      1. Forceful Suggestion
      2. Reverse Psychology
      3. Misdirection
      4. Universal and Automated Adversarial Prompting
    2. The Impacts of Prompt Injection
    3. Direct Versus Indirect Prompt Injection
      1. Direct Prompt Injection
      2. Indirect Prompt Injection
      3. Key Differences
    4. Mitigating Prompt Injection
      1. Rate Limiting
      2. Rule-Based Input Filtering
      3. Filtering with a Special-Purpose LLM
      4. Adding Prompt Structure
      5. Adversarial Training
      6. Pessimistic Trust Boundary Definition
    5. Conclusion
  6. 5. Can Your LLM Know Too Much?
    1. Real-World Examples
      1. Lee Luda
      2. GitHub Copilot and OpenAI’s Codex
    2. Knowledge Acquisition Methods
    3. Model Training
      1. Foundation Model Training
      2. Security Considerations for Foundation Models
      3. Model Fine-Tuning
      4. Training Risks
    4. Retrieval-Augmented Generation
      1. Direct Web Access
      2. Accessing a Database
    5. Learning from User Interaction
    6. Conclusion
  7. 6. Do Language Models Dream of Electric Sheep?
    1. Why Do LLMs Hallucinate?
    2. Types of Hallucinations
    3. Examples
      1. Imaginary Legal Precedents
      2. Airline Chatbot Lawsuit
      3. Unintentional Character Assassination
      4. Open Source Package Hallucinations
    4. Who’s Responsible?
    5. Mitigation Best Practices
      1. Expanded Domain-Specific Knowledge
      2. Chain of Thought Prompting for Increased Accuracy
      3. Feedback Loops: The Power of User Input in Mitigating Risks
      4. Clear Communication of Intended Use and Limitations
      5. User Education: Empowering Users Through Knowledge
    6. Conclusion
  8. 7. Trust No One
    1. Zero Trust Decoded
    2. Why Be So Paranoid?
    3. Implementing a Zero Trust Architecture for Your LLM
      1. Watch for Excessive Agency
      2. Securing Your Output Handling
    4. Building Your Output Filter
      1. Looking for PII with Regex
      2. Evaluating for Toxicity
      3. Linking Your Filters to Your LLM
      4. Sanitize for Safety
    5. Conclusion
  9. 8. Don’t Lose Your Wallet
    1. DoS Attacks
      1. Volume-Based Attacks
      2. Protocol Attacks
      3. Application Layer Attacks
      4. An Epic DoS Attack: Dyn
    2. Model DoS Attacks Targeting LLMs
      1. Scarce Resource Attacks
      2. Context Window Exhaustion
      3. Unpredictable User Input
    3. DoW Attacks
    4. Model Cloning
    5. Mitigation Strategies
      1. Domain-Specific Guardrails
      2. Input Validation and Sanitization
      3. Robust Rate Limiting
      4. Resource Use Capping
      5. Monitoring and Alerts
      6. Financial Thresholds and Alerts
    6. Conclusion
  10. 9. Find the Weakest Link
    1. Supply Chain Basics
      1. Software Supply Chain Security
      2. The Equifax Breach
      3. The SolarWinds Hack
      4. The Log4Shell Vulnerability
    2. Understanding the LLM Supply Chain
      1. Open Source Model Risk
      2. Training Data Poisoning
      3. Accidentally Unsafe Training Data
      4. Unsafe Plug-ins
    3. Creating Artifacts to Track Your Supply Chain
      1. Importance of SBOMs
      2. Model Cards
      3. Model Cards Versus SBOMs
      4. CycloneDX: The SBOM Standard
      5. The Rise of the ML-BOM
      6. Building a Sample ML-BOM
    4. The Future of LLM Supply Chain Security
      1. Digital Signing and Watermarking
      2. Vulnerability Classifications and Databases
    5. Conclusion
  11. 10. Learning from Future History
    1. Reviewing the OWASP Top 10 for LLM Apps
    2. Case Studies
      1. Independence Day: A Celebrated Security Disaster
      2. 2001: A Space Odyssey of Security Flaws
    3. Conclusion
  12. 11. Trust the Process
    1. The Evolution of DevSecOps
      1. MLOps
      2. LLMOps
    2. Building Security into LLMOps
    3. Security in the LLM Development Process
      1. Securing Your CI/CD
      2. LLM-Specific Security Testing Tools
      3. Managing Your Supply Chain
    4. Protect Your App with Guardrails
      1. The Role of Guardrails in an LLM Security Strategy
      2. Open Source Versus Commercial Guardrail Solutions
      3. Mixing Custom and Packaged Guardrails
    5. Monitoring Your App
      1. Logging Every Prompt and Response
      2. Centralized Log and Event Management
      3. User and Entity Behavior Analytics
    6. Build Your AI Red Team
      1. Advantages of AI Red Teaming
      2. Red Teams Versus Pen Tests
      3. Tools and Approaches
    7. Continuous Improvement
      1. Establishing and Tuning Guardrails
      2. Managing Data Access and Quality
      3. Leveraging RLHF for Alignment and Security
    8. Conclusion
  13. 12. A Practical Framework for Responsible AI Security
    1. Power
      1. GPUs
      2. Cloud
      3. Open Source
      4. Multimodal
      5. Autonomous Agents
    2. Responsibility
      1. The RAISE Framework
      2. The RAISE Checklist
    3. Conclusion

Product information

  • Title: The Developer's Playbook for Large Language Model Security
  • Author(s): Steve Wilson
  • Release date: September 2024
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098162207