Building LLM Powered Applications

Building LLM Powered Applications

by Valentina Alto
Released May 2024
Publisher(s): Packt Publishing
ISBN: 9781835462317

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Get hands-on with GPT 3.5, GPT 4, LangChain, Llama 2, Falcon LLM and more, to build LLM-powered sophisticated AI applications

Key Features

  • Embed LLMs into real-world applications
  • Use LangChain to orchestrate LLMs and their components within applications
  • Grasp basic and advanced techniques of prompt engineering

Book Description

Building LLM Powered Applications delves into the fundamental concepts, cutting-edge technologies, and practical applications that LLMs offer, ultimately paving the way for the emergence of large foundation models (LFMs) that extend the boundaries of AI capabilities.

The book begins with an in-depth introduction to LLMs. We then explore various mainstream architectural frameworks, including both proprietary models (GPT 3.5/4) and open-source models (Falcon LLM), and analyze their unique strengths and differences. Moving ahead, with a focus on the Python-based, lightweight framework called LangChain, we guide you through the process of creating intelligent agents capable of retrieving information from unstructured data and engaging with structured data using LLMs and powerful toolkits. Furthermore, the book ventures into the realm of LFMs, which transcend language modeling to encompass various AI tasks and modalities, such as vision and audio.

Whether you are a seasoned AI expert or a newcomer to the field, this book is your roadmap to unlock the full potential of LLMs and forge a new era of intelligent machines.

What you will learn

  • Explore the core components of LLM architecture, including encoder-decoder blocks and embeddings
  • Understand the unique features of LLMs like GPT-3.5/4, Llama 2, and Falcon LLM
  • Use AI orchestrators like LangChain, with Streamlit for the frontend
  • Get familiar with LLM components such as memory, prompts, and tools
  • Learn how to use non-parametric knowledge and vector databases
  • Understand the implications of LFMs for AI research and industry applications
  • Customize your LLMs with fine tuning
  • Learn about the ethical implications of LLM-powered applications

Who this book is for

Software engineers and data scientists who want hands-on guidance for applying LLMs to build applications. The book will also appeal to technical leaders, students, and researchers interested in applied LLM topics. We don’t assume previous experience with LLM specifically. But readers should have core ML/software engineering fundamentals to understand and apply the content.

Table of contents

  1. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Get in touch
  2. Introduction to Large Language Models
    1. What are large foundation models and LLMs?
      1. AI paradigm shift – an introduction to foundation models
      2. Under the hood of an LLM
    2. Most popular LLM transformers-based architectures
      1. Early experiments
      2. Introducing the transformer architecture
    3. Training and evaluating LLMs
      1. Training an LLM
      2. Model evaluation
    4. Base models versus customized models
      1. How to customize your model
    5. Summary
    6. References
  3. LLMs for AI-Powered Applications
    1. How LLMs are changing software development
    2. The copilot system
    3. Introducing AI orchestrators to embed LLMs into applications
      1. The main components of AI orchestrators
      2. LangChain
      3. Haystack
      4. Semantic Kernel
      5. How to choose a framework
    4. Summary
    5. References
  4. Choosing an LLM for Your Application
    1. The most promising LLMs in the market
      1. Proprietary models
        1. GPT-4
        2. Gemini 1.5
        3. Claude 2
      2. Open-source models
        1. LLaMA-2
        2. Falcon LLM
        3. Mistral
    2. Beyond language models
    3. A decision framework to pick the right LLM
      1. Considerations
      2. Case study
    4. Summary
    5. References
  5. Prompt Engineering
    1. Technical requirements
    2. What is prompt engineering?
    3. Principles of prompt engineering
      1. Clear instructions
      2. Split complex tasks into subtasks
      3. Ask for justification
      4. Generate many outputs, then use the model to pick the best one
      5. Repeat instructions at the end
      6. Use delimiters
    4. Advanced techniques
      1. Few-shot approach
      2. Chain of thought
      3. ReAct
    5. Summary
    6. References
  6. Embedding LLMs within Your Applications
    1. Technical requirements
    2. A brief note about LangChain
    3. Getting started with LangChain
      1. Models and prompts
      2. Data connections
      3. Memory
      4. Chains
      5. Agents
    4. Working with LLMs via the Hugging Face Hub
      1. Create a Hugging Face user access token
      2. Storing your secrets in an .env file
      3. Start using open-source LLMs
    5. Summary
    6. References
  7. Building Conversational Applications
    1. Technical requirements
    2. Getting started with conversational applications
      1. Creating a plain vanilla bot
      2. Adding memory
      3. Adding non-parametric knowledge
      4. Adding external tools
    3. Developing the front-end with Streamlit
    4. Summary
    5. References
  8. Search and Recommendation Engines with LLMs
    1. Technical requirements
    2. Introduction to recommendation systems
    3. Existing recommendation systems
      1. K-nearest neighbors
      2. Matrix factorization
      3. Neural networks
    4. How LLMs are changing recommendation systems
    5. Implementing an LLM-powered recommendation system
      1. Data preprocessing
      2. Building a QA recommendation chatbot in a cold-start scenario
      3. Building a content-based system
    6. Developing the front-end with Streamlit
    7. Summary
    8. References
  9. Using LLMs with Structured Data
    1. Technical requirements
    2. What is structured data?
    3. Getting started with relational databases
      1. Introduction to relational databases
      2. Overview of the Chinook database
      3. How to work with relational databases in Python
    4. Implementing the DBCopilot with LangChain
      1. LangChain agents and SQL Agent
      2. Prompt engineering
      3. Adding further tools
    5. Developing the front-end with Streamlit
    6. Summary
    7. References
  10. Working with Code
    1. Technical requirements
    2. Choosing the right LLM for code
    3. Code understanding and generation
      1. Falcon LLM
      2. CodeLlama
      3. StarCoder
    4. Act as an algorithm
    5. Leveraging Code Interpreter
    6. Summary
    7. References
  11. Building Multimodal Applications with LLMs
    1. Technical requirements
    2. Why multimodality?
    3. Building a multimodal agent with LangChain
    4. Option 1: Using an out-of-the-box toolkit for Azure AI Services
      1. Getting Started with AzureCognitiveServicesToolkit
        1. Setting up the toolkit
        2. Leveraging a single tool
        3. Leveraging multiple tools
        4. Building an end-to-end application for invoice analysis
    5. Option 2: Combining single tools into one agent
      1. YouTube tools and Whisper
      2. DALL·E and text generation
      3. Putting it all together
    6. Option 3: Hard-coded approach with a sequential chain
    7. Comparing the three options
    8. Developing the front-end with Streamlit
    9. Summary
    10. References
  12. Fine-Tuning Large Language Models
    1. Technical requirements
    2. What is fine-tuning?
    3. When is fine-tuning necessary?
    4. Getting started with fine-tuning
      1. Obtaining the dataset
      2. Tokenizing the data
      3. Fine-tuning the model
      4. Using evaluation metrics
      5. Training and saving
    5. Summary
    6. References
  13. Responsible AI
    1. What is Responsible AI and why do we need it?
    2. Responsible AI architecture
      1. Model level
      2. Metaprompt level
      3. User interface level
    3. Regulations surrounding Responsible AI
    4. Summary
    5. References
  14. Emerging Trends and Innovations
    1. The latest trends in language models and generative AI
      1. GPT-4V(ision)
      2. DALL-E 3
      3. AutoGen
      4. Small language models
    2. Companies embracing generative AI
      1. Coca-Cola
      2. Notion
      3. Malbek
      4. Microsoft
    3. Summary
    4. References
  15. Other Books You May Enjoy
  16. Index

Product information

  • Title: Building LLM Powered Applications
  • Author(s): Valentina Alto
  • Release date: May 2024
  • Publisher(s): Packt Publishing
  • ISBN: 9781835462317