Video description
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!In Build a Large Language Model (from Scratch), you’ll discover how LLMs work from the inside out. In this insightful book, bestselling author Sebastian Raschka guides you step by step through creating your own LLM, explaining each stage with clear text, diagrams, and examples. You’ll go from the initial design and creation to pretraining on a general corpus, all the way to finetuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
- Plan and code all the parts of an LLM
- Prepare a dataset suitable for LLM training
- Finetune LLMs for text classification and with your own data
- Apply instruction tuning techniques to ensure your LLM follows instructions
- Load pretrained weights into an LLM
The process you use to train and develop your own small-but-functional model in this book follows the same steps used to deliver huge-scale foundation models like GPT-4. Your small-scale LLM can be developed on an ordinary laptop, and you’ll be able to use it as your own personal assistant. Build a Large Language Model (from Scratch) is a one-of-a-kind guide to building your own working LLM. In it, machine learning expert and author Sebastian Raschka reveals how LLMs work under the hood, tearing the lid off the Generative AI black box. The book is filled with practical insights into constructing LLMs, including building a data loading pipeline, assembling their internal building blocks, and finetuning techniques. As you go, you’ll gradually turn your base model into a text classifier tool, and a chatbot that follows your conversational instructions.
Sebastian Raschka has been working on machine learning and AI for more than a decade. Sebastian joined Lightning AI in 2022, where he now focuses on AI and LLM research, developing open-source software, and creating educational material. Prior to that, Sebastian worked at the University of Wisconsin-Madison as an assistant professor in the Department of Statistics, focusing on deep learning and machine learning research. He has a strong passion for education and is best known for his bestselling books on machine learning using open-source software.
Table of contents
- Chapter 1. Understanding Large Language Models
- Chapter 2. Working with Text Data
- Chapter 3. Coding Attention Mechanisms
- Chapter 4. Implementing a GPT model from Scratch To Generate Text
- Chapter 5. Pretraining on Unlabeled Data
- Chapter 6. Finetuning for Classification
- Chapter 7. Finetuning to Follow Instructions
- Appendix A. Introduction to PyTorch
- Appendix D. Adding Bells and Whistles to the Training Loop
- Appendix E Parameter-efficient Finetuning with LoRA
Product information
- Title: Build a Large Language Model from Scratch (early access)
- Author(s):
- Release date: July 2024
- Publisher(s): Manning Publications
- ISBN: None
You might also like
book
Designing Large Language Model Applications
Transformer-based language models are powerful tools for solving a variety of language tasks and represent a …
book
Hands-On Large Language Models
AI has acquired startling new language capabilities in just the past few years. Driven by the …
video
Quick Guide to ChatGPT, Embeddings, and Other Large Language Models (LLMs)
9+ Hours of Video Instruction Learn how to use and launch large language models (LLMs) like …
book
Quick Start Guide to Large Language Models, 2nd Edition
The Practical, Step-by-Step Guide to Using LLMs at Scale in Projects and Products Large Language Models …