book

Fine-Tuning AI

Name: Fine-Tuning AI
Author: Laurence Moroney
ISBN: 9798341673373

by Laurence Moroney

October 2026

Intermediate to advanced

406 pages

12h 2m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Includes

Quizzes

Preface
Intended AudiencePrerequisite KnowledgeHardware and Software DependenciesBook StructureConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
1. From Generalist to Specialist
The Cost of Renting IntelligenceThe Token TaxThe Agentic MultiplierThe Instruction TaxThe Latency KillerThe Data FirewallThe Metadata LeakCompliance IssuesModel DriftThe Specialist RevolutionHardware Is No Longer the BlockerThe Spectrum of Customization
2. Model Architectures: The DNA of AI
The Flow of Information: How Transformers WorkAttention MechanismsMultihead AttentionThe Feed-Forward NetworkLayer Normalization and Residual ConnectionsDecoder-Only DominanceGrouped Query AttentionMistral and MixtralSliding Window AttentionMixture of Experts and SparsenessGemma: Google’s Open ApproachTokenizers and EmbeddingsTokenizationUnderstanding Language and TokenizationByte Pair EncodingEmbeddingsUnderstanding Positional Embeddings with RoPEOther ConsiderationsInstruction FollowingReasoning DepthContext Windows and the “Lost in the Middle” ProblemConsiderations for Choosing a ModelSummary
3. The Fine-Tuner’s Workshop
Why Setup MattersThe GPU: Your First DecisionVRAM: One Number to Rule Them AllUnderstanding GPUsThe Software StackChoosing Your Operating SystemYour Programming EnvironmentThe Core Library StackExperiment TrackingCloud and Pay-As-You-GoLocal Versus CloudSummary
4. Excavating Data
Sourcing Raw DataPublic Datasets with Hugging FaceKaggle Datasets and CompetitionsScraping Domain-Specific DataCleaning Your DataThe Risk of Leaking PIIDetecting PIIPII Redaction StrategiesUnderstanding Data QualityStatistical ProfilingSpot CheckingContamination ChecksBuilding Your Data PipelineYour Output FormatDataset CardsSummary
5. Synthetic Data
The Teacher-Student ParadigmGenerating Instruction-Response PairsThe Naive ApproachSeed-Based GenerationPrompting the TeacherBatching for Cost EfficiencyUnderstanding Evol-InstructGenerating Chain-of-Thought ReasoningQuality Control for Synthetic DataHallucination DetectionData Diversity MetricsFiltering the Final DatasetMixing Real and Synthetic DataSummary
6. Preparing Your Dataset for Fine-Tuning
The Gap Between Data and TrainingThe Two Paradigms: Completion and ChatChat Templates: From Messages to TokensChatMLLlamaMistral/MixtralGemmaUsing Templates with Hugging FaceGetting Your Data into the Messages FormatFormatting for TRLUsing Alpaca with SFTTrainerLoss MaskingOther Frameworks for Fine-TuningAxolotlUnslothSplitting and Validating Your DatasetFormat ValidationInspecting Tokenized ExamplesSummary
7. The Mechanics of Adaptation
The Problems with Full Fine-TuningThe Memory ProblemCatastrophic ForgettingThe Storage ProblemParameter Efficient Fine-TuningLoRAThe Mathematics of LoRAWhere LoRA Is AppliedThe Merge StepWhy It WorksUnderstanding LoRA HyperparametersRankAlpha and the Scaling FactorTarget ModulesDropoutBiasTask TypePutting It All TogetherQLoRA: An Even Smaller FootprintDoRA: Redefining the AdaptationChoosing Your ApproachUnder the HoodSummary
8. The Training Loop
Assembling the PiecesThe Hugging Face Training StackTraining Configuration with SFTConfigTrainer and SFTTrainerData CollatorsTraining HyperparametersLearning RateLearning Rate ScheduleBatch Size and Gradient AccumulationNumber of EpochsWeight DecayMaximum Gradient NormOptimizerMemory OptimizationsGradient CheckpointingMixed-Precision TrainingYour Memory Budget: Putting It All TogetherPreparing the Training DataThe Loss Function and Label MaskingTraining and Validation SplitThe Complete Training ScriptPerforming the Training RunA Practical Workflow for Hyperparameter TuningStart with the DefaultsTune the Learning RateAdjust the Epoch CountExperiment with RankApply the “Good Enough” PrincipleLogging and Experiment TrackingTensorBoardWeights & BiasesCustom CallbacksReproducibilityOther Training TechniquesSummary
9. Using the Adapter
The Anatomy of an AdapterThe Adapter for the ModelThe Adapter Config JSON FileThe Tokenizer FilesThe Chat TemplateThe Training ArgsDownloading Your AdapterDownload as a ZIPMount Google DrivePush to Hugging Face HubUsing the AdapterMerging the AdapterTesting Inference with an AdapterGoing Beyond Eyeball TestingSide-by-Side ComparisonMeasuring PerplexityLLM-as-a-JudgeDomain-Specific BenchmarksCommon PitfallsImpact of Using QLoRASummary

10. Alignment and Preference Optimization
The Gap: From Knowledge to BehaviorRLHF: The Original PipelineDirect Preference OptimizationBuilding a Preference DatasetGenerating the DataThe DPO Loss FunctionThe ObjectiveThe Role of the Reference Model and KL DivergenceRunning the DPO TrainingPreparing the DatasetLoading the ModelConfiguring and Launching the TrainerThings to Watch for During TrainingEvaluating the Aligned ModelWhen to Use DPO (and When Not To!)Beyond DPOSummary
11. Vision Language Models
Why Vision MattersThe Two-Tower ArchitectureThe Vision EncoderThe Projection LayerWhat Is LLaVA?How Gemma 4 Puts It All TogetherThe Forward PassWhat to Fine-Tune, and What to Freeze!Fine-Tuning StrategiesThe VRAM BudgetHow to Choose Your StrategyBuilding a Vision DatasetThe Data SchemaGetting the Data from Hugging FaceHow the Dataset Was BuiltThe Vision Fine-Tuning ScriptLoading the Model for VisionConfiguring LoRA for VisionPreparing the DatasetThe Training ConfigurationInference with the Fine-Tuned VLMEvaluating the Vision ModelThe Hallucination TestLLM-as-a-Judge with VisionBase Models Versus Fine-Tuned ModelsDomain Examples: Opportunities for Vision Fine-TuningMedical ImagingManufacturing and Quality InspectionDocument UnderstandingPractical ConsiderationsImage Resolution and Token BudgetMulti-image ConversationsWhen to Fine-Tune the Vision TowerTroubleshooting Common IssuesThe Complete WorkflowSummary
12. Diffusion Models
Beyond TransformersThe Big Idea: Reverse a Destruction ProcessGaussian NoiseThe Forward Process: Controlled DestructionThe Reverse Process: DenoisingThe UNet ArchitectureStable Diffusion 1.5The Latent TrickThe Text EncoderThe Scheduler: Managing DenoisingThe Full PipelineClassifier-Free GuidanceFine-TuningSummary
13. Fine-Tuning Diffusion Models
Two Recipes, One ArchitectureDreamBooth: Teaching a SubjectThe Core Ideas of DreamBoothThe General RecipeUsing Your DreamBooth AdapterThe Training Hyperparameters That MatterStyle LoRAs: Teaching a Visual VocabularyBuilding a Style DatasetTrainingUsing the Style AdapterUnderstanding and Sweeping for StrengthCompositing Multiple LoRAsGoing Beyond Stable Diffusion 1.5Preparing Adapters for DeploymentUnderstanding Common Failure ModesSummary
14. Optimization and Quantization
The Deployment GapMerging the Adapter into the BaseWhat Merging Actually ComputesThe 4-bit Merge TrapVerifying ParityWhen Not to MergeHow Quantization Actually WorksA Quick Refresher on Floating PointFrom Floats to Integers: Scale and Zero-PointThe Format ZooUsing GGUF with llama.cppConvert, Then QuantizeUnderstanding the K-Quant NamesThe Importance MatrixA Better Way Forward: QATHow to Get QAT GemmaUsing QAT for Fine-TuningMeasuring Quantization ImpactTwo Metrics to TrackThe Bit-Width SweepKV-Cache OptimizationWhat Is the KV Cache?Grouped Query AttentionKV-Cache QuantizationPagedAttentionPrefix CachingOffloadingPutting It All Together: The Quantization PipelineAvoiding Some Common PitfallsSummary
15. Serving the Expert
From Artifact to ServiceUsing OllamaThe OpenAI Endpoint FormatUsing vLLMGemma 4 ConsiderationsServing the Unmerged AdapterServing Many Experts from One BaseProduction ConcernsStructured OutputA Gateway in FrontObservability and CapacityConclusion: The Future Is Specialized
Index
About the Author

Overview

The era of generic, one-size-fits-all AI is ending. While foundational models like GPT, Claude, and Gemini excel at general knowledge, the next wave of innovation belongs to fine-tuned, domain-expert models that know your business, your code, and your data. This book is a practical guide to moving from the fragility of prompt engineering to the robustness, security, and control of model ownership.

Written by award-winning researcher and bestselling author Laurence Moroney, Fine-Tuning AI leads you through the complete process of using data to train expert models that you own and run locally and privately on your laptop, phone, or data center. Aimed at developers and data owners who have hit the limits of APIs, high per-token costs, and latency, or are concerned about privacy, this book provides a clear blueprint for building efficient, cost-effective systems that can outperform generalist models on specialized tasks.

Understand the value of data and prepare it for fine-tuning

Build workflows for fine-tuning language models

Apply advanced adaptation techniques such as PEFT and LoRA

Move beyond text with multimodal image understanding and generation

Optimize, quantize, and serve private expert models in production

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 0642572310455Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills