Prompt Engineering for LLMs

Chapter 3. Moving Toward Chat

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

In the previous chapter, you learned about generative pre-trained transformer architecture. The way that these models are trained drastically influences their behavior. A base model, for example, has merely gone through the pre-training process—it has been trained on billions of arbitrary documents from the internet, and if you prompt a base model with the first half of a document, it will generate a plausible-sounding completion for that document. This behavior alone can be quite useful—and throughout this book, we will show how you can “trick” such a model into accomplishing all sorts of tasks besides pure document completion.

However, for a number of reasons, base models can be difficult to use in an application setting. For one thing, because it’s been trained on arbitrary documents from the internet, the base model is equally capable of mimicking both the light side and dark side of the internet. ...