Designing Large Language Model Applications

Chapter 11. Embeddings and Representation Learning

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 11th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at [e-mail].

Representation Learning

In the previous chapter, we learned how we can interface language models with external tools, including data stores. External data can be present in the form of text files, database tables, knowledge graphs, and just about any type you can think of. Data can span a wide variety of content types, ranging from proprietary domain-specific knowledge bases to intermediate results and outputs generated by LLMs.

If the data is structured, for example residing in a relational database, the language model can issue a SQL query to retrieve the data it needs. But what if the data is present in unstructured form?

One way to retrieve data from unstructured text datasets is to search by keywords or use regular expressions. For the Apple CFO example in the previous chapter, we can retrieve text containing CFO mentions from a corpus containing financial disclosures, hoping ...