RESTful Web API Patterns and Practices Cookbook

Preface

Welcome to the world of the RESTful Web API Patterns and Practices Cookbook.

That’s quite a moniker—one worth explaining and exploring. And that’s what we’ll be doing in this preface. I will tell you now that I’m going to break the rules a bit and include a substantial amount of pertinent text in the front matter of this book (front matter is all these pages with roman numerals as page numbers). I’ll save the details for the next section (Part I). Let’s first take care of some logistics.

About This Book

The goal of this book is to enable software designers, architects, developers, and maintainers to build service interfaces (APIs) that take advantage of the strengths of the web, while lowering the costs and risks of creating reliable high-level services that hold dependencies on other APIs and services reachable only over the network.

To do that, I’ve gathered a collection of more than 70 recipes and patterns that I’ve learned and used over the several decades I’ve spent helping clients design, build, and deploy successful business services on the open web. I suspect you will be familiar with at least some of the recipes you’ll find here—possibly by other names or in different forms. I also hope that you will find novel approaches to similar problems.

Note

Over the years, I’ve found that the challenges of software design rarely change. The solutions to those problems change frequently based on technology advances and fashion trends. We’ll focus on the challenges in this book, and I’ll leave the up-to-date technology and fashion choices to you, the reader.

Since this is a cookbook, there won’t be much runnable code. There will, however, be lots of diagrams, code snippets, and network message examples along with explanations identifying the problems. The challenges and discussion will always be technology and platform agnostic. These recipes are presented in a way that will let you translate them into code and components that will work within your target environment.

Who Should Read This Book

The primary audience for the book is the people tasked with planning, architecting, and implementing service interfaces that run over HTTP. For some, that will mean focusing on creating enterprise-wide service producers and consumers. For others, it will mean building services that can live on the open web and run in a scalable and reliable way for consumers across the globe. For all, it will mean creating usable application programming interfaces that allow programmers to solve the challenges before them.

Whether you are hosting your solutions locally on your own hardware or creating software that will run in the cloud, the recipes here will help you understand the challenges and will offer a set of techniques for anticipating problems and building in recovery to handle cases where the unanticipated occurs.

What’s Covered

Since the book is meant to be useful to a wide audience, I’ve divided it into chapters focused on related topics. To start, Chapters 1 and 2 make up Part I of the book, where we explore the background and foundations of shared services on the web. To stretch the cookbook analogy, consider Part I as the story behind the “hypermedia cusine” we’ll be exploring in Part II. Like any good cookbook, each of the main chapters in Part II contains a set of self-contained recipes that you can use to meet particular challenges as you design, build, and deploy your web API “dishes.”

Online Resources

The book has a number of associated online resources, including a GitHub repository and related web pages, some examples, and the latest updates to the recipe catalog. You can reach all these resources via http://WebAPICookbook.com.

Here is a quick listing of the chapters and what they cover.

Part I: Understanding RESTful Hypermedia

The opening chapters (Chapters 1 and 2) describe the foundation that underpins all the recipes in the book. They are a mix of history, philosophy, and pragmatic thinking. These are the ideas and principles that reflect the lessons I’ve learned over my years of designing, building, and supporting network software applications running on the web.

Chapter 1, Introducing RESTful Web APIs

This is a general overview of the rationale behind the selected recipes in this book. It includes a section answering the question “what are RESTful web APIs (RWAs)?,” reasons hypermedia plays such an important role in the creation of RWAs, and some base-level shared principles that guide the selection and explanation of the recipes in this book. This chapter “sets the table” for all the material that follows.

Chapter 2, Thinking and Designing in Hypermedia

This chapter explores the background of hypermedia-driven distributed systems that form the foundation for web applications. Each recipe collection covered in Part II (design, clients, services, data, and workflow) is explored with a mix of history, philosophy, and pragmatic thinking. Reading this chapter will help you understand some of the key design ideas and technical bases for all the patterns and practices outlined in the rest of the book.

Part II: Hypermedia Recipe Catalog

Part II holds all the recipes I’ve selected for this volume. You’ll notice that most of the chapters start with the word “hypermedia.” This should give you a clue to the overall approach we’ll be taking throughout the book.

Chapter 3, Hypermedia Design

Reliable and resilient services start with thoughtful designs. This chapter covers a set of common challenges you’ll need to deal with before you even get to the level of coding and releasing your services. This chapter will be particularly helpful to architects as well as service designers, and helps set the tone for the various recipes that follow.

Chapter 4, Hypermedia Clients

This chapter focuses on challenges you’ll face when creating service/API consumer applications. I made a point of discussing client apps before talking about recipes for service interfaces themselves. A common approach for creating flexible and resilient service consumers is necessary for any program that plans on creating a stable and reliable platform for open services that can live on the web as well as within an enterprise.

Chapter 5, Hypermedia Services

With a solid foundation of design principles and properly architected client applications, it can be easier to build and release stable service producers that can be safely updated over time without breaking existing API consumers. This set of recipes focuses not only on principles of solid service interface design but also on the importance of supporting runtime error recovery and reliability patterns to make sure your solutions stay up and running even when parts of your system experience failures.

Chapter 6, Distributed Data

This chapter focuses on the challenges of supporting persisted data in an online, distributed environment. Most of the recipes here are aimed at improving the responsiveness, scalability, and reliability of your data services by ensuring data integrity—even when changing internal data models and implementations at runtime.

Chapter 7, Hypermedia Workflow

The last set of recipes focuses on creating and managing service workflow on the web. The key challenge to face for open services workflow is to create a safe and reliable set of solutions for enlisting multiple unrelated services into a single, resilient workflow to solve a problem none of the individual services knows anything about. I saved this chapter for last since it relies on many of the recipes covered earlier in the book.

Chapter 8, Closing Remarks

The final chapter is a short wrap-up of the material as well as a “call-forward” to help you decide on your own “next steps” as you set out to apply these recipes to your environment.

Appendices

There are a series of appendices for the book that you can use as additional support materials. These are sometimes referred to in the text but can also be treated as stand-alone references.

Appendix A, Guiding Principles

This appendix is a short “motivational poster” version of the single guiding principle behind the selected recipes, as well as some secondary principles used to shape the description and, ultimately, the implementation of these patterns in general.

Appendix B, Additional Reading

Throughout the book, I’ll be recommending additional reading, quoting from books and articles, and calling out presentations and videos that are the source of much of the advice in the book. This appendix contains a self-standing list of reading and viewing materials that you can use as references and a guide when working through the recipes.

Appendix C, Related Standards

Since the goal of this book is to create services that can successfully live “on the web,” the recipes depend upon a number of important open web standards. This appendix contains a list of the related standards documents.

Appendix D, Using the HyperCLI

In several places in the book, I reference a command-line interface tool called HyperCLI. You can use this tool to interact with hypermedia-aware services. This appendix provides a short introduction to the tool and some pointers to other online resources on how to take advantage of HyperCLI and HyperLang.

What’s Not Covered

As a book of recipes, this text is not suited for teaching the reader how to implement the patterns and ideas listed here. If you are new to any of the pillars upon which this book is built, you’ll want to look to other sources for assistance.

The following books are some that I have used in training and consulting engagements on topics not covered in detail in this book:

HTTP protocol

Most of the recipes in this book were developed for HTTP protocol implementations. For more on the power and challenges of HTTP, I recommend the HTTP Developer’s Handbook by Chris Shiflett (Sams). Shiflett’s text has been a great help to me in learning the inside details of the HTTP protocol. Published in 2003, it is still a valuable book that I highly recommend.

API design

For details on designing APIs for distributed services, I suggest readers check out my Building Hypermedia APIs with HTML5 and Node (O’Reilly). For those looking for a book focused on coding APIs, my more recent book, Design and Build Great Web APIs (Pragmatic Bookshelf), offers a detailed hands-on guide to the full API lifecycle.

API clients

The work of coding API/service clients is a skill unto itself. For an extended look at the process of creating flexible hypermedia-driven client applications, I refer readers to my RESTful Web Clients (O’Reilly).

Web APIs

For details on creating web APIs themselves, I encourage readers to check out the book RESTful Web APIs (O’Reilly), which I coauthored with Leonard Richardson, and my book Design and Build Great Web APIs (O’Reilly). Other books I keep close at hand include Principles of Web API Design by James Higginbotham (Addison-Wesley) and Arnaud Lauret’s The Design of Web APIs (Manning).

Data

For more on handling data at scale, I recommend Data Management at Scale by Piethein Strengholt (O’Reilly) and Data Governance: The Definitive Guide by Evren Eryurek et al. (O’Reilly).

Workflow

The books Practical Process Automation by Bernd Ruecker (O’Reilly) and Service Orchestration as Organization by Malinda Kapuruge et al. (O’Reilly) are a good place to start exploring the world of workflow engineering.

There are many other sources of sage advice on designing and building distributed services, and you’ll find a list of suggested reading in Appendix B.

About These Recipes

While the recipes in this cookbook are grouped by topic (design, client, server, data, registry, and workflow), each recipe within the chapters follows the same general pattern:

Problem

This is a short description of the problem you may run into as you design and build your services.

Solution

This section is a narrative of the suggested solution (or solutions) you can employ to solve the stated problem.

Example

In some cases, the recipe will include an example. This might be an HTTP message exchange (request/response) or even a short snippet of pseudocode to show an internal workflow related to the solution.

Discussion

Recipes will also contain a more lengthy discussion section where trade-offs, downsides, and advantages are covered. Often this is the most important section of the recipe, since very few of these challenges have just one possible solution.

Related Recipes

Many of the recipes will end with a list of one or more other related recipes covered elsewhere in the book. Some recipes rely on other recipes or enable them, and this is where you’ll learn how the recipes interact with each other in actual running systems.

How to Use This Book

I highly recommend reading the book from start to finish to get the full effect of the concepts and recipes contained here. However, I also recognize that time may be short and that you might not need a total immersion experience in order to get the benefits of the book. With this in mind, here are a couple of different ways you can read this book, depending on your focus, goals, and the amount of time you want to devote to the text.

I’m in a hurry

If you recently picked up this book and are looking to solve a pressing problem, just check out the Table of Contents for a recipe that sounds like it fits the bill and jump right in. Like all good recipes, each one is written to be a complete offering. There may be some references to other recipes in the book (especially check out the “Related” subsections), and you can follow up with them as needed.

Getting the “big picture” quickly

If you want to quickly get the big picture, I suggest you read all of Chapters 1 and 2 along with Chapter 8. Part I will give you the “tone” of the collection as well as the history of the recipes and the techniques behind them. From there you can decide whether you want to focus on a particular set in Part II or just roam the collection.

Topic reference for focused teams

If you’re part of a team tasked with focusing on one or more of the topics covered here (design, client-side, services, data, workflow, etc.), I suggest you first get the big picture (Part I) and then dive into your particular topic chapter(s) in Part II. You can then use the focus chapters as references as you move ahead with your implementations.

Architect’s deep dive

A thorough read, cover to cover, can be helpful if your primary task is architecting openly available producer and consumer services. Many of the recipes in this book can be used to implement a series of enterprise-level approved components that can be safely stitched together to form a resilient, reliable foundation for a custom service. In this way, the book can act as a set of recommendations for shareable libraries within a single enterprise.

Checklist for managing enterprise-wide programs

For readers tasked with leading enterprise-wide or other large-scale programs, I suggest getting the big picture first, and then using each topic chapter as a guide for creating your own internal management checklists for creating and releasing RESTful web APIs.

Finally, the book was designed to be a helpful reference as well as a narrative guide. Feel free to use the parts that are helpful to you and skim the sections that don’t seem to apply to your situation right now. At some future point, you might find it valuable to go back and (re)read some sections as new challenges arise.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Warning

This element indicates a warning or caution.

Using Code Examples

Supplemental material (code examples, exercises, etc.) is available for download at http://www.webapicookbook.com.

If you have a technical question or a problem using the code examples, please send email to [e-mail].

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “RESTful Web API Patterns and Practices Cookbook by Mike Amundsen (O’Reilly). Copyright 2023 Amundsen.com, Inc., 978-1-098-10674-4.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

O’Reilly Online Learning

Note

For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.

Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/restful-web-api.

Email [e-mail] to comment or ask technical questions about this book.

For news and information about our books and courses, visit https://oreilly.com.

Find us on LinkedIn: https://linkedin.com/company/oreilly-media.

Follow us on Twitter: https://twitter.com/oreillymedia.

Watch us on YouTube: https://youtube.com/oreillymedia.

Acknowledgments

No one who achieves success does so without acknowledging the help of others. The wise and confident acknowledge this help with gratitude.

Alfred North Whitehead

So many people have taught me, inspired me, advised me, and encouraged me, that I hesitate to start a list. But several were particularly helpful in the process of writing this book and they deserve notice.

As all of us do, I stand on the shoulders of giants. Over the years many have inspired me, and some of those I’ve had the pleasure to meet and learn from. Those whose thoughts and advice have shaped this book include Subbu Allamaraju, Belinda Barnet, Tim Berners-Lee, Mel Conway, Roy Fielding, James Gleick, Ted Nelson, Mark Nottingham, Holger Reinhardt, Leonard Richardson, Ian Robinson, and Jim Webber.

I especially want to thank Lorinda Brandon, Alianna Inzana, Ronnie Mitra, Sam Newman, Irakli Nadareishvili, Vicki Reyzelman, and Erik Wilde for their help in reading portions of the text and providing excellent notes and feedback.

I also need to thank all the folks at O’Reilly for their continued support and wise counsel on this project. Specifically, I am deeply indebted to Mike Loukides and Melissa Duffield, who believed in this project long before I was certain about its scope and shape. I also want to say thanks to Angela Rufino for supporting me at every step along the way. Also thanks to Katherine Tozer, Sonia Saruba, and so many others for all the behind-the-scenes work that makes a book like this possible. A special thanks to Kate Dullea and Diogo Lucas for supplying the book’s illustrations.

Finally, a big shout-out to all those I’ve encountered over the years: conference organizers and track chairs, companies large and small that hosted me for talks and consulting, course attendees, and the myriad social media denizens that asked me questions, allowed me to peek into the workings of their organizations, and helped me explore, test, and sharpen the ideas in this book. Everything you see here is due, in large part, to the generosity of all those who came before me and those who work tirelessly each day to build systems that leverage the concepts in Appendix A.