Chapter 5. Designing Read-Only Resource-Oriented Services
We’ve got some information we want to expose to people elsewhere on the network. We want to reach the widest possible combination of clients. Every programming language has an HTTP library, so the natural choice is to expose the data over HTTP. Every programming language has an XML parsing library, so we can format the data with XML and always be understood. Whee!
Sometimes that’s as far as the train of thought goes. The solution is obvious, so the programmers set to work. Despite its vagueness, this technique gives surprisingly good results. Most people are intuitively familiar with what makes a good web site, and a good web service works much the same way.
Unfortunately, this gut-feeling approach combines everyone’s gut feelings into a stew of web services that are usually not RESTful (they’re REST-RPC hybrids), and which work alike only in superficial ways. If you understand why REST works, you can make your services safer, easier to use, and accessible through standard tools.
Some “web services” were never intended to be used as such, and have RESTful qualities seemingly by accident. Into this category fall the many well-designed web sites that have been screen-scraped over the years. So do many providers of images: for instance, the static map tiles served up to the Google Maps application, where you change the URI to address a different part of the Earth. An amusing example is Amazon product images, which can be manipulated in funny ways by putting extra strings in the URI.[16]
It is no accident that so many web sites are RESTful. A well-designed web site presents uncluttered representations of sensibly named resources, accessible through HTTP GET. Uncluttered representations are easy to parse or screen-scrape, and sensibly named resources are easy to address programmatically. Using GET to fetch a representation respects HTTP’s uniform interface. Design a web site by these rules, and it will fit well with my Resource-Oriented Architecture.
Now that I’ve introduced the principles of REST, within the ROA, I’ll show how to use the ROA to design programmatic services that serve data across the network. These simple services provide client access to a data set. They may even let clients filter or search the data. But they don’t let clients modify the data or add to it. In Chapter 6 I talk about web services that let you store and modify information on the server. For now I’m focused on letting clients retrieve and search a data set.
I’ve split the discussion because many excellent web services do nothing more than send useful information out to the people that need it. These are not toy services. Any web-based database search falls into this category: web searches, book searches, even the stereotypical stock quote web service (okay, that one’s probably just a toy). It’s more manageable to cover the simpler cases—which do happen in real life—than to try to cover everything in one huge chapter. The lessons in the next chapter build directly on what I say in this one. After all, a web service that lets clients modify information must also let them retrieve it.
In this chapter I design a web service that serves information about maps. It’s inspired by web applications like Google Maps, but those sites (and the third-party sites build atop them) are designed for ad hoc use by humans. As with any well-designed web site, you can consume Google Maps image tiles as a web service, but only somewhat illicitly and with difficulty. The fantasy service I design here is a programmer-friendly way to retrieve map data for any purpose, including a browser-based map navigation application like the Google Maps Ajax application.
I won’t actually implement this service. An implementation would be too complex to fit in this book, and I don’t own the necessary data anyway. (Note, though, that in Chapter 7 I use the lessons of this chapter to implement a social bookmarking service similar to del.icio.us). This chapter and the next aim to teach you how to see a problem from a resource-oriented point of view. Along the way I hope to demonstrate that the ROA’s simple rules and uniform interface can represent an extremely powerful and fairly complex distributed service.
Resource Design
The standard design technique for object-oriented programs is to break a system down into its moving parts: its nouns. An object is something. Each noun (“Reader,” “Column,” “Story,” “Comment”) gets its own class, and behavior for interacting with the other nouns. By contrast, a good design technique for an RPC-style architecture is to break the system into its motions: its verbs. A procedure does something (“Subscribe to,” “Read,” “Comment on”).
A resource is something, so I take an object-oriented approach to designing resources. In fact, the resource-oriented design strategy could be called “extreme object-oriented.” A class in a programming language can expose any number of methods and give them any names, but an HTTP resource exposes a uniform interface of at most six HTTP methods. These methods allow only the most basic operations: create (PUT or POST), modify (PUT), read (GET), and delete (DELETE). If necessary, you can extend this interface by overloading POST, turning a resource into a small RPC-style message processor, but you shouldn’t need to do that very often.
A service can expose a Story resource, and a Story can exist in either draft or published form, but a client can’t publish a draft Story to the live site. Not in so many words, anyway: “publish” isn’t one of the six actions. A client can PUT a new representation for the Story which depicts it as published. The resource may then be available at a new URI, and may no longer require authentication to read. This is a subtle distinction, but one that keeps you from making dangerous design mistakes like exposing a special RPC-style “publish this article” URI through GET.
The uniform interface means that a resource-oriented design must treat as objects what an object-oriented design might consider verbs. In the ROA, a Reader can’t subscribe to a regularly appearing Column, because “subscribe to” is not part of the uniform interface. There must be a third object, Subscription, representing that relationship between a Reader and a Column. This relationship object is subject to the uniform interface: it can be created, fetched (perhaps as a syndication feed), and deleted. “Subscription” might not show up as a first-class object in an object-oriented analysis, but it probably would appear as a table in an underlying database model. In a resource-oriented analysis, all object manipulation happens through resources that respect the uniform interface. Whenever I’m tempted to add a new method to one of my resource “classes,” I’ll resolve the problem by defining a new kind of resource.
Turning Requirements Into Read-Only Resources
I’ve come up with a procedure to follow once you have an idea of what you want your program to do.[17]It produces a set of resources that respond to a read-only subset of HTTP’s uniform interface: GET and possibly HEAD. Once you get to the end of this procedure, you should be ready to implement your resources in whatever language and framework you like. If you want to expose a larger subset of the uniform interface, I present a slightly extended procedure in Chapter 6.
Figure out the data set
Split the data set into resources
For each kind of resource:
Name the resources with URIs
Design the representation(s) served to the client
Integrate this resource into existing resources, using hypermedia links and forms
Consider the typical course of events: what’s supposed to happen?
Consider error conditions: what might go wrong?
In the sections to come, I’ll show, step by step, how following this procedure results in a RESTful web service that works like the Web. The only difference between what I do and what this procedure says is that I’m going to design all my resources at once, rather than take you through the same steps over and over again for each kind of resource.
Figure Out the Data Set
A web service starts with a data set, or at least an idea for one. This is the data set you’ll be exposing and/or getting your users to build. Earlier I said my data set would be maps, but which maps? This is a fantasy, so I’ll spread the net wide. My imaginary web service will serve maps in all projections and at all scales. Maps of the past, the present, and the supposed future. Maps of other planets and of individual cities. Political maps, road maps (which are just very detailed political maps), physical maps, geological maps, and topographic maps.
This is not every kind of map. I’ll only serve maps that use a standard 2-D coordinate system: a way of identifying any given point on the map. The map need not be accurate, but it must be addressable (there’s that word again) using latitude and longitude. This means I won’t serve most maps of fictional places, maps that arbitrarily distort geography (the way subway maps do), or maps created before longitude could be measured accurately.
Maps are made out of points: in this case, points of latitude and longitude. Every map contains an infinite number of points, but I can have a map without keeping every one of those points in my data set. I just need some image data and a couple basic pieces of information about the map: what are the latitude and longitude of the map’s corners? Or, if the map covers an entire planet, where on the map is the prime meridian?[18]Given that information, I can use standard geographical algorithms to locate and move between the infinitely many points on a map.[19]
A map is a map of some planet. (I say “planet” for simplicity’s sake, but my system will serve maps of moons, asteroids, and any other body that has latitude and longitude.) A map is an interesting part of my data set, but so is the actual planet it represents. It’s convenient to refer to points on a planet, independent of any particular map, even though a planet doesn’t have physical lines of latitude and longitude running around it. One obvious use: I want to be able to see what maps there are for a particular point on Earth. There are probably more maps covering a point in New York City than a point in the middle of the Pacific Ocean.
So my data set includes not only the maps and the points on the maps, but the very planets themselves, and every point on the planets. It may seem hubristic to treat the entire planet Earth as a resource, but remember that I’m not obliged to give a complete account of the state of any resource. If my representation of “Earth” is just a list of my maps of Earth, that’s fine. The important thing is that the client can say “tell me about Earth,” as opposed to “tell me about the political map of Earth,” and I can give an answer.
Speaking of New York City and the Pacific Ocean, some points on a planet are more interesting than others. Most points have nothing much underneath them. Some points correspond to a cornfield or flat lunar plain, and others correspond to a city or a meteor crater. Some points on a planet are places. My users will be disproportionately interested in these points on the planets, and the corresponding points on my maps. They won’t want to specify these places as latitude-longitude pairs. Indeed, many of my users will be trying to figure out where something is: they’ll be trying to turn a known place into a point on a planet.
Fortunately, most places have agreed-upon names, like “San Francisco,” “Eratosthenes,” and “Mount Whitney.” To make it easy for my users to identify places, my data set will include a mapping of place names to the corresponding points on the planets.[20] Note that a single planet may have multiple places with the same name. There might be one “Joe’s Diner” on the Moon and a hundred on Earth, all distinct. If my user wants to find a particular Joe’s Diner on Earth, they’ll have to specify its location more precisely than just “Earth.”
What about places that aren’t points, like cities, countries, and rivers? For simplicity’s sake, I’ll make a well-chosen point stand for an area on a planet. For instance, I’ll have a point on Earth near the geographic center of the U.S. that stands for the place called “the United States of America.” (This is obviously a vast oversimplification. Many real GIS mapping programs represent such areas as lists of points, which form lines or polygons.)
Every place is of a certain type. Some places are cities, some mountains, some hot springs, some the current locations of ships, some areas of high pollution, and so on. I’ll keep track of the type of each place. Two places of different types may correspond to the same point on a planet: some unfortunate’s house may be built on top of a toxic waste dump.
My service can find a place on a planet, given its name, type, or description. It can show the place on any appropriate maps, and it can find places nearby. Given a street address, my service can locate the corresponding point on the planet Earth, and show it on a road map. Given the name of a country, it can locate the corresponding place on the planet (as a representative point), and show it on a political map.
If the client tries to find a place whose name is ambiguous (for instance, “Springfield”) my service can list all appropriate points within the given scope. The client will also be able to search for places of a certain type, without requiring the user give specific names. So a user can search for “pollution sites near Reno, Nevada.”
General Lessons
This is a standard first step in any analysis. Sometimes you get to choose your data set, and sometimes you’re trying to expose data you’ve already got. You may come back to this step as you see how best to expose your data set as resources. I went through the design process two or three times before I figured out that points on a planet needed to be considered distinct from points on any particular map. Even now, the data set is chaotic, just a bundle of ideas. I’ll give it shape when I divide it into resources.
I presented the results of a search operation (“places on Earth
called Springfield”) as part of the data set. An RPC-oriented analysis
would treat these as actions that the client invokes—remember
doGoogleSearch from the Google
SOAP service. Compare this to how the Google web site works: in a
resource-oriented analysis, ways of looking at the data are themselves
pieces of data. If you consider an algorithm’s output to be a
resource, running the algorithm can be as simple as sending a GET to
that resource.
So far I’ve said nothing about how a web service client can access this data set through HTTP. Right now I’m just gathering everything together in one place. I’m also ignoring any consideration of how these features should be implemented. If I actually planned to provide this service, the features I’ve announced so far would have a profound effect on the structure of my database, and I could start designing that part of the application as well. As it is, I’m going to wave away details of the backend implementation, and press on with the design of the web service.
Split the Data Set into Resources
Once you have a data set in mind, the next step is to decide how to expose the data as HTTP resources. Remember that a resource is anything interesting enough to be the target of a hypertext link. Anything that might be referred to by name ought to have a name. Web services commonly expose three kinds of resources:
- Predefined one-off resources for especially important aspects of the application.
This includes top-level directories of other available resources. Most services expose few or no one-off resources.
Example: A web site’s homepage. It’s a one-of-a-kind resource, at a well-known URI, which acts as a portal to other resources.
The root URI of Amazon’s S3 service serves a list of your S3 buckets. There’s only one resource of this type on S3. You can GET this resource, but you can’t DELETE it, and you can’t modify it directly: it’s modified only by operating on its buckets. It’s a predefined resource that acts as a directory of child resources (the buckets).
- A resource for every object exposed through the service.
One service may expose many kinds of objects, each with its own resource set. Most services expose a large or infinite number of these resources.
Example: Every S3 bucket you create is exposed as a resource. You can create up to 100 buckets, and they can have just about any names you want (it’s just that your names can’t conflict with anyone else’s). You can GET and DELETE these resources, but once you’ve created them you can’t modify them directly: they’re modified only by operating on the objects they contain.
Every S3 object you create is also exposed as a resource. A bucket has room for any number of objects. You can GET, PUT, and DELETE these resources as you see fit.
- Resources representing the results of algorithms applied to the data set.
This includes collection resources, which are usually the results of queries. Most services either expose an infinite number of algorithmic resources, or they don’t expose any.
Example: A search engine exposes an infinite number of algorithmic resources. There’s one for every search request you might possibly make. The Google search engine exposes one resource at http://google.com/search?q=jellyfish (that’d be “a directory of resources about jellyfish”) and another at http://google.com/search?q=chocolate (“a directory of resources about chocolate”). Neither of these resources were explicitly defined ahead of time: Google translates any URI of the form http://google.com/search?q={query} into an algorithmic resource “a directory of resources about
{query}.”I didn’t cover this in much detail back in Chapter 3, but S3 also exposes an infinite number of algorithmic resources. If you’re interested, look back to Example 3-7 and the implementation of
S3::Bucket#getObjects. Some of S3’s algorithmic resources work like a search engine for the objects in a bucket. If you’re only interested in objects whose names start with the string “movies/”, there’s a resource for that: it’s exposed through the URI https://s3.amazonaws.com/MyBucket?Prefix=movies/. You can GET this resource, but you can’t manipulate it directly: it’s just a view of the underlying data set.
Let’s apply these categories to my fantasy map service. I need one special resource that lists the planets, just as S3 has a top-level resource that lists the buckets. It’s reasonable to link to “the list of planets.” Every planet is a resource: it’s reasonable to link to “Venus.” Every map of a planet is also a resource: it’s reasonable to link to “the radar map of Venus.” The list of planets is a resource of the first type, since there’s only one of them. The planets and maps are also one-off resources: my service will serve a small number of maps for a small number of planets.
Here are some of the resources so far:
The list of planets
Mars
Earth
The satellite map of Mars
The radar map of Venus
The topographic map of Earth
The political map of Earth
But I can’t just serve entire maps and let our clients figure out the rest. Then I’d just be running a hosting service for huge static map files: a RESTful service to be sure, but not a very interesting one. I must also serve parts of maps, oriented on specific points and places.
Every point on a planet is potentially interesting, and so should be a resource. A point might represent a house, a mountain, or the current location of a ship. These are resources of the second type, because there are an infinite number of points on any planet. For every point on a planet there’s a corresponding point on one or more maps. This is why I limited myself to addressable maps. When the map can be addressed by latitude and longitude, it’s easy to turn a point on the planet into a point on a map.
Here are some more of the resources so far:
24.9195N 17.821E on Earth
24.9195N 17.821E on the political map of Earth
24.9195N 17.821E on Mars
44N 0W on the geologic map of Earth
I’ll also serve places: points on a planet identified by name rather than by coordinates. My fantasy database contains a large but finite number of places. Each place has a type, a latitude and longitude, and each might also have additional associated data. For instance, an area of high pollution should “know” what pollutant is there and what the concentration is. As with points identified by latitude and longitude, the client should be able to move from a place on the planet to the corresponding point on any map.
I said earlier that place names are ambiguous. There are about 6,000 (an approximation) cities and towns in the United States called Springfield. If a place name is unusual you can just say what planet it’s on, and it’s as good as specifying latitude and longitude. If a place name is common, you might have to specify more scoping information: giving a continent, country, or city along with the name of your place. Here are a few more sample resources:
The Cleopatra crater on Venus
The Ubehebe crater on Earth
1005 Gravenstein Highway North, Sebastopol, CA
The headquarters of O’Reilly Media, Inc.
The place called Springfield in Massachusetts, in the United States of America, on Earth
So far, this is pretty general stuff. Users want to know which maps we have, so we expose a one-off resource that lists the planets. Each planet is also a one-off resource that links to the available maps. A geographic point on a planet is addressable by latitude and longitude, so it makes sense to expose each point as an addressable resource. Every point on a planet corresponds to a point on one or more maps. Certain points are interesting and have names, so places on a planet are also accessible by name: a client can find them on the planet and then see that point on a map.
All I’ve done so far is describe the interactions between parts of a predefined data set. I haven’t yet exposed any algorithmically-generated resources, but it’s easy enough to add some. The most common kind of algorithmic resource is the list of search results. I’ll allow my clients to search for places on a planet that have certain names, or that match place-specific criteria. Here are some sample algorithmic resources:
Places on Earth called Springfield
Container ships on Earth
Craters on Mars more than 1 km in diameter
Places on the moon named before 1900
Search results can be restricted to a particular area, not just a planet. Some more sample resources:
Places in the United States named Springfield
Sites of hot springs in Colorado
Oil tankers or container ships near Indonesia
Pizza restaurants in Worcester, MA
Diners near Mount Rushmore
Areas of high arsenic near 24.9195N 17.821E
Towns in France with population less than 1,000
These are all algorithmically-generated resources, because they rely on the client providing an arbitrary search string (“Springfield”) or combining unrelated elements (“Mount Rushmore” + diners, or “France” + towns + “population < 1000”).
I could come up with new kinds of resources all day (in fact, that’s what I did while writing this). But all the resources I’ve thought up so far fit into five basic types, just enough to make the fantasy interesting. Example 5-1 gives the master list of resource types.
The list of planets
A place on a planet—possibly the entire planet—identified by name
A geographic point on a planet, identified by latitude and longitude
A list of places on a planet that match some search criteria
A map of a planet, centered around a particular point
A real-life web service might define additional resources. Real web sites like Google Maps expose one obvious bit of functionality I haven’t mentioned: driving directions. If I wanted to enhance my service I might expose a new algorithmically-generated resource which treats a set of driving directions as a relationship between two places. The representation of this resource might be a list of textual instructions, with references to points on a road map.
General Lessons
A RESTful web service exposes both its data and its algorithms through resources. There’s usually a hierarchy that starts out small and branches out into infinitely many leaf nodes. The list of planets contains the planets, which contain points and places, which contain maps. The S3 bucket list contains the individual buckets, which contain the objects.
It takes a while to get the hang of exposing an algorithm as a set of resources. Instead of thinking in terms of actions (“do a search for places on the map”), you need to think in terms of the results of that action (“the list of places on the map matching a search criteria”). You may find yourself coming back to this step if you find that your design doesn’t fit HTTP’s uniform interface.
Name the Resources
I’ve decided on five types of resources (see Example 5-1). Now they need names. Resources are named with URIs, so let’s pick some. Remember, in a resource-oriented service the URI contains all the scoping information. Our URIs need to answer questions like: “Why should the server operate on this map instead of that map?” and “Why should the server operate on this place instead of that place?”
I’ll root my web service at
http://maps.example.com/. For brevity’s sake I
sometimes use relative URIs in this chapter and the next; understand
that they’re relative to http://maps.example.com/.
If I say /Earth/political, what I
mean is
http://maps.example.com/Earth/political.
Now let’s consider the resources. The most basic resource is the list of planets. It makes sense to put this at the root URI, http://maps.example.com/. Since the list of planets encompasses the entire service, there’s no scoping information at all for this resource (unless you count the service version as scoping information).
For the other resources I’d like to pick URIs that organize the scoping information in a natural way. There are three basic rules for URI design, born of collective experience:
Use path variables to encode hierarchy:
/parent/childPut punctuation characters in path variables to avoid implying hierarchy where none exists:
/parent/child1;child2Use query variables to imply inputs into an algorithm, for example:
/search?q=jellyfish&start=20
Encode Hierarchy into Path Variables
Let’s make URIs for the second class of resource: planets and places on planets. There’s one piece of scoping information here: what planet are we looking at? (Earth? Venus? Ganymede?) This scoping information fits naturally into a hierarchy: the list of planets is at the top, and underneath it is every particular planet. Here are the URIs to some of my planets. I show hierarchy by using the slash character to separate pieces of scoping information.
http://maps.example.com/Venus
http://maps.example.com/Earth
http://maps.example.com/Mars
To identify geographical places by name I’ll just extend the hierarchy to the right. You’ll know you’ve got a good URI design when it’s easy to extend hierarchies by tacking on additional path variables. Here are some URIs to various places on planets:
http://maps.example.com/Venus
http://maps.example.com/Venus/Cleopatra
http://maps.example.com/Earth/France/Paris
http://maps.example.com/Earth/Paris,%20France
http://maps.example.com/Earth/Little%20Rock,AR
http://maps.example.com/Earth/USA/Mount%20Rushmore
http://maps.example.com/Earth/1005%20Gravenstein%20Highway%20North,%20Sebastopol,%20CA%2095472
We’re now deep into web service territory. Sending a GET to one of these URIs invokes a remote operation that takes a variable number of arguments, and can locate a place on a planet to any desired degree of precision. But the URIs themselves look like normal web site URIs you can bookmark, cache, put on billboards, and pass to other services as input—because that’s what they are. Path variables are the best way to organize scoping information that can be arranged hierarchically. The same structure you see in a filesystem, or on a static web site, can correspond to an arbitrarily long list of path variables.
No Hierarchy? Use Commas or Semicolons
The next resources I need to name are geographic points on the globe,
represented by latitude and longitude. Latitude and longitude are tied
together, so a hierarchy isn’t appropriate. A URI like
/Earth/24.9195/17.821 doesn’t make sense. The
slash makes it look like longitude is a subordinate concept to
latitude, the way /Earth/Chicago
signals that Chicago is part of Earth.
Instead of using the slash to put two pieces of scoping information into a hierarchy, I recommend combining them on the same level of a hierarchy with a punctuation character: usually the semicolon or the comma. I’m going to use a comma to separate latitude and longitude. This yields URIs like the following:
http://maps.example.com/Earth/24.9195,17.821
http://maps.example.com/Venus/3,-80
Latitude and longitude can also be used as scoping information
to uniquely identify a named place. A human would probably identify
Mount Rushmore as /Earth/USA/Mount%20Rushmore or as /v1/Earth/USA/SD/Mount%20Rushmore, but
/v1/Earth/43.9,-103.46/Mount%20Rushmore
would be more precise.
From a URI design perspective, the interesting thing here is that I’m stuffing two pieces of scoping information into one path variable. The first path variable denotes a planet, and the second one denotes both latitude and longitude. This kind of URI may look a little strange, because not many web sites or services use them right now, but they’re catching on.
I recommend using commas when the order of the scoping information is important, and semicolons when the order doesn’t matter. In this case the order matters: if you switch latitude and longitude, you get a different point on the planet. So I used commas to separate the two numbers. It doesn’t hurt that people already use commas in written language to separate latitude and longitude: URIs should use our existing conventions when possible.
In another case the order might not matter. Consider a web
service that lets you mix colors of paint to get the shade you want.
If you’re mixing red and blue paint, it doesn’t matter whether you
pour the red into the blue or the blue into the red: you get purple
either way. So the URI /color-blends/red;blue identifies the same
resource as /color-blends/blue;red.
I think the semicolon is better than the comma here, because the order
doesn’t matter. This is just a typographical convention, but it helps
a human being make sense of your web service URIs. The use of the
semicolon feeds into an obscure idea called matrix
URIs, a way of defining key-value pairs in URIs without using
query variables. Some newer standards, like WADL, offer support for
matrix URIs. They’re especially useful if you ever need to put
key-value pairs in the middle of a
hierarchy.
Tip
URIs can become very long, especially when there’s no limit to
how deep you can nest the path variables. My web service might let
clients name a place using a lot of explicit scoping information:
/Earth/North%20America/USA/California/Northern%20California/San%20Francisco%20Bay%20Area/Sebastopol/...
The HTTP standard doesn’t impose any restrictions on URI length, but real web servers and clients do. For instance, Microsoft Internet Explorer can’t handle URIs longer than 2,083 characters, and Apache won’t respond to requests for URIs longer than 8 KBs. If some of your resources are only addressable given a great deal of scoping information, you may have to accept some of it in HTTP headers, or use overloaded POST and put scoping information in the entity-body.
Map URIs
Now that I’ve designed the URI to a geographic point on a planet, what about the corresponding point on a road map or satellite map? After all, the main point of this service is to serve maps.
Earlier I said I’d expose a resource for every point on a map. For simplicity’s sake, I’m not exposing maps of named places, only points of latitude and longitude. In addition to a set of coordinates or the name of a place, I need the name of the planet and the type of map (satellite map, road map, or whatever). Here are some URIs to maps of planets, places, and points:
http://maps.example.com/radar/Venus
http://maps.example.com/radar/Venus/65.9,7.00
http://maps.example.com/geologic/Earth/43.9,-103.46
Scale
A URI like /satellite/Earth/41,-112 says nothing
about how detailed the map should be. I’m going to extend the first
path variable so that it doesn’t just specify the type of map: it
can also specify the scale. I’ll expose a very small-scale map at
/satellite.10/Earth, a very
large-scale map at /satellite.1/Earth, and maps of other
scales in between. I’ll choose a sensible default scale: probably a
large scale like 2. Here are some possible URIs for the same map at
different scales:
/satellite.10/Earth/41,-112: 1:24,000; 2,000 feet to the inch. A map for hiking or prospecting. Centered on 41°N 112°W on Earth, this map would show the banks of Utah’s Great Salt Lake./satellite.5/Earth/41,-112: 1:250,000; 4 miles to the inch. The scale of a highway map. Centered on 41°N 112°W, this map would show the northern suburbs of Salt Lake City./satellite.1/Earth/41,-112: 1:51,969,000; 820 miles to an inch. (That’s 820 miles/inch at the equator. At this scale, the curvature of the earth distorts the scale of a 2D map.) The scale of a world map. Centered on 41°N 112°W, this map would show much of Utah and surrounding states.
The scale affects not only the natural size of the map in
pixels, but which features are shown. A small town would be
represented in fair detail on a map at scale 10, but would only be a point at scale
5 if it showed up at all.
How did I decide that scale 1 would be a large-scale map, and scale
10 would be a small-scale map?
Why not the reverse? I used a common technique for URI design. I
exaggerated the decision I was making, figured out how the
generalized situation should work, and then scaled my decision back
down.
Maps can always get more detailed,[21] but there’s a limit how small they can get. If I
decide to acquire some new data for my map service, I’d never buy a
map that shows the world in less detail than the world map at scale
1. There’d be no point. However,
it’s quite possible that I’ll find maps that are more detailed than
the one at scale 10. When I find
those maps, I can make them available through my service and assign
them scales of 11, 12, and so on. If I’d assigned the most detailed
map a scale of 1, I’d have to assign scales of 0, –1, and so on to
any new maps. The URIs would look strange. This means larger numbers
make good URIs for more detailed maps. I may never actually get
those more detailed maps, but thinking about them revealed a truth
about my URI design.
Algorithmic Resource? Use Query Variables
Most web applications don’t store much state in path variables: they use query variables instead. You may have seen URIs like this:
http://www.example.com/colorpair?color1=red&color2=blue
http://www.example.com/articles?start=20061201&end=20071201
http://www.example.com/weblog?post=My-Opinion-About-Taxes
Those URIs would look better without the query variables:
http://www.example.com/colorpair/red;blue
http://www.example.com/articles/20061201-20071201
http://www.example.com/weblog/My-Opinion-About-Taxes
Sometimes, though, query variables are appropriate. Here’s a Google search URI: http://www.google.com/search?q=jellyfish. If the Google web application used path variables, its URIs would look more like directories and less like the result of running an algorithm: http://www.google.com/search/jellyfish.
Both of those URIs would be legitimate resource-oriented names
for the resource “a directory of web pages about jellyfish.” The
second one doesn’t look quite right, though, because of how we’re
socialized to look at URIs. Path variables look like you’re traversing
a hierarchy, and query variables look like you’re passing arguments
into an algorithm. “Search” sounds like an algorithm. For example,
http://www.google.com/directory/jellyfish"
might work better than /search/jellyfish.
This perception of query variables is reinforced whenever we use the Web. When you fill out an HTML form in a web browser, the data you input is turned into query variables. There’s no way to type “jellyfish” into a form and then be sent to http://www.google.com/search/jellyfish. The destination of an HTML form is hard-coded to http://www.google.com/search/, and when you fill out that form you end up at http://www.google.com/search?q=jellyfish. Your browser knows how to tack query variables onto a base URI. It doesn’t know how to substitute variables into a generic URI like http://www.google.com/search/{q}.
Because of this precedent, a lot of REST-RPC hybrid services use query variables when it would be more idiomatic to use path variables. Even when a hybrid service happens to expose resources RESTfully, the resources have URIs that make them look like function calls: URIs such as http://api.flickr.com/services/rest/?method=flickr.photos.search&tags=penguin. Compare that URI to the corresponding URI on the human-usable Flickr site: http://flickr.com/photos/tags/penguin.
I’ve managed to avoid query variables so far: every planet, every point on a planet, and every corresponding map is addressable without them. I don’t really like the way query variables look in a URI, and including them in a URI is a good way to make sure that URI gets ignored by tools like proxies, caches, and web crawlers. Think back to the Google Web Accelerator I mentioned in Why safety and idempotence matter” in Split the Data Set into Resources. It never pre-fetches a URI that includes a query variable, because that’s the kind of URI exposed by poorly-designed web applications that abuse HTTP GET. My service won’t abuse GET, of course, but outside applications have no way of knowing that.
But I’ve got one more type of resource to represent—lists of search results—and I’m out of tricks. It doesn’t make sense to keep going down the hierarchy of place, and I can’t keep piling on punctuation just to avoid the impression that my service is running an algorithm. Besides, this last type of resource is the result of running an algorithm. My search algorithm finds places that match map-specific criteria, just as a search engine finds web sites that match the client’s keywords. Query variables are perfectly appropriate for naming algorithmic resources.
The search interface for places can get as complex as I need it
to be. I could expose a name query
variable for place names and pollutant for sites of high pollution and
cuisine for restaurants and all
sorts of other query variables. But let’s imagine I’ve got the
technology to make it simple. The only query variable I’ll add is
show, which lets the client specify
in natural language what feature(s) they’re searching for. The server
will parse the client’s values for show and figure out what places should be in
the list of search results.
In Split the Data Set into Resources” earlier in this
chapter, I gave a whole lot of sample search resources: “places on
Earth called Springfield,” and so on. Here’s how a client might use
show to construct URIs for some of
those resources.
http://maps.example.com/Earth?show=Springfield
http://maps.example.com/Mars?show=craters+bigger+than+1km
http://maps.example.com/Earth/Indonesia?show=oil+tankers&show=container+ships
http://maps.example.com/Earth/USA/Mount%20Rushmore?show=diners
http://maps.example.com/Earth/24.9195,17.821?show=arsenic
Note that all of these URIs are searching the planet, not any particular map.
URI Recap
That’s a lot of details. After all, this is the first place where my fantasy resources come into contact with the real world of HTTP. Even so, my service only supports three basic kinds of URI. To recap, here they are:
The list of planets:
/.A planet or a place on a planet:
/: The value of the optional variable{planet}/[{scoping-information}/][{place-name}]{scoping-information}will be a hierarchy of place names like/USA/New%20England/Maine/or it will be a latitude/longitude pair. The value of the optional variable{name}will be the name of the place.This type of URI can have values for
showtacked onto its query string, to search for places near the given place.A map of a planet, or a point on a map:
/. The value of the optional variable{map-type}{scale}/{planet}/[{scoping-information}]{scoping-information}will always be a latitude/longitude pair. The value of the optional variable{scale}will be a dot and a number.
Design Your Representations
I’ve decided which resources I’m exposing, and what their URIs will look like. Now I need to decide what data to send when a client requests a resource, and what data format to use. This is just a warmup, since much of Chapter 9 is devoted to a catalog of useful representation formats. Here, I have a specific service in mind, and I need to decide on a format (or a set of formats) that can meet the goals of any RESTful representation: to convey the current state of the resource, and to link to possible new application and resource states.
The Representation Talks About the State of the Resource
The main purpose of any representation is to convey the state of the resource. Remember that “resource state” is just any information about the underlying resource. In this case, the state is going to answer questions like: what does this part of the world look like, graphically? Where exactly is that meteor crater, in latitude and longitude? Where are the nearby restaurants and what are their names? Where are the container ships right now? Representations of different resources will represent different items of state.
The Representation Links to Other States
The other job of the representation is to provide levers of state. A resource’s representation ought to link to nearby resources (whatever “nearby” means in context): possible new application states. The goal here is connectedness: the ability to get from one resource to another by following links.
This is how web sites work. You don’t surf the Web by typing in URIs one after the other. You might type in one URI to get to a site’s home page, but then you surf by following links and filling out forms. One web page (a “state” of the web site) contains links to other, related web pages (nearby “states”).
Of course, a computer program can’t look at a document and decide which links it wants to follow. It only has the wants the programmer gives it. If a web service includes links in its representations, the representations must also contain machine-readable signals as to where each link leads. A programmer can write his or her client to pick up on those signals and decide which link matches up with the goals of the moment.
These links are the levers of application state. If a resource can be modified with PUT, or it can spawn new resources in response to POST, its representation ought to also expose the levers of resource state. The representation ought to provide any necessary information about what the POST or PUT request should look like. I’m getting a little ahead of myself here, since all the resources in this chapter are read-only. For now, I’ll be creating representations that expose the levers of application state.
Representing the List of Planets
The “home page” of my map service is a good place to start, and a good place to introduce the issues behind choosing a representation format. Basically, I want to display a list of links to the planets for which I have maps. What’s a good format for a representation of a list?
There’s always plain text. This representation in Example 5-2 shows one planet per line: the URI and then the name.
http://maps.example.com/Earth Earth http://maps.example.com/Venus Venus ...
This is simple but it requires a custom parser. I generally think a structured data format is better than plain text, especially as representations get more complex. (Of course, if plain text is what you’re serving, there’s no need to dress it up as something else.) JSON keeps the simplicity of plain text but adds a little bit of structure (see Example 5-3).
[{"url": "http://maps.example.com/Earth", "description": "Earth"},
{"url": "http://maps.example.com/Venus", "description": "Venus"},
...]The downside is that neither JSON nor plain text are generally considered “hypermedia” formats. Another popular option is a custom XML vocabulary, either with or without a schema definition (see Example 5-4).
<?xml version="1.0" standalone='yes'?> <planets> <planet href="http://maps.example.com/Earth" name="Earth" /> <planet href="http://maps.example.com/Venus" name="Venus" /> ... </planets>
These days, a custom XML vocabulary seems to be the default choice for web service representations. XML is excellent for representing documents, but I think it’s actually pretty rare that you would have to come up with a custom vocabulary. The basic problems have already been solved, and most of the time you can reuse an existing XML vocabulary. As it happens, there’s already an XML vocabulary for communicating lists of links called Atom.
I cover Atom in detail in Chapter 9. Atom will work to represent the list of planets, but it’s not a very good fit. Atom is designed for lists of published texts, and most of its elements don’t make sense in this context—what does it mean to know the “author” of a planet, or the date it was last modified? Fortunately, there’s another good XML language for displaying lists of links: XHTML. Example 5-5 shows one more representation of the planet list, and this is the one I’m actually going to use.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>Planet List</title>
</head>
<body>
<ul class="planets">
<li><a href="/Earth">Earth</a></li>
<li><a href="/Venus">Venus</a></li>
...
</ul>
</body>
</html>It might seem a little odd to use XHTML, a technology associated
with the human web, as a representation format for a web service. I
chose it for this example because HTML solves many general markup
problems and you’re probably already familiar with it. I’d probably
choose it for a real web service, for exactly the same reasons. Though
it’s human-readable and easy to render attractively, nothing prevents
well-formed HTML from being processed automatically like XML. XHTML is
also extensible. I turned a generic XHTML list into a list of
“planets” using XHTML’s class
attribute. This is a simple example of an XHTML
microformat: a way of adding semantic meaning
to XHTML’s markup tags. I cover some standard microformats in Chapter 9.
Representing Maps and Points on Maps
What about the maps themselves? What do I serve if someone asks for a satellite map of the Moon? The obvious thing to send is an image, either in a traditional graphics format like PNG or as a SVG scalar graphic. Except for the largest-scale maps, these images will be huge. Is this OK? It depends on the audience for my web service.
If I’m serving clients with ultra-high bandwidth who expect to process huge chunks of map data, then huge files are exactly what they want. But it’s more likely my clients will be like the users of existing map applications like Google and Yahoo! Maps: clients who want smaller-sized maps for human browsing.
If the client asks for a medium-scale hiking map centered around 43N 71W, it’s surely a waste of bandwidth to send a map of the whole world centered around that point. Instead I should send a little bit of a hiking map, centered around that point, along with navigation links that let the client change the focus of the map. Even if the client asks for a detailed map of the whole world, I don’t need to send the entire map: I can send part of the map and let the client fetch the rest as needed.
This is more or less how the online map sites work. If you visit http://maps.google.com/, you get a political map centered on the continental United States: that’s its representation of “a map of Earth.” If you visit http://maps.google.com/maps?q=New+Hampshire, you get a road map centered on Concord, the capital city. In either case, the map is divided into square “tile” images 256 pixels on a side. The client (your web browser) fetches tiles as needed and stitches them together to form a navigable map.
Google Maps splits the globe into a grid of 256-pixel square tiles, pretty much ignoring issues of latitude and longitude, and generates static images for each tile. It does this 10 times, once for every zoom level. This is efficient (though it does use a lot of storage space), but for pedagogical purposes I’ve chosen a conceptually simpler system. I’m assuming my map service can dynamically generate and serve a 256 ×256 image at any scale, centered on any point of latitude and longitude on any map.
Tip
Google Maps’s static tile system is more complex because it adds another coordinate system to the map. Besides latitude and longitude, you can also refer to a place by which tile it’s on. This makes the navigation representation simpler, at the expense of complicating the design.
When the client requests a point on a map, I’ll serve a hypermedia file that includes a link to a tiny map image (a single, dynamically-generated tile) centered on that point. When the client requests a map of an entire planet, I’ll pick a point on that planet somewhat arbitrarily and serve a hypermedia file that links to an image centered on that point. These hypermedia files will include links to adjacent points on the map, which will include more links to adjacent points, and so on. The client can follow the navigation links to stitch many tiles together into a map of any desired size.
So Example 5-6 is one possible representation of http://maps.example.com/road/Earth. Like my representation of the list of planets, it uses XHTML to convey resource state and to link to “nearby” resources. The resource state here is information about a certain point on the map. The “nearby” resources are nearby in a literal sense: they’re nearby points.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>Road Map of Earth</title>
</head>
<body>
...
<img class="map" src="/road.2/Earth/images/37.0,-95.png" alt="Map tile"/>
...
<a class="map_nav" href="46.0518,-95.8">North</a>
<a class="map_nav" href="41.3776,-89.7698">Northeast</a>
<a class="map_nav" href="36.4642,-84.5187">East</a>
<a class="map_nav" href="32.3513,-90.4459">Southeast</a>
...
<a class="zoom_in" href="/road.1/Earth/37.0;-95.8">Zoom out</a>
<a class="zoom_out" href="/road.3/Earth/37.0;-95.8">Zoom in</a>
...
</body>
</html>Now when a client requests the resource “a road map of Earth” at
the URI /road/Earth, the
representation they get is not an enormous, insanely detailed image
that they can’t deal with. It’s a small XHTML document, one that
includes links to several other resources.
A human being can just look at this document and know what it means. A computer program doesn’t have that ability; it has to be programmed in advance by someone who can think about a whole class of these documents and write code to find which bits have the meaning. A web service works by making promises that it will serve representations of resources with a certain structure. That’s why my representation is full of semantic cues like “zoom_in” and “Northeast”. Programmers can write clients that pick up on the semantic cues.
Representing the Map Tiles
The representation of a road map of Earth, given in Example 5-6, has a lot of links in it. Most
of these are links to XHTML documents that look a lot like “a road map
of Earth” does: they’re representations of points on the map at
various zoom levels. The most important link, though, is the one in
the IMG tag. That tag’s src attribute references the URI
http://maps.example.com/road/Earth.8/images/37.0,-95.png.
This is a new kind of resource, and I haven’t really considered it before, but it’s not hard to figure out what it is. This resource is “an image centered around 37°N 95.8°W on the road map of Earth.” In my service, the representation of that resource will be a 256 ×256 image showing the geographic center of the continental U.S. (see Figure 5-1).
The image in Figure 5-1 is 256 pixels square, and represents an area of the Earth about 625 miles square. This image is distinct from the representation of “39°N 95.8°W on the road map of Earth.” that would be an XHTML file like the one in Example 5-6. The XHTML file would include this image by reference, and also link to a lot of nearby points on the map.
Here’s another example: if the client requests /road.8/Earth/32.37,-86.30, my service will
send an XHTML representation whose IMG tag references /road.8/Earth/images/32.37,-86.30.png (see
Figure 5-2). This is a very detailed road map
centered on 32.37°N, 86.30°W on Earth.
That image too is 256 pixels square, but it represents an area of the Earth only a half-mile square. Scale makes the difference.
The important thing here is not the exact setup of the tile
system or the precise format of the URIs to the tile resources. What’s
important is what I’m putting into my representations. The URI
/road/Earth refers to a resource:
“a road map of Earth”. You’d expect a pretty big image as the
representation of that resource. You’d at least expect one that showed
all of Earth. But my service sends an XHTML document that references a
256 ×256 tile image that doesn’t even cover four U.S. states. How can
that document be a good representation for “a road map of
Earth”?
A representation conveys the state of its resource, but it doesn’t have to convey the entire state of the resource. It just has to convey some state. The representation of “Earth” (coming up in a bit) isn’t the actual planet Earth, and the representation of “a road map of the Earth” can reference just a simple image tile. But this representation does more than that: the XHTML file links this arbitrarily chosen point on the map to other nearby points on the part of the map directly to the north of this tile, the part directly to the east, and so on. The client can follow these links to other resources and piece together a larger picture. The map is made of a bunch of connected resources, and you can get as many graphical tiles as you need by following the links. So in a sense, this representation does convey all the state there is about the road map of Earth. You can get as much of that state as you want by following its links to other resources.
It’s worth repeating here that if my clients actually need
detailed multigigabyte maps, there’s no point in me chopping up the
state of the map into these tiny tiles. It’d be more efficient to have
the representation of /road/Earth?zoom=1 convey the entire state
of the map with one huge image. I’ve designed for clients that only
really want part of a map, and wouldn’t know what to do with one huge
map of the earth if I gave it to them. The clients I have in mind can
consume the XHTML files, load the appropriate images, and
automatically follow links to stitch together a map that’s as big as
necessary. You could write an Ajax client for my web service that
worked like the Google Maps application.
Representing Planets and Other Places
I’ve shown representations for the planet list, for maps of the
planets, and for points on the maps. But how are you supposed to get
from the planet list to, say, the road map of Earth? Presumably you
click “Earth” in the planet list, sending a GET request to /Earth, and get back a
representation of Earth. This representation includes a bunch of links
to maps of Earth. At this point you follow a second link to the road
map of Earth. Well, I just described the representation of Earth. My
representation of a planet contains whatever useful information I have
about the planet, as well as a set of links to other resources: maps
of the planet (see Example 5-7).
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head><title>Earth</title></head>
<body>
<dl class="place">
<dt>name</dt> <dd>Earth</dd>
<dt>maps</dt>
<dd>
<ul class="maps">
<li><a class="map" href="/road/Earth">Road</a></li>
<li><a class="map" href="/satellite/Earth">Satellite</a></li>
...
</ul>
</dd>
<dt>type</dt> <dd>planet</dd>
<dt>description</dt>
<dd>
Third planet from Sol. Inhabited by bipeds so amazingly primitive
that they still think digital watches are a pretty neat idea.
</dd>
</dl>
</body>
</html>I’ve chosen to represent places as lists of key-value pairs.
Here, the “place” is the planet Earth itself. Earth in this system is
a named place, just like San Francisco or Egypt. I’m representing it
using the dd tag: HTML’s
standard way of presenting a set of key-value pairs. Like any place,
Earth has a name, a type, a description, and a list of maps: links to all the resources that map
this place.
Why am I representing a planet as a place? Because now my
clients can parse the representation of a planet with the same code
they use to parse the representation of a place. Example 5-8 is a representation for Mount
Rushmore on Earth. You might get this XHTML file back in response to a
GET request for /Earth/USA/Mount%20Rushmore.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head><title>Mount Rushmore</title></head>
<body>
<ul class="places">
<li>
<dl class="place">
<dt>name</dt> <dd>Mount Rushmore</dd>
<dt>location</dt>
<dd>
<a class="coordinates" href="/Earth/43.9,-95.9">43.9°N 95.8°W</a>
</dd>
<dt>maps</dt>
<dd>
<ul class="maps">
<li><a class="map" href="/road/Earth/43.9,-95.9">Road</a></dd>
<li><a class="map" href="/satellite/Earth/43.9,-95.9">Satellite</a></li>
...
</ul>
</dd>
<dt>type</dt> <dd>monument</dd>
<dt>description</dt>
<dd>
Officially dedicated in 1991. Under the jurisdiction of the
<a href="http://www.nps.gov/">National Park Service</a>.
</dd>
</dl>
</li>
</ul>
</body>
</html>Rather than serve a map image of Mount Rushmore, or even an
XHTML page that links to that image, this representation links to
resources I’ve already defined: maps of the geographical point where
Mount Rushmore happens to be located. Those resources take care of all
the imagery and navigation details. The purpose of this resource is to
talk about the state of the place, and what it looks like on a map is
just one bit of that state. There’s also its name, its type (“monument”), and its description. The only difference between the
representation of a planet and that of a place is that a place has a
location in its definition list,
and a planet doesn’t. A client can parse both representations with the
same code.
You may also have noticed that you don’t have to write a special client for this web service at all. You can use a plain old web browser. Starting at the home page (http://maps.example.com/), you click a link (“Earth”) to select a planet. You get the representation shown in Example 5-7, and you click “Road” to see a road map of Earth. Then you navigate that map by clicking links (“North,” “Zoom out”). My web service is also a web site! It’s not a very pretty web site, because it’s designed to be used by a computer program, but nothing prevents a human from consuming it (or debugging it) with a web browser.
If you get only one thing out of this book, I hope it’s that this idea starts seeming natural to you (assuming it didn’t before). Web services are just web sites for robots. My map service is particularly web site-like: it connects its resources together with hypermedia, the hypermedia representations happen to be HTML documents, and (so far) it doesn’t use any features that web browsers don’t support. But all RESTful resource-oriented web services partake of the nature of the Web, even if you can’t use them with a standard web browser.
Example 5-9 shows one more representation: the representation of a point on the map.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>43.9°N 103.46°W on Earth</title>
</head>
<body>
<p>
Welcome to
<a class="coordinates" href="/Earth/43.9,-103.46">43.9°N
103.46°W</a>
on scenic <a class="place" href="/Earth">Earth</a>.
</p>
<p>See this location on a map:</p>
<ul class="maps">
<li><a class="map" href="/road/Earth/43.9,-95.9">Road</a></li>
<li><a class="map" href="/satellite/Earth/43.9,-95.9">Satellite</a></li>
...
</ul>
<p>Things that are here:</p>
<ul class="places">
<li><a href="/Earth/43.9,-95.9/Mount%20Rushmore">Mount Rushmore</a></li>
</ul>
<form id="searchPlace" method="get" action="">
<p>
Show nearby places, features, or businesses:
<input name="show" repeat="template" /> <input class="submit" />
</p>
</form>
</body>
</html>This representation consists entirely of links: links to maps centered around this point, and links to places located at this point. It has no state of its own. It’s just a gateway to other, more interesting resources.
Representing Lists of Search Results
I’ve shown representations for the planet list, for a planet, for points and places on a
planet, and for the maps themselves. What about my algorithmic
resources, the search results? What’s a good representation of the
resource “diners near Mount Rushmore” (/Earth/USA/Mount%20Rushmore?show=diners)?
What about “Areas of high arsenic near 24.9195°N 17.821°E” (/Earth/24.9195,17.821?show=arsenic)?
A list of search results is of course associated with the place being “searched,” so a representation of “diners near Mount Rushmore” should link to the place “Mount Rushmore.” That’s a start.
When the client searches in or around a place, they’re searching for more places. Whether the search string is an ambiguous place name (“Springfield”) or a more general description of a place (“diners,” “arsenic”), the results will be places on the map: cities named Springfield, diners, or sites with high arsenic readings. So a list of search results takes one place (“Mount Rushmore”), and associates certain other places (“Joe’s Diner”) with it.
A list of search results, then, can be nothing but a list of links to resources I’ve already defined: named places on the map. If the client is interested in a place, it can follow the appropriate link and find out about its state.
Example 5-10 shows the representation
of a set of search results. The search is an attempt to find places
called “Springfield” in the United States: its URI would be /Earth/USA?show=Springfield.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head><title>Search results: "Springfield"</title></head>
<body>
<p>
Places matching <span class="searchterm">Springfield</span>
in or around
<a class="place" href="/Earth/USA">the United States of America</a>:
</p>
<ul>
<li>
<a class="place" href="/Earth/USA/IL/Springfield">Springfield, IL</a>
</li>
<li>
<a class="place" href="/Earth/USA/MA/Springfield">Springfield, MA</a>
</li>
<li>
<a class="place" href="/Earth/USA/MO/Springfield">Springfield, MO</a>
</li>
...
</body>
</html>This representation is made up almost entirely of links to places. There’s the link to the place that was searched, “the United States of America” (a place of type “country”). There are also links to various places that matched the search criteria. Each of these places is a resource, and exposes a representation that looks like Example 5-8. Each link contains enough scoping information to uniquely identify the Springfield in question.
A client can follow the links in the representations to find
information about the places, as well as maps of them. Figure 5-3 shows some of the links you can follow
from /Earth/USA?show=Springfield.
Google Maps presents search results by sewing image tiles
together to make a large-scale map, and annotating the map with
graphical markers. You can write a client for this web service that
does the same thing. The first step is to build the large-scale map.
The client follows the initial link to /Earth/USA and gets a representation like
the one in Example 5-4. This gives the
client the address of one graphical tile. The client can get adjacent
tiles by following navigation links, and stitch them together into a
large-scale tile map of the whole country.
The second step is to stick markers on the map, one for each search result. To find out where a marker should go, the client follows one of the search result links, fetching a representation like the one in Example 5-8. This representation lists the latitude and longitude of the appropriate Springfield.
That’s potentially a lot of link following, but my representations are simple and so are the rules for going from one to another. I’ve spread out my data set over a huge number of resources, and made it easy to find the resource you want by following links. This strategy works on the human web, and it works on the programmable web too.
Link the Resources to Each Other
Since I designed all my resources in parallel, they’re already full of links to each other (see Figure 5-3). A client can get the service’s “home page” (the planet list), follow a link to a specific planet, follow another link to a specific map, and then follow navigation and zoom links to jump around the map. A client can do a search for places that meet certain criteria, click one of the search results to find out more about the place, then follow another link to locate the place on a map.
One thing is still missing, though. How is the client supposed to get to a list of search results? I’ve set up rules for what the URI to a set of search results looks like, but if clients have to follow rules to generate URIs, my service isn’t well connected.
I want to make it possible for a client to get from /Earth/USA/Mount%20Rushmore to /Earth/USA/Mount%20Rushmore?show=diners.
But it does no good to link to “diners” specifically: that’s just one of
infinitely many things a client might search for. I can’t put infinitely
many links in the representation of /Earth/USA/Mount%20Rushmore just in case
someone decides to search for pet stores or meteor craters near Mount
Rushmore.
HTML solves this problem with forms. By sending an appropriate form in a representation, I can tell the client how to plug variables into a query string. The form represents infinitely many URIs, all of which follow a certain pattern. I’m going to extend my representations of places (like the one in Example 5-8) by including this HTML form (see Example 5-11).
<form id="searchPlace" method="get" action=""> <p> Show places, features, or businesses: <input id="term" repeat="template" name="show" /> <input class="submit" /> </p> </form></screen>
A person using a web browser would see this form as a set of GUI
elements: a text box, a button, and a set of labels. They’d put some
data into the text box, click the button, and be taken to another URI.
If they were at /Earth/USA/Mount%20Rushmore, and they’d typed
in “diners,” their web browser would make a GET request to /Earth/USA/Mount%20Rushmore?show=diners. An
automatic client can’t display the form to a human, but it would work
the same way. Given a preprogrammed desire to search a place, it would
look for the searchPlace form and use
the form definition as a guide to constructing the URI /Earth/USA/Mount%20Rushmore?show=diners.
Note
You probably haven’t seen the repeat="template" syntax before. It’s a
feature of XHTML 5, which is still being designed as this book goes to
press. Occasionally in this chapter and the next, I’ll introduce a
feature of XHTML 5 to work around the shortcomings of XHTML 4 as a
hypermedia format.
The problem here is that my service accepts any number of values
for the query variable show. A
client can make a simple search such as ?show=diners or perform a complicated
search such as
?show=diners&show=arsenic&show=towns&show=oil+tankers.
A form in XHTML 4 could allow the latter request if it showed
four text boxes, all called show.
But an HTML form can never show an arbitrary
number of text boxes, which is what I need to truly capture
the capabilities of my service. XHTML 5 has a feature called
the repetition model, which allows
me to express an arbitrary number of text boxes without writing an
infinitely long HTML page.
Now my service is better connected. It’s now possible to get from
the list of planets to a description of a diner near Mount Rushmore
(assuming there is one). Figure 5-4 illustrates the
journey. Starting at the service root (/), the client selects the planet Earth
(/Earth). The client uses the HTML
form in that representation to search for places called “Mount Rushmore”
on Earth (/Earth?show=Mount%20Rushmore). Hopefully the
top search result will be Mount Rushmore itself, and the client can
follow the first search result link to /Earth/USA/Mount%20Rushmore. The
representation of Mount Rushmore has a search form too, and the client
enters “diners” in that. Assuming there are any nearby diners, the
client can follow the first search result link to find a diner near
Mount Rushmore.
The search function doesn’t just keep clients from having to mint
their own URIs. It resolves human place names, which are always fuzzy,
into canonical resource URIs. A client should be also be able to search
for “Mount Rushmore National Monument” and get /Earth/USA/Mount%20Rushmore as a search
result, just like the client can search for “Springfield” and pick which
Springfield they mean. This is a useful feature for any client that lets
users type in their own place names.
The HTTP Response
I’m almost done with this design. I know what data I’m serving. I know which HTTP requests clients will send when asking for the data. I know how the data will be represented as I serve it. I still have to consider the HTTP response itself. I know what my possible representations will look like, and that’s what’s going in the entity-body, but I haven’t yet considered the possible response codes or the HTTP headers I’ll send. I also need to think about possible error conditions: cases where my response signals an error instead of delivering a representation.
What’s Supposed to Happen?
This step of the design is conceptually simple, but it’s the gateway to where you’re going to spend much of your implementation time: making sure that client requests are correctly turned into responses.
Most read-only resources have a pretty simple typical course of events. The user sends a GET request to a URI, and the server sends back a happy response code like 200 (“OK”), some HTTP headers, and a representation. A HEAD request works the same way, but the server omits the representation. The only main question is which HTTP headers the client should send in the request, and which ones the server should send in the response.
The HTTP response headers are a fairly large toolkit, and most
of them don’t apply to this simple service. (For descriptions of the
standard HTTP headers, see Appendix C.) In my
service, the main HTTP response header is Content-Type,
which tells the client the media type of the representation. My media
types are application/xhtml+xml for
the map representations and search results, and image/png for the map images. If you’ve done
any server-side web programming you already know about Content-Type: every HTTP server and
framework uses it.
I don’t use HTTP request headers very often. I think it’s best if the client can tweak the representation by tweaking the URI to the resource, rather than tweaking the request headers. But there is one set of headers that I think ought to be built into every HTTP client, every web service, and every service hacker’s brain: the ones that make conditional GET possible.
Conditional HTTP GET
Conditional HTTP GET saves client and server time and bandwidth. It’s
implemented with two response headers (Last-Modified and ETag), and two request headers (If-Modified-Since and If-None-Match).
I cover conditional GET in detail in Chapter 8, but the discussion there is somewhat detached from specific services. This discussion is tied to the map service, and covers just enough to get you thinking about conditional GET as you design your services.
Certain resources are likely to be very popular: “A road map of the United States,” “a satellite map of Earth,” or “restaurants in New York City.” A single client is likely to make a request for certain resources many times over its lifespan.
But this data is not constantly changing. Map data stays pretty constant over time. Satellite imagery is updated every few months at most. Restaurants come and go, but not on a minute-by-minute basis. Only a few resources are based on data that’s constantly changing. Most of the time, the client’s second and subsequent HTTP requests for a resource are wasted. They could have just reused the representation from their first request. But how are they supposed to know this?
This is where conditional GET comes in. Whenever a server
serves a representation, it should include a time value for the
Last-Modified HTTP header. This
is the last time the data underlying the representation was changed.
For “a road map of the United States,” the Last-Modified is likely to be the time the
map imagery was first imported into the service. For “restaurants in
New York City,” the Last-Modified
may only be a few days old: whenever a restaurant was last added to
the database of places. For “container ships near San Francisco,”
the value of Last-Modified may be
only a few minutes prior.
The client can store this value of Last-Modified and use it later. Let’s say
the client requests “a road map of the United States” and gets a
response that says:
Last-Modified: Thu, 30 Nov 2006 20:00:51 GMT
The second time the client makes a GET request for that
resource, it can provide that time in the If-Modified-Since header:
GET /road/Earth HTTP/1.1 Host: maps.example.com If-Modified-Since: Thu, 30 Nov 2006 20:00:51 GMT
If the underlying data changed between the two requests, the server sends a response code of 200 (“OK”) and provides the new representation in the entity-body. That’s the same thing that happens during a normal HTTP request. But if the underlying data has not changed, the server sends a response code of 304 (“Not Modified”), and omits any entity-body. Then the client knows it’s okay to reuse its cached representation: the underlying data hasn’t changed since the first request.
There’s a little more to it than that (again, I cover this in more detail in Chapter 8). But you can see the advantages. A client that fetches detailed maps is going to be making lots of HTTP requests. If most of those HTTP requests give a status code of 304, the client will be able to reuse old images and place lists instead of downloading new ones. Everyone saves time and bandwidth.
What Might Go Wrong?
I also need to plan for requests I can’t fulfill. When I hit an error condition I’ll send a response code in the 3xx, 4xx, or 5xx range, and I may provide supplementary data in HTTP headers. If they provide an entity-body, it’ll be a document describing an error condition, not a representation of the requested resource (which, after all, couldn’t be served).
I provide a full list of the HTTP response codes in Appendix B, along with examples where you might use each of them. Here are some likely error conditions for my map application:
The client may try to access a map that doesn’t exist, like
/road/Saturn. I understand what the client is asking for, but I don’t have the data. The proper response code in this situation is 404 (“Not Found”). I don’t need to send an entity-body along with this response code, though it’s helpful for debugging.The client may use a place name that doesn’t exist in my database. The end user might have mistyped the name, or used a name the application doesn’t recognize. They may have described the place instead of naming it, they might have the right name but the wrong planet. Or they might just be constructing URIs with random strings in them.
I can return a 404 response code, as in the previous example, or I can try to be helpful. If I can’t exactly match a requested place name, like
/Earth/Mount%20Rushmore%20National%20Monument, I might run it through my search engine and see if it comes up with a good match. If I do get a match, I can offer a redirect to that place: say,/Earth/43.9,-95.9/Mount%20Rushmore.The response code for the helpful case here would be 303 (“See Other”), and the HTTP response header
Locationwould contain the URI of the resource I think the client was “really” trying to request. It’s the client’s responsibility to take the hint and request that URI, or not.If I try a search and still have no idea what place the client is talking about, I’ll return a response code of 404 (“Not Found”).
The client may use logically impossible latitudes or longitudes, like
500,-181(500 degrees north latitude, 181 degrees west longitude). A 404 (“Not Found”) is a good response here, just as it is for a place that doesn’t exist. But a 400 (“Bad Request”) would be more precise.What’s the difference between the two cases? Well, there’s nothing obviously wrong with a request for a nonexistent place name like “Tanhoidfog.” It just doesn’t exist right now. Someone could name a town or a business “Tanhoidfog” and then it would be a valid place name. The client doesn’t know there’s no such place: one of the nice things a client can do with my map service is check to see which places really exist.
But there is something wrong with a request for the latitude/longitude pair
500,-181. The laws of geometry prevent such a place from ever existing. A minimally knowledgeable client could have figured that out before making the request. A 400 response code is appropriate in that case: the problem is the client’s fault for even making the request.A search for places on a map might return no search results. There might be no racing speedways near Sebastopol, CA. This is disappointing, but it’s not an error. I can treat this like any other search: send a 200 response code (“OK”) and a representation. The representation would include a link to the place that was searched, along with an empty list of search results.
The server may be overloaded with requests and unable to fulfil this particular request. The response code is 503 (“Service Unavailable”). An alternative is to refuse to handle the request at all.
The server may not be functioning correctly. This might be due to missing or corrupted data, a software bug, a hardware failure, or any of the other things that can go wrong with a computer program. In this case the response code is 500 (“Internal Server Error”).
This a frustrating response code (the whole 5xx series is frustrating, actually) because there’s nothing the client can do about it. Many web application frameworks automatically send this error code when an exception happens on the server side.
Conclusion
I’ve now got a design for a map web service that’s simple enough for a client to use without a lot of up-front investment, and useful enough to be the driver for any number of useful programs. It’s so closely attuned to the philosophy of the Web that you can use it with a web browser. It’s RESTful and resource-oriented. It’s addressable, stateless, and well connected.
It’s also read-only. It assumes that my clients have nothing to offer but their insatiable appetites for my data. Lots of existing web services work this way, but read-only web services are only half the story. In the next chapter I’ll show you how clients can use HTTP’s uniform interface to create new resources of their own.
[16] This trick is detailed in Nat Gertler’s enjoyable article, “Abusing Amazon Images”.
[17] This procedure has a lot in common with Joe Gregorio’s “How to create a REST Protocol”.
[18] Fun fact: prime meridians for planetary bodies are usually chosen by reference to some arbitrary feature like a crater. For bodies like Jupiter and Io, whose features are always changing, the prime meridian is defined according to which way the body was facing at an arbitrary time.
[19] A good reference for these algorithms is Ed Williams’s “Aviation Formulary”.
[20] You may have a private name for a seemingly boring point on the map, like “the cornfield where I kissed Betty.” This will come into play in Chapter 6 when I expand my web service so that clients can create their own place names. For now, I’ve got a preset database of names for each planet.
[21] Up to a point, anyway. See On Exactitude in Science by Jorge Luis Borges.
Get RESTful Web Services now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
