Chapter 4. HAL Clients

“It is a mistake to look too far ahead. Only one link of the chain of destiny can be handled at a time.”

Winston Churchill

As we saw in Chapter 2, JSON Clients, baking all Object, Address, and Action information into the client app means that even minor changes to the service API will be ignored by that client. The common practice is for services to do their best to make non-breaking changes to the API and then inform clients of the new features and hope that client developers will recode and redeploy their apps sometime soon. This can be acceptable when both the client and server teams are in the same company, but becomes harder to support as more and more client developers start building against a single web API—especially when those client developers are not part of the organization that is responsible for the web API releases.

Around 2010, API developers started to tackle this problem head on by defining some new message formats for API responses. These formats were based on JSON but were more structured. The media type designs included definitions for links and, in some cases, even for query and update actions. Essentially, these formats started to restore some of the lost features of HTML (links and forms) to the plain JSON message format that was so common for web APIs.

One of the most popular of these structured formats is the Hypertext Application Language (HAL). We’ll explore HAL in this chapter by building a general HAL client for our TPS web API and then, as before, modifying the web API to see how the HAL client holds up when the backend changes.

But first, let’s take a brief look at the HAL format itself.

The HAL Format

Mike Kelly designed the Hypertext Application Language (HAL) in early 2011 and registered it with the Internet Authority for Names and Addresses (IANA) in July 2011. Initially created to solve a specific problem Kelly was having with a product release, HAL has gone on to become one of the more popular of the new hypermedia-style media formats created in the last several years.

HAL’s design goal is rather simple—to make it easier for client applications to handle changes in backend service URLs. As he explained in a 2014 interview:

The roadmap for the product [I was working on] included significant backend changes which were going to affect URL structures in the API. I wanted to design the API so that we could roll out those changes with as little friction as possible, and hypermedia seemed like the ideal style for that.

For Kelly, the HAL model focuses on two concepts: Resources and Links. Links have an identifier and a target URL along with some additional metadata. Resources carry state data, while Links carry other (embedded) Resources. Figure 4-1 shows a diagram of the HAL model.

rwcl 0401
Figure 4-1. The HAL Design Model

Example 4-1 shows a simple HAL message (we’ll cover the details in the following section).

Example 4-1. A simple HAL message
{
  "_links": {
    "self": {"href": "/orders"},
    "next": {"href": "/orders?page=2"},
    "find": {"href": "/orders{?id}", "templated": true},
    "admin": [
      {"href": "/admins/2", "title": "Fred"},
      {"href": "/admins/5", "title": "Kate"}
    ]
  },
  "currentlyProcessing": 14,
  "shippedToday": 20,
  "_embedded": {
    order": [
      {
        "_links": {
          "self": {"href": "/orders/123"},
          "basket": {"href": "/baskets/98712"},
          "customer": {"href": "/customers/7809"}
        },
        "total": 30.00,
        "currency": "USD",
        "status": "shipped"
      },
      {
        "_links": {
          "self": {"href": "/orders/124"},
          "basket": {"href": "/baskets/97213"},
          "customer": {"href": "/customers/12369"}
        },
        "total": 20.00,
        "currency": "USD",
        "status": "processing"
      }
    ]
  }
}

The model is very simple and, at the same time, powerful. With HAL, Kelly introduces the idea that a link is more than just a URL. It also has an identifier (rel) and other important metadata (e.g., type, name, title, templated and other properties). Kelly also points out in his 2013 blog post that he had some other important goals when designing HAL. These can be summed up as:

  • HAL reduces coupling between client and server through the _link elements.

  • HAL’s _link convention makes the APIs “browsable by developers.”

  • The practice of connecting HAL’s _link elements to human-readable documentation “makes the API discoverable by developers.”

  • Services can use HAL _link elements to introduce changes “in a granular way.”

When considering the three important things API clients need to deal with— Objects, Addresses, and Actions—we can see that HAL is optimized to support varying Addresses (Links) at runtime. HAL’s use of the _link as a key design element makes it possible for services to change URL values without breaking client applications.

HAL and the OAA Challenge

While HAL makes changing the Addresses easy, its design doesn’t attempt to optimize for changes in the exposed Objects or the associated Actions on those objects. That’s not what Kelly was aiming for. We’ll look at other hypermedia-style formats that tackle the other aspects of the web client challenge in later chapters.

Objects and Properties

HAL also supports passing plain JSON objects and name–value pairs. These typically appear as a collection of properties at the root of a HAL document. For example, the Amazon API Gateway service emits HAL responses. Following are a couple of examples of the 26 resource models the AWS Gateway API produces:

{
  "domainName" : "String",
  "certificateName" : "String",
  "certificateUploadDate" : "Timestamp",
  "distributionDomainName" : "String"
}
{
  "id" : "String",
  "description" : "String",
  "createdDate" : "Timestamp",
  "apiSummary" : {
    "String" : {
      "String" : {
        "authorizationType" : "String",
        "apiKeyRequired" : "Boolean"
      }
    }
  }
}

As we discussed in Chapter 2, JSON Clients, tracking domain objects is an important aspect of API clients. By design, the HAL format does not offer any additional design elements to make this any different than working with plain JSON responses. So HAL clients will need to know all the possible objects and models the API will be emitting before that client app can interact with the API.

Embedded Links and Objects

Finally, the HAL design allows for additional resources (and their associated links and properties) to appear as embedded models within responses. This makes it possible to return a single HAL document that includes multiple server-side resources. This acts as a way to reduce the number of HTTP requests and improves the perceived responsiveness of the service.

Following is a more extensive HAL document that includes all the features we’ve covered so far:

{
  "_links": {
    "self": { "href": "/orders" },
    "curies": [{ "name":"ea", "href":"http://example.com/docs/rels/{rel}", 1
      "templated": true }],
    "next": { "href": "/orders?page=2" },
    "ea:find": {
      "href": "/orders{?id}",
      "templated": true
    },
    "ea:admin": [{
      "href": "/admins/2",
      "title": "Fred"
    }, {
      "href": "/admins/5",
      "title": "Kate"
    }]
  },
  "currentlyProcessing": 14,
  "shippedToday": 20,
  "_embedded": {
    "ea:order": [{
      "_links": {
        "self": { "href": "/orders/123" },
        "ea:basket": { "href": "/baskets/98712" },
        "ea:customer": { "href": "/customers/7809" }
      },
      "total": 30.00,
      "currency": "USD",
      "status": "shipped"
    }, {
      "_links": {
        "self": { "href": "/orders/124" },
        "ea:basket": { "href": "/baskets/97213" },
        "ea:customer": { "href": "/customers/12369" }
      },
      "total": 20.00,
      "currency": "USD",
      "status": "processing"
    }]
  }
}

The preceding example also shows HAL’s curies support (1). HAL uses the W3C Compact URI Syntax (CURIES) as a way to shorten long unique link identifiers. This is an optional HAL element but one that you may encounter “in the wild.”

CURIES

The CURIES syntax was documented in 2010 by a W3C Working Group Note. The acronym CURIE (pronounced cure-ee) stands for Compact URI (the E is added for pronunciation). It was created by the XHTML group and is used in XML and RDF documents similar to the way XML namespaces are used. You can learn more about CURIEs by reading the W3C documents.

Quick Summary

So the HAL hypermedia format focuses on making it possible for services to change URLs at runtime without breaking client applications. It does this by formalizing the way URLs are expressed in responses using the HAL Link element (e.g., "_links:{"self":{"href":"/home/"}}). HAL also supports passing state data in plain JSON properties or objects and provides the ability to nest Resources using the "_embedded":{…} element.

That gives us enough information to add HAL support for our TPS web API. First we’ll modify the TPS API to emit HAL-compliant responses. Then we’ll build a general HAL client that reads those responses.

The HAL Representor

As we learned in Chapter 3, The Representor Pattern, our TPS server allows us to separately create representor modules based on the WeSTL format to support multiple output formats. For our TPS web API to support HAL clients, we first need to write that representor module and add it to the existing service in production. This can be done in just a few hundred lines of server-side JavaScript by accounting for the three main aspects of the HAL format:

  • Links

  • Properties

  • Embedded resources

Tip

The source code for the HAL edition of the TPS web API can be found in the associated GitHub repo. A running version of the app described in this chapter can be found online.

The top-level routine in the representor creates the empty HAL document and populates it with Links, Properties, and (if available) embedded objects. The code looks like Example 4-2.

Example 4-2. The top-level routine in the HAL representor
function haljson(wstlObject, root, relRoot) {
  var hal;

  hal = {};
  hal._links = {};

  for(var segment in wstlObject) {
    hal._links = getLinks(wstlObject[segment], root, segment, rels); 1
    if(wstlObject[segment].content) { 2
      hal.content = wstlObject[segment].content;
    }
    if(wstlObject[segment].related) { 3
      hal.related = wstlObject[segment].related;
    }
    if(wstlObject[segment].data && wstlObject[segment].data.length===1) { 4
      hal = getProperties(hal, wstlObject[segment]);
    }
    else {
      hal._embedded = getEmbedded(wstlObject[segment], root, segment, rels); 5
    }
  }
  return JSON.stringify(hal, null, 2); 6
}

After initializing an empty HAL document, this routine does the following:

1

Add all the related Links to the _link collection.

2

If there is any associated content for this response, add that as a root-level property.

3

If there are any related records for this response, add them as another property.

4

If there is only one data record associated with the response, add it as a set of properties.

5

Otherwise, add the collection of data objects to the optional _embedded element in the HAL document.

6

Finally, the results are returned as a plain-text JSON object for sending back to the client over HTTP.

That’s it. Not too hard. Now let’s look at the main subroutines for generating HAL responses: Links, Properties, and Embedded.

Links

The getLinks routine searches through the runtime internal WeSTL document and composes the appropriate link elements for the response. It looks like this:

// emit _links object
function getLinks(wstlObject, root, segment, relRoot) {
  var coll, items, links, i, x;

  links = {};

  // list-level actions
  if(wstlObject.actions) { 1
    coll = wstlObject.actions;
    for(i=0,x=coll.length;i<x;i++) {
      links = getLink(links, coll[i], relRoot);
    }

    // list-level objects
    if(wstlObject.data) { 2
      coll = wstlObject.data;
      items = [];
      for(i=0,x=coll.length;i<x;i++) {
        item = {};
        link = getItemLink(wstlObject.actions);
        if(link.href) {
          item.href = link.href.replace(/{key}/g, coll[i].id)||"#";
        }
        item.title = coll[i].title||coll[i].nick;
        items.push(item);
      }
      links[checkRel(segment, relRoot)] = items; 3
    }
  }
}

There are just a few interesting elements to the getLinks routine:

1

If there are actions in the WeSTL document, compose them as valid HAL links (that’s the getLink routine) and add them to the document.

2

If there is data associated with the response, walk through all the objects and resolve any direct item link data and add it to the document.

3

Finally, check each of the links (with the checkRel routine) to see if it’s a registered IANA link relation value or an application-specific link identifier.

That last step makes sure to follow the rules for link relations (from RFC5988) and include a fully qualified domain name (FQDN) to the link’s rel value for any identifiers that are not already registered.

Properties

The next step is to emit any HAL properties in the response. Our TPS API could emit some plain HTML (via the internal content element). It may also have some related data objects (via the internal related object collection) to help support drop-down lists or other rendering. We also need to emit root-level HAL properties if the response has just one associated data object. This is not a requirement of HAL—it just makes emitting compliant HAL documents easy for our representor.

Warning

The content and related elements are part of the TPS web API and are not defined by HAL. I’m using them here to make sure the new HAL edition of the TPS web API provides the same features and functionality as the JSON edition.

The code that handles the content and related elements appears in the haljson routine (refer back to Example 4-2). The code that emits the data object as properties (getProperties) looks like this:

// emit root properties
function getProperties(hal, wstlObject) {
  var props;

  if(wstlObject.data && wstlObject.data[0]) { 1
    props = wstlObject.data[0]; 2
    for(var p in props) {
      hal[p] = props[p]; 3
    }
  }
  return hal;
}

The getProperties routine does the following:

1

If there is only one data object in the resource collection…

2

Get an index collection of that object’s property names.

3

Add each property name–value pair to the HAL document.

Embedded

The last step in supporting valid HAL responses is dealing with the optional _embedded section. This element of the HAL design is meant to optimize long lists of HAL link elements by including the associated objects in the response. This is a kind of internal caching helper. For our HAL representor, we will generate an _embedded section (using the getEmbedded function) whenever the resource behind the HAL response has multiple objects in the internal WeSTL data collection.

The code looks like this:

// emit embedded content
function getEmbedded(wstlObject, root, segment, relRoot) {
  var coll, items, links, i, x;

  links = {};

  // list-level objects
  if(wstlObject.data) {
    coll = wstlObject.data;
    items = [];
    for(i=0,x=coll.length;i<x;i++) { 1
      item = {};
      link = getItemLink(wstlObject.actions); 2
      if(link.href) {
        item.href = link.href.replace(/{key}/g, coll[i].id)||"#"; 3
      }
      for(var p in coll[i]) {
        item[p] = coll[i][p]; 4
      }
      items.push(item); 5
    }
    links[checkRel(segment, relRoot)] = items;
  }

  return links; 6
}

There are a number of things going on here. After grabbing the collection of resource data objects, the getEmbedded routine does the following:

1

Walk through the object collection to create an embedded item object.

2

Pull that item’s direct link (using the getItemLink function).

3

Resolve the direct link (if there is a URI Template).

4

Populate the embedded item with the internal object’s properties.

5

Add the completed item to the embedded collection.

6

Finally, after checking the item link for RFC5988 compliance, return the _embedded collection for inclusion in the HAL document.

There are a couple of internal routines not shown here. They handle finding the data object’s direct link template (getItemLink), creating valid HAL links (getLink), and RFC5988 compliance (checkRel). You can check the source code for details behind these.

Sample TPS Output from the HAL Representor

With the HAL representor completed, the TPS web API now emits proper HAL representations. Example 4-3 shows the output from the TPS server for the Home resource.

Example 4-3. HAL Response for TPS web API Home Resource
{
  "_links": {
    "self": {
      "href": "http://rwcbook06.herokuapp.com/home/",
      "title": "Home",
      "templated": false,
      "target": "app menu hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-home-task": {
      "href": "http://rwcbook06.herokuapp.com/task/",
      "title": "Tasks",
      "templated": false,
      "target": "app menu hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-home-user": {
      "href": "http://rwcbook06.herokuapp.com/user/",
      "title": "Users",
      "templated": false,
      "target": "app menu hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-home-home": []
  },
  "content": "<div class=\"ui segment\"><h3>Welcome to TPS at BigCo!</h3> _
  <p><b>Select one of the links above.</b></p></div>",
  "related": {},
  "_embedded": {
    "http://rwcbook06.herokuapp.com/files/hal-home-home": []
  }
}

And Example 4-4 shows the HAL output for a single TPS Task object.

Example 4-4. A single TPS Task object as a HAL document
{
  "_links": {
    "http://rwcbook06.herokuapp.com/files/hal-task-home": {
      "href": "http://rwcbook06.herokuapp.com/home/",
      "title": "Home",
      "templated": false,
      "target": "app menu hal"
    },
    "self": {
      "href": "http://rwcbook06.herokuapp.com/task/",
      "title": "Tasks",
      "templated": false,
      "target": "app menu hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-user": {
      "href": "http://rwcbook06.herokuapp.com/user/",
      "title": "Users",
      "templated": false,
      "target": "app menu hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-item": {
      "href": "http://rwcbook06.herokuapp.com/task/{key}",
      "title": "Detail",
      "templated": true,
      "target": "item hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-edit": {
      "href": "http://rwcbook06.herokuapp.com/task/{key}",
      "title": "Edit Task",
      "templated": true,
      "target": "item edit hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-remove": {
      "href": "http://rwcbook06.herokuapp.com/task/{key}",
      "title": "Remove Task",
      "templated": true,
      "target": "item edit hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-markcompleted": {
      "href": "http://rwcbook06.herokuapp.com/task/completed/{id}",
      "title": "Mark Completed",
      "templated": true,
      "target": "item completed edit post form hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-assignuser": {
      "href": "http://rwcbook06.herokuapp.com/task/assign/{id}",
      "title": "Assign User",
      "templated": true,
      "target": "item assign edit post form hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-task": [
      {
        "href": "//rwcbook06.herokuapp.com/task/1m80s2qgsv5",
        "title": "Run client-side tests"
      }
    ]
  },
  "id": "1m80s2qgsv5",
  "title": "Run client-side tests",
  "tags": "test",
  "completeFlag": "false",
  "assignedUser": "alice",
  "dateCreated": "2016-01-28T07:14:07.775Z",
  "dateUpdated": "2016-01-31T16:49:47.792Z"
}
Note

The output would be greatly improved if my representor used CURIEs for the URL keys. CURIEs don’t make the service or the client work better, but they make the API responses much more browsable and discoverable for developers. And, you may recall from Mike Kelly’s design guidelines, these were two very important goals in the design of HAL messages. I leave it to readers to improve my simple HAL representor by adding support for CURIEs and submitting the update to GitHub.

Note that the output includes link objects marked with the target property. This is not a defined HAL property. Our TPS server emits this property to help the HAL client know how to handle the link—whether it should be rendered at the top of every page ("app"), just for object lists ("list"), or only when there is a single object to display ("item"). We’ll cover more of that in the next section of this chapter.

The HAL SPA Client

OK, now let’s walk through the HAL client SPA implementation. As we did in Chapter 2, JSON Clients, we’ll review the HTML container, the top-level parse loop, and how our client handles HAL’s Links, Properties, and Embedded elements.

Tip

The source code for the HAL client can be found in the associated GitHub repo. A running version of the app described in this chapter can be found online.

The HTML Container

Like all single-page apps (SPAs), this one starts with a single HTML document acting as the container for all the client–server interactions. Our HAL client container looks like this:

<!DOCTYPE html>
<html>
  <head>
    <title>HAL</title>
    <link href="./semantic.min.css" rel="stylesheet" />
    <style>#dump {display:none;}</style>
  </head>
  <body>
    <div id="toplinks"></div> 1
    <h1 id="title" class="ui page header"></h1>
    <div id="content"></div>
    <div id="links"></div>
    <div class="ui two column grid" style="margin-top: 2em">
      <div class="column">
        <div id="embedded" class="ui segments"></div>
        <div id="properties"></div>
      </div>
      <div class="column">
        <div id="form"></div>
      </div>
    </div>
    <div>
      <pre id="dump"></pre>
    </div>
  </body>
  <script src="uritemplate.js">//na</script>  2
  <script src="dom-help.js">//na</script>
  <script src="hal-client.js">//na </script>
  <script>
    window.onload = function() {
      var pg = hal();
      pg.init("/home/", "TPS - Task Processing System"); 3
    }
  </script>
</html>

Most of the content should look familiar. Our HTML layout for the app (starting at 1) is a bit more involved than in previous apps in order to accommodate the HAL elements (links, properties, embedded) but the rest is similar to our JSON client app. Note that there are three scripts for this client (2). The familiar dom-help.js file, the expected hal-client.js library and a new JS module: uritemplate.js. The HAL design relies upon RFC5988 (URI Template) compliant links so we’ll use a standard library to handle these.

Finally, at 3, the initial URL is supplied and the hal-client.js library is kicked off. We’ll spend the rest of our walk-through in the hal-client.js file.

The Top-Level Parse Loop

After making the initial HTTP request, the response is passed to the parseHAL routine. Like all our SPA apps, the HAL client relies on a simple request, parse, render loop. Here’s the top-level parse loop for this app.

  // primary loop
  function parseHAL() {
    halClear();
    title();
    setContext(); 1
    if(g.context!=="") { 2
      selectLinks("app", "toplinks");
      selectLinks("list", "links");
      content();
      embedded();
      properties();
    }
    else {
      alert("Unknown Context, can't continue"); 3
    }
  }

After clearing the user interface of any previously rendered content and setting the title, the web client kicks off a set of HAL-specific routines. Here are the highlights:

1

First, the setContext routine determines which Object is being returned (Home, Task, or User).

2

If we have a valid context, the HAL-specific elements are parsed and rendered (including filtering the link pile for display in the proper locations).

3

Finally, if the service returned a context the client is not prepared to deal with, an alert is displayed.

Note that we’re hardcoding the Object knowledge into this app (via setContext). We are also expecting special information from the service about each link object to know how and where to display it. We’ll see more on that in the next section.

Links

HAL’s strength is the ability to return well-identified link objects for API clients. For our client, the selectLinks routine parses the HAL _links collection and determines how to build and render the link information in the proper place on the screen.

Here’s the code:

// select and render HAL links
function selectLinks(filter, section, itm) { 1
  var elm, coll;
  var menu, item, a, sel, opt, id;

  elm = d.find(section);
  d.clear(elm);
  if(g.hal._links) { 2
    coll = g.hal._links;
    menu = d.node("div");
    menu.className = "ui blue menu";

    for(var link in coll) {
      if(coll[link].target && coll[link].target.indexOf(filter)!==-1) { 3
        id = (itm && itm.id?itm.id:"");

        a = d.anchor({ 4
          rel:link,
          href:coll[link].href.replace('{key}',id),
          title:(coll[link].title||coll[link].href.replace('{key}',id)),
          text:(coll[link].title||coll[link].href.replace('{key}',id))
        });

        // add internal attributes 5
        a.setAttribute("templated", coll[link].templated||"false");
        a = halAttributes(a,coll[link]);

        item = d.node("li");
        item.onclick = halLink; 6
        item.className = "item";

        d.push(a, item);
        d.push(item, menu);
      }
    }
    d.push(menu, elm); 7
  }
}

There is a lot going on in this routine. That’s because most of the information in a HAL response is stored in the _links collection. Let’s do the walk-through…

First (at 1) the signature of the selectLinks function takes up to three values: The string (filter) to use against the our custom link.target field, the name of the HTML DOM element (section) where we’ll render the links, and (optionally) the current data object (itm) to use when resolving item-level links. We’ll see this last argument in use when we scan the code for handling HAL’s _embedded collection.

After making sure this response actually has links (2) and filtering the collection to match our needs (3), the code builds up an HTML <a>…</a> element (4), adds some local attributes the code uses to handle user clicks at runtime, marks the link if it is using a URL template (5), and then (at 6) attaches a local click event (halLink) that can sort out just how to handle the requested action. Finally, (7) the resulting links are added to the HTML DOM and rendered on the screen.

Figure 4-2 shows how the elements in the HAL _links collection is rendered on the screen for the list of Task objects.

rwcl 0402
Figure 4-2. Rendering HAL _links for task objects

Embedded

HAL’s _link collection doesn’t just contain a list of action links (like the ones rendered before). The HAL response may also contain one or more links to associated resources. For example, the TPS web API returns a list of Task objects as link elements in a HAL response. The same response also includes the associated Task objects in the _embedded section:

{
  "_links": {
    "self": {
      "href": "http://rwcbook06.herokuapp.com/task/",
      "title": "Tasks",
      "templated": false,
      "target": "app menu hal"
    },
    "http://rwcbook06.herokuapp.com/files/hal-task-task": [
      {
        "href": "//rwcbook06.herokuapp.com/task/1m80s2qgsv5",
        "title": "Run client-side tests"
      },
      {
        "href": "//rwcbook06.herokuapp.com/task/1sog9t9g1ob",
        "title": "Run server-side tests"
      },
      {
        "href": "//rwcbook06.herokuapp.com/task/1xya56y8ak1",
        "title": "Validate client-side code changes"
      }
    ]
  },
  "_embedded": {
    "http://rwcbook06.herokuapp.com/files/hal-task-task": [
      {
        "href": "//rwcbook06.herokuapp.com/task/1m80s2qgsv5",
        "id": "1m80s2qgsv5",
        "title": "Run client-side tests",
        "tags": "test",
        "completeFlag": "false",
        "assignedUser": "alice",
        "dateCreated": "2016-01-28T07:14:07.775Z",
        "dateUpdated": "2016-01-31T16:49:47.792Z"
      },
      {
        "href": "//rwcbook06.herokuapp.com/task/1sog9t9g1ob",
        "id": "1sog9t9g1ob",
        "title": "Run server-side tests",
        "tags": "test",
        "completeFlag": "false",
        "assignedUser": "",
        "dateCreated": "2016-01-28T07:16:53.044Z",
        "dateUpdated": "2016-01-28T07:16:53.044Z"
      },
      {
        "href": "//rwcbook06.herokuapp.com/task/1xya56y8ak1",
        "id": "1xya56y8ak1",
        "title": "Validate client-side code changes",
        "completeFlag": "true",
        "assignedUser": "",
        "dateCreated": "2016-01-14T17:46:32.384Z",
        "dateUpdated": "2016-01-16T04:50:57.618Z"
      }
    ]
  }
}

Our HAL client takes advantage of the _embedded objects and renders them on screen. That’s handled by the embedded routine, which looks like this:

// handle any embedded content
function embedded() {
  var elm, embeds, links;
  var segment, table, tr;

  elm = d.find("embedded");
  d.clear(elm);

  if(g.hal._embedded) {
    embeds = g.hal._embedded;
    for(var coll in embeds) { 1

      p = d.para({text:coll, className:"ui header segment"});
      d.push(p,elm);

      // get all the objects in each collection
      items = embeds[coll];
      for(var itm of items) { 2
        segment = d.node("div");
        segment.className = "ui segment";
        links = d.node("div");
        links.id = itm.id;
        d.push(links,segment);

        // emit all the properties for this item
        table = d.node("table"); 3
        table.className = "ui very basic collapsing celled table";
        for(var prop of g.fields[g.context]) {
          if(itm[prop]) {
            tr = d.data_row({className:"property "+prop,text:prop+"&nbsp;",
            value:itm[prop]+"&nbsp;"});
            d.push(tr,table);
          }
        }

        // push the item element to the page
        d.push(table,segment);
        d.push(segment, elm);

        // emit any item-level links
        selectLinks("item",itm.id, itm); 4
      }
    }
  }
}

Here is a quick rundown of what’s happening in this routine:

1

Our embedded routine is able to handle multiple object collections in the response since the HAL spec allows this.

2

After collecting a set of objects, the code loops through each item in the collection for rendering.

3

Each of the embedded objects has their properties rendered within an HTML table.

4

And, lastly, any item-level links (from the _link collection) associated with the object are rendered, too.

Caching or Object Lists?

It should be noted that our HAL client treats the _embedded section as a list of one or more object collections. This (technically) pushes the boundaries of caching intent of the HAL _embedded design. We can get by with that here, but it might not work with just any service that returns HAL responses.

Figure 4-3 shows a screenshot of our HAL client rendering a list of User objects.

rwcl 0403
Figure 4-3. Rendering a list of user objects with the HAL client

We have one more HAL element to deal with: Properties.

Properties

The HAL design includes support for sending objects or name–value pairs at the root level of the response. We saw an example of this earlier in Example 4-1. Our routine to handle these root elements is the properties function:

// emit any root-level properties
function properties() {
  var elm, coll;
  var segment, table, tr;

  elm = d.find("properties");
  d.clear(elm);
  segment = d.node("div");
  segment.className = "ui segment";
  if(g.hal && g.hal.id) { 1
    links = d.node("div");
    links.id = g.hal.id;
    d.push(links,segment);
  }

  table = d.node("table");
  table.className = "ui very basic collapsing celled table";

  for(var prop of g.fields[g.context]) { 2
    if(g.hal[prop]) {
      tr = d.data_row({className:"property "+prop,text:prop+"&nbsp;",
        value:g.hal[prop]+"&nbsp;"});
      d.push(tr,table);
    }
  }

  d.push(table,segment);
  d.push(segment,elm);

  // emit any item-level links
  if(g.hal && g.hal.id) {
    selectLinks("item",g.hal.id, g.hal); 3
  }
}

This routine is a bit less busy than the one handling the _links section.

First (at 1), we confirm that we have one or more root level properties. Note the check for hal.id. This is a specific bit of knowledge of TPS Objects (they always have an id property) baked into the client. This would not work with just any HAL service response. Once we have a set of properties, we loop through the internal collection of fields for this context (2) using the g.fields[g.context] state value and only display properties the client knows about ahead of time. Finally, at 3, we insert any item-level links associated with this object. This is another TPS-specific element.

Figure 4-4 shows the HAL client rendering a single Task record.

That covers all the HAL response elements. But we still have a very important aspect of the web client that we have not shown: handling the Actions for Tasks and Users. You may recall that HAL doesn’t include any Action metadata in responses. It is up to the client application to handle these details.

rwcl 0404
Figure 4-4. Rendering a single item in HAL

Handling Actions for HAL

Since HAL doesn’t include Action details such as HTTP methods and arguments, our client needs to handle these instead. For the JSON client, we used a set of action objects to hold all the information (see “Addresses”). In the HAL client, we’ll do something similar. Since HAL links all have a unique identifier, we can use that identifier as a pointer to a list of Action definitions in code. And we can write a short routine to access these definitions. I created a small halForms JavaScript library for this.

Here is an example of the halForms Action definition:

{
  rel:"/files/hal-task-edit", 1
  method:"put", 2
  properties:   [ 3
    {name:"id",required:true, value:"{id}", prompt:"ID", readOnly:true},
    {name:"title",required:true, value:"{title}", prompt:"Title"},
    {name:"tags", value:"{tags}", prompt:"Tags"},
    {name:"completeFlag",value:"{completeFlag}", prompt:"Completed"},
    {name:"assignedUser",value:"{assignedUser}", prompt:"Assigned User"}
  ]
};

Note the use of the rel property to hold the Action identifier at 1. This is used to match the same value found in a link element in the _links section of a HAL response. The HTTP method to use (2) and all the input arguments (3) are included, too. There is an Action definition for every query and write operation documented for the TPS web API. These are then accessed at runtime when someone clicks on an action link in the user interface.

To use these at runtime in the HAL client, every link in the UI is tied to a single function—the halLink function. That function looks like this:

// handle GET for links
function halLink(e) {
  var elm, form, href, accept;

  elm = e.target;
  accept = elm.getAttribute("type");

  form = forms.lookUp(elm.rel); 1
  if(form && form!==null) {
    halShowForm(form, elm.href, elm.title); 2
  }
  else {
    req(elm.href, "get", null, null, accept||g.ctype); 3
  }
  return false;
}

As you can see, when a person clicks on a link in the HAL client, this code checks to see if there is an Action definition available (1) and, if it is, shows the HTML FORM in the UI (2). Otherwise, the HAL client just executes a simple HTTP GET on the clicked link (3).

The halShowForm routine knows how to convert the Action definition into a valid HTML FORM and can associate any current task or user record with the inputs (to make it easy to render existing objects for editing). Figure 4-5 shows a screen showing the TaskEdit operation at runtime.

rwcl 0405
Figure 4-5. Handling inputs in the HAL client

Quick Summary

In building our HAL client, we learned how to create a general HAL response parser that would render the data in a user interface. Along the way, we built a few key HAL-aware operations into our general library:

selectLinks

This routine was used to find and filter the collection of link elements in the HAL _links collection and render them when and where they are needed.

embedded

We used the HAL _embedded section to carry object lists (tasks and users) and wrote code that rendered these at runtime as well as any associated item-level links (using the selectLinks function again).

properties

Finally, the properties function handles any root-level name–value pairs (or objects) and also uses the selectLinks routine to render any item-level links associated with the root object.

There is more to the hal-client.js library that we didn’t cover here including the same low-level HTTP routines we used in all the other SPA examples and some HAL-specific functions to support URI Templates, manage screen display, and handle other small chores.

Now that we have a fully functioning HAL client, let’s introduce some change on the service API to see how our client adapts.

Dealing with Change

Just as we did in Chapter 2, JSON Clients, we’ll now test the HAL client to see how it stands up to changes that occur on the backend API after the initial release. We saw that the JSON client didn’t do very well. When the service added a new data field and filter option, the JSON client just ignored them. It didn’t break, but we needed to recode and redeploy the app before the new API functionality would appear in the client.

From our earlier review of HAL and the OAA Challenge, we know that the HAL design is focused on handling changes to Addresses. As long as we keep the initial URL for the API the same—the entry URL—the HAL spec allows us to change all the other URLs and the client will work just fine.

For example, if the TPS service changed the URL used for handling the ChangePassword operation from /user/pass/{id} to user/changepw/{id}, the HAL application would continue working. That’s because the HAL client doesn’t have the actual operation URLs baked into the code.

However, since HAL responses do not include Object metadata or Action definitions, changes to these elements—even adding them—can cause problems for HAL client apps.

Adding an Action

For our example, let’s assume the TPS team decides to add a new Action element to the web API—the TaskMarkActive operation. This one allows people to mark any single Task object as “active” by setting the completeFlag="false". We can do something similar with the TaskMarkCompleted already.

Updating the docs

So, we can add the following Action definition to the API documentation (Table 4-1).

Table 4-1. TPS TaskMarkActive action
Operation URL Method Returns Inputs

TaskMarkActive

/task/active/{id}

POST

TaskList

none
Tip

The source code for the HAL client with the new MarkActive feature built in can be found in the associated GitHub repo. A running version of the app described in this chapter can be found online.

Updating the TPS web API

With the docs done, we can update our server-side task connector with the new functionality (see 1):

case 'POST':
  if(parts[1] && parts[1].indexOf('?')===-1) {
    switch(parts[1].toLowerCase()) {
      case "completed":
        markCompleted(req, res, respond, parts[2]);
        break;
      case "active":
        markActive(req, res, respond, parts[2]); 1
        break;
      case "assign":
        assignUser(req, res, respond, parts[2]);
        break;
      default:
        respond(req, res,
          utils.errorResponse(req, res, 'Method Not Allowed', 405)
        );
    }

Now, when we make a request for a single task record against the API—for example, GET /task/1m80s2qgsv5)—we’ll see the new TaskMarkActive appear (1 in the following code):

{
  "_links": {
    "collection": {
      "href": "http://localhost:8181/task/",
      "title": "Tasks",
      "templated": false,
      "target": "app menu hal"
    },
    "http://localhost:8181/files/hal-task-item": {
      "href": "http://localhost:8181/task/{key}",
      "title": "Detail",
      "templated": true,
      "target": "item hal"
    },
    "http://localhost:8181/files/hal-task-edit": {
      "href": "http://localhost:8181/task/{key}",
      "title": "Edit Task",
      "templated": true,
      "target": "item edit hal"
    },
    "http://localhost:8181/files/hal-task-remove": {
      "href": "http://localhost:8181/task/{key}",
      "title": "Remove Task",
      "templated": true,
      "target": "item edit hal"
    },
    "http://localhost:8181/files/hal-task-markcompleted": {
      "href": "http://localhost:8181/task/completed/{id}",
      "title": "Mark Completed",
      "templated": true,
      "target": "item completed edit post form hal"
    },
    "http://localhost:8181/files/hal-task-assignuser": {
      "href": "http://localhost:8181/task/assign/{id}",
      "title": "Assign User",
      "templated": true,
      "target": "item assign edit post form hal"
    },
    "http://localhost:8181/files/hal-task-markactive": { 1
      "href": "http://localhost:8181/task/active/{id}",
      "title": "Mark Active",
      "templated": true,
      "target": "item active edit post form hal"
    },
    "http://localhost:8181/files/hal-task-task": [
      {
        "href": "//localhost:8181/task/1m80s2qgsv5",
        "title": "Run client-side tests"
      }
    ]
  },
  "id": "1m80s2qgsv5",
  "title": "Run client-side tests",
  "tags": "test",
  "completeFlag": "false",
  "assignedUser": "alice",
  "dateCreated": "2016-01-28T07:14:07.775Z",
  "dateUpdated": "2016-01-31T22:30:25.578Z"
}

The failing HAL client

The new link (“Mark Active”) will appear in the HAL client automatically since HAL understands how to deal with links. However, since the TaskMarkActive operation requires using an HTTP POST, the HAL client fails (see Figure 4-6).

rwcl 0406
Figure 4-6. HAL client fails on new Action

HAL responses don’t include Action definitions. For this new functionality, it is not enough to recognize the link, the client also needs to know what to do with it. And that only appears in the human documentation.

Recoding the Action definition on the HAL client

To fix this, we need to translate the new API documentation into a new Action definition and recode and redeploy our app again:

{
  rel:"/files/hal-task-markactive", 1
  method:"post",
  properties: [
    {name:"id",value:"{id}", prompt:"ID",readOnly:true},
    {name:"completeFlag",value:"false", prompt:"Completed",readOnly:true}
  ]
};

Note in the preceding Action definition (1), the rel value is set to match that of the link identifier in the HAL response output by the TPS web API.

Now, as Figure 4-7 shows, the HAL client knows how to handle the new link, shows the proper HTML form, and successfully marks the Task object as “active.”

rwcl 0407
Figure 4-7. New Action definition for HAL client

We can actually improve the HAL client’s ability to deal with unexpected Actions by relying on a custom HAL extension. We’ll take a quick look at that in the last section of this chapter.

The HAL-FORMS Extension

One way to improve the HAL client is to extend the HAL format to include more Object or Action information than the current design contains. This kind of extension is not explicitly described in the original HAL spec. However, as long as your extensions do not break existing clients or redefine existing features of HAL, they should be OK.

For this book, I created an extension to hold all the Action definitions that I’ve been adding to code. With this extention in place and a client that understands how to use it, it is possible to add new Action definitions at runtime without having to recode and redeploy the HAL client.

Tip

The source code for the HAL client that supports HAL-FORMS can be found in the associated GitHub repo. A running version of the app described in this chapter can be found online.

The Specification

The HAL-FORMS specification is a more formalized version of the in-app Action definitions used earlier in this chapter. However, that simple JSON object has been updated to be more in line with Mike Kelly’s HAL specification and offers a few more future possibilities including the ability to load the definitions on-demand at runtime.

Tip

I won’t go into the details on the HAL-FORMS extension here—you can read all about it in the associated GitHub repo and read the latest documentation online. For now, I’ll just show what a HAL-FORMS document looks like and then get into how it’s used in our updated HAL client.

A HAL-FORMS document

Here is the standalone HAL-FORMS document for adding a task to the TPS system:

{
  "_links" : {
    "self": {
      "href": "/files/hal-task-create-form" 1
    }
  },
  "_templates" : {
    "default" : {
      "title" : "Add Task",
      "method":"post", 2
      "properties": [ 3
        {"name":"title", "required":true, "value":"", "prompt":"Title"},
        {"name":"tags", "value":"", "prompt":"Tags"},
        {"name":"completeFlag", "required":false,
        "value":"false", "prompt":"Completed"}
      ]
    }
  }
}

In the preceding document three things are worth pointing out:

1

The rel value used to locate this document is included here since it is standard practice to include a "self" reference in HAL docs.

2

The HTTP method is included. This is usually POST, PUT, or DELETE but might be a GET for complex query forms.

3

The properties array contains all the details for rendering the HTML form and then sending the data to the service.

Notice that there is no href in this document. That is because the client app gets the Address for this operation from the HAL document at runtime. That makes these HAL-FORMS a bit more reusable, too.

Requesting HAL-FORMS documents

The HAL-FORMS document are returned with a unique IANA media type string (application/prs.hal-forms+json) in HTTP’s content-type header. That’s also how HAL-FORMS documents are requested—using the application/prs.hal-forms+json media type string in HTTP’s accept header. The HAL client uses the link identifier as the URL to see if there is an associated Action definition for this link.

Tip

The prs in the media type string is part of the IANA registration standard for media types covered in RFC6838. It indicates a “personal” registration. As this book goes to press, I’ve applied for the registration but it has not yet been completed.

Here’s how it works:

If the client app sees this link in the HAL response:

"http://localhost:8181/files/hal-task-create-form": {
  "href": "http://localhost:8181/task/",
  "title": "Add Task",
  "templated": false
},

When someone activates (clicks) the Add Task link in the app, the client makes an HTTP request that looks like this:

GET /files/hal-task-create-form HTTP/1.1
accept: application/prs.hal-forms+json
...

If it exists, the server responds with a HAL-FORM like this:

HTTP/1.1 200 OK
content-type: application/prs.hal-forms+json
....
{
  ... HAL-FORMS document here
}

Once the document is loaded by the client, it can be used to build and render an HTML FORM for user input. Some sample implementation details are in the next section.

The Implementation

Adding support for the HAL-FORMS extension at runtime is pretty simple. I needed to make some adjustments to the low-level HTTP calls to make them aware of HAL-FORMS responses. I also needed to modify the way the client responds to the initial link clicks (halLink), and to update the way the halShowForm routine worked to make sure it included information from the new HAL-FORMS document.

Here is an excerpt from the client library that shows the halLink routine and related code:

// handle GET for links
function halLink(e) {
  var elm, form, href, accept, fset;

  elm = e.target;
  accept = elm.getAttribute("type");

  // build stateless block 1
  fset = {};
  fset.rel = elm.rel;
  fset.href = elm.href;
  fset.title = elm.title;
  fset.accept = elm.accept;
  fset.func = halFormResponse;

  // execute check for a form
  formLookUp(fset); 2

  return false;
}

function formLookUp(fset) { 3
  req(fset.rel, "get", null, null, "application/prs.hal-forms+json", fset);
}

function halFormResponse(form, fset) {
  if(form && form!==null && !form.error && fset.status<400)  {
    // valid form resonse? show it
    halShowForm(form, fset.href, fset.title); 4
  }
  else {
    // must be a simple HAL response, then
    req(fset.href, "get", null, null, fset.accept||g.ctype, null); 5
  }
}

The highlights are:

1

Build up a shared block of properties to send to the low-level HTTP caller.

2

Use that information to make a request for a HAL-FORMS document.

3

This is the line that makes the HAL-FORMS request (not the media type string).

4

If a valid HAL-FORMS document was returned, pass it to the halShowForm routine.

5

Otherwise, just perform a simple HTTP GET on the original URL that initiated the click.

There is more to making the mod work and you can check out the associated repo for details.

So, with this new extension, I’ve moved the Action definitions to a standalone set of files that can be updated by the API service in a way that our HAL client application can understand and use safely without the need to recode and redeploy the client code. That means when the service API adds new functionality—for example, MarkTaskActive or any other new interaction—the service can just offer up a new HAL-FORMS document and the existing client will be able to handle the details (e.g., URL, HTTP method, arguments, etc.).

Note

Benjamin Greenberg, Senior Software Engineer at Comcast, gave a talk on how they created their own custom HAL-FORMS implementation (called _forms). I wasn’t able to find a written specification for it as I was writing this chapter, but I added a pointer to a video of his presentation to the References section of this chapter.

Summary

OK, we covered quite a bit in this chapter. Let’s do a quick summary:

Changing URLs is safe

Thanks to HAL’s design, we no longer need to store many URLs in the code. I was able to get by with storing only one (the starting URL), but you could even have users supply that at runtime. Now the TPS API can modify URLs any time it wants without breaking the client application (as long as that first URL is still honored).

Objects and Actions are still missing

While the HAL responses have lots of available information about Addresses, it supplies almost nothing about Objects and Actions. We needed to handle that ourselves. We included a setContext routine in the client code to look for Objects we already expected (Home, Task, and User). We also used the halForms routine and custom Action definitions baked into the code to handle the query and write operations. Adding this information into our app makes it tightly bound to both the service Object model and the predefined Action elements from the human documentation.

The HAL-FORMS extension solves the Actions challenge

I was able to create a custom extension to the HAL spec that allowed me to store all my Actions in a separate document and load them at runtime. This means my HAL client doesn’t need to know about all the Actions ahead of time and that the TPS API can add new Actions in the future without breaking my app. The bad news is that this is just a convention I invented. It’s not going to be available from other HAL servers and even my extension will likely be ignored by any other HAL client that accesses the TPS service.

An update on HAL clients and the OAA Challenge

HAL clients do OK in our OAA Challenge. They are built to handle changes to Addresses (URLs), but they need to be recoded and redeployed any time an Object or Action is added, removed, or changed. We can improve our OAA Challenge standing by implementing our custom HAL-FORMS extension, but that is not likely to work for any other HAL services or clients.

So, we are certainly better off than we were when relying on just JSON clients—those that only receive custom JSON object graph responses. But we can do better. As we’ll see later in the book, there are a couple of more media type designs to review, and both of them do better at the OAA Challenge.

References

  1. When this book was released, the most reliable documentation on the Hypertext Application Language (HAL) was hosted on Mike Kelly’s web server.

  2. There is an IETF specification document started for JSON HAL, but it had expired by the time this book was written. It may be revived by the time you read it, though.

  3. The IANA Registration for HAL can be found at IANA.

  4. Mike Kelly agreed to an interview with me in 2014 for InfoQ Magazine.

  5. Mike Kelly’s 2013 blog post—“The Case for Hyperlinks in APIs”—can be found at The Stateless Blog.

  6. The URI Template IETF specification (RFC6570) defines patterns for variable expansion for URLs.

  7. The W3C released the CURIES 1.0 Working Group Note in 2010.

  8. The Web Linking spec RFC5988 “specifies relation types for web links, and defines a registry for them.”

  9. The standards for registering a media type with the IETF are written up in RFC6836, RFC4289, and RFC6657.

  10. Benjamin Greenberg, Senior Software Engineer at Comcast, did a presentation on a forms extension for HAL. You can watch a video of his talk on YouTube.

Image Credits

Get RESTful Web Clients now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial