The web browser provides JavaScript users a wonderful environment for building applications that run in the browser. Using ExtJS or jQuery, it is possible to build an app that for many users can rival what can be done in a desktop application, and provide a method of distribution that is about as simple as it gets. But however nice the browser has been in terms of providing a user experience, it has fallen flat when it comes to data storage.
Historically, browsers did not have any way to store data. They were,
in effect, the ultimate thin client. The closest that could happen was the
HTTP cookie mechanism, which allows a piece of data to be attached to each
HTTP request. However, cookies suffer from several problems. First, each
cookie is sent back and forth with every request. So the browser sends the cookie for
each JavaScript file, image, Ajax request, and so on. This can add a lot of
bandwidth use for no good reason. Second, the cookie specification tried to
make it so that a cookie could be shared among different subdomains. If a company had app.test.com and images.test.com, a cookie could be set to be
visible to both. The problem with this is that outside of the United States,
three-part domain names become common. For example, it would be possible to
set a cookie for all the hosts in .co.il
that would allow a cookie to leak to almost every host in Israel. And it is
not possible to simply require a three-part domain name whenever the name
contains a country suffix, because some countries such as Canada do not
follow the same convention.
Having local storage on the browser can be a major advantage in terms of speed. A normal Ajax query can take anywhere from half a second to several seconds to execute, depending on the server. However, even in the best possible case it can be quite slow. A simple ICMP ping between my office in Tel Aviv and a server in California will take an average of about 250 ms. Of that 250 ms, a large part is probably due to basic physical limitations: data can travel down the wire at just some fraction of the speed of light. So there is very little that can be done to make that go faster, as long as the data has to travel between browser and server.
Local storage options are a very good option for data that is static or mostly static. For example, many
applications have a list of countries as part of their data. Even if the
list includes some extra information, such as whether a product is being
offered in each country, the list will not change very often. In this case,
it often works to have the data preloaded into a
localStorage object, and then do a conditional reload
when necessary so that the user will get any fresh data, but not have to
wait for the current data.
Local storage is, of course, also essential for working with a web application that may be offline. Although Internet access may seem to be everywhere these days, it should not be regarded as universal, even on smartphones. Users with devices such as the iPod touch will have access to the Internet only where there is WiFi, and even smartphones like the iPhone or Android will have dead zones where there is no access.
With the development of HTML5, a serious movement has grown to provide the browser with a way to create persistent local storage, but the results of this movement have yet to gel. There are currently at least three different proposals for how to store data on the client.
In 2007, as part of Gears, Google introduced a browser-based SQLite database. WebKit-based browsers, including Chrome, Safari, and the browsers on the iPhone and Android phones, have implemented a version of the Gears SQLite database. However, SQLite was dropped from the HTML5 proposal because it is a single-sourced component.
The localStorage mechanism provides
a JavaScript object that persists across web reloads. This mechanism seems to be reasonably
well agreed on and stable. It is good for storing small-sized data such as
session information or user preferences.
This current chapter explains how to use current localStorage implementations. In Chapter 5, we’ll look at a more complicated and
sophisticated form of local storage that has appeared on some browsers:
IndexedDB.
Modern browsers provide two objects to the programmer for
storage, localStorage and sessionStorage. Each can hold data as keys and
values. They have the same interface and work in the same way, with one
exception. The localStorage object is
persistent across browser restarts, while the sessionStorage object resets itself when a
browser session restarts. This can be when the browser closes or a window
closes. Exactly when this happens will depend on the specifics of the
browser.
Setting and getting these objects is pretty simple, as shown in Example 4-1.
Example 4-1. Accessing localStorage
//set
localStorage.sessionID = sessionId;
localStorage.setItem('sessionID', sessionId);
//get
var sessionId;
sessionId = localStorage.sessionID;
sessionId = localStorage.getItem('sessionId');
localStorage.sessionId = undefined;
localStorage.removeItem('sessionId');Browser storage, like cookies, implements a “same origin” policy, so different websites can’t interfere with one another or read one another’s data. But both of the storage objects in this section are stored on the user’s disk (as cookies are), so a sophisticated user can find a way to edit the data. Chrome’s Developer Tools allow a programmer to edit the storage object, and you can edit it in Firefox via Firebug or some other tool. So, while other sites can’t sneak data into the storage objects, these objects still should not be trusted.
Cookies are burdened with certain restrictions: they are
limited to about 4 KB in size and must be transmitted to the server with
every Ajax request, greatly increasing network traffic. The browser’s localStorage is much more generous. The HTML5
specification does not list an exact size limit for its size, but most
browsers seem to limit it to about 5 MB per web host. The programmer
should not assume a very large storage area.
Data can be stored in a storage object with direct object access or
with a set of access functions. The session object can store only strings,
so any object stored will be typecast to a string. This means an object
will be stored as [object Object],
which is probably not what you want. To store an object or array, convert
it to JSON first.
Whenever a value in a storage object is changed, it fires a storage event. This event will show the key, its old value, and its new value. A typical data structure is shown in Example 4-2. Unlike some events, such as clicks, storage events cannot be prevented. There is no way for the application to tell the browser to not make a change. The event simply informs the application of the change after the fact.
Example 4-2. Storage event interface
var storageEvent = {
key: 'key',
oldValue: 'old',
newValue: 'newValue',
url: 'url',
storageArea: storage // the storage area that changed
};WebKit provides a screen in its Developer Tools where a
programmer can view and edit the localStorage and sessionStorage objects (see Figure 4-1). From the Developer Tools, click on
the Storage tab. This will show the localStorage and sessionStorage objects for a page. The Storage
screen is also fully editable: keys can be added, deleted, and edited
there.
Although Firebug does not provide an interface to the localStorage and sessionStorage objects as Chrome and other
WebKit-based browsers do, the objects can be accessed via the JavaScript console, and you
can add, edit, and delete keys there. Given time, I expect someone will
write a Firebug extension to do this.
Of course, it is possible to write a customized interface to view
and edit the storage objects on any browser. Create a widget on-screen
that exposes the objects using the getItem and removeItem calls shown in Example 4-1 and allow editing
through text boxes. The skeleton of a widget is shown in Example 4-3.
Example 4-3. Storage Viewer
(function createLocalStorageViewer()
{
$('<table></table>').attr(
{
"id": "LocalStorageViewer",
"class": 'hidden viewer'
}).appendTo('body');
localStorage.on('update', viewer.load);
var viewer =
{
load: function loadData()
{
var data, buffer;
var renderLine = function (line)
{
return "<tr key='{key}' value='{value}'>\n".populate(line) +
"<td class='remove'>Remove Key</td>" +
"<td class='storage-key'>{key}</td><td>{value}</td></tr>".populate(line);
};
buffer = Object.keys(localStorage).map(function (key)
var rec =
{
key: key,
value: localStorage[data]
};
return rec;
});
};
$("#LocalStorageViewer").html(buffer.map(renderLine).join(''));
$("#LocalStorageViewer tr.remove").click(function ()
{
var key = $(this).parent('tr').attr('key').remove();
localStorage[key] = undefined;
});
$("#LocalStroageViewer tr").dblclick(function ()
{
var key = $(this).attr('key');
var value = $(this).attr('value');
var newValue = prompt("Change Value of " + key + "?", value);
if (newValue !== null)
{
localStorage[key] = newValue;
}
});
};
}());ExtJS, some examples of which appeared in earlier
chapters, is a very popular JavaScript framework allowing very
sophisticated interactive displays. This section shows how to use
localStorage with ExtJS.
One nice feature of ExtJS is that many of its objects can remember their state. For example, the ExtJS grid object allows the user to resize, hide and show, and reorder columns, and these changes are remembered and redisplayed when a user comes back to the application later. This allows each user to customize the way the elements of an application work.
ExtJS provides an object to save state, but uses cookies to store the data. A complex application can
create enough state to exceed the size limits of cookies. An application
with a few dozen grids can overflow the size of a cookie, which can lock
up the application. So it would be
much nicer to use localStorage,
taking advantage of its much larger size and avoiding the overhead of
sending the data to the server on every request.
Setting up a custom state provider object is, in fact, pretty
easy. The provider shown in Example 4-4
extends the generic provider object and must provide three methods:
set, clear, and get. These methods simply read and write the
data into the store. In Example 4-4, I have
chosen to index the data in the store with the rather simple method of
using the string state_ with the
state ID of the element being saved. This is a reasonable method.
Example 4-4. ExtJS local state provider
Ext.ux.LocalProvider = function() {
Ext.ux.LocalProvider.superclass.constructor.call(this);
};
Ext.extend(Ext.ux.LocalProvider, Ext.state.Provider, {
//************************************************************
set: function(name, value) {
if (typeof value == "undefined" || value === null) {
localStorage['state_' + name] = undefined;
return;
}
else {
localStorage['state_' + name] = this.encodeValue(value);
}
},
//************************************************************
// private
clear: function(name) {
localStorage['state_' + name] = undefined;
},
//************************************************************
get: function(name, defaultValue) {
return Ext.value(this.decodeValue(localStorage['state_' + name]), defaultValue);
}
});
// set up the state handler
Ext.onReady(function setupState() {
var provider = new Ext.ux.LocalProvider();
Ext.state.Manager.setProvider(provider);
});It would also be possible to have all the state data in one large object and to store it into one key in the store. This has the advantage of not creating a large number of elements in the store, but makes the code more complex. In addition, if two windows try to update the store, one could clobber the changes made by the other. The local storage solution in this chapter offers no great solution to the issue of race conditions. In places where it can be a problem, it is probably better to use IndexedDB or some other solution.
When some of the persistent data used in an application
will be relatively static, it can make sense to load it to local storage
for faster access. In this case, the Ext.data.JsonStore object will need to be
modified so that its load() method will look
for the data in the localStorage area
before attempting to load the data from the server. After loading the data from localStorage, Ext.data.JsonStore should call the server to
check whether the data has changed. By doing this, the application can
make the data available to the user right away at the cost of possibly
short-term inconsistency. This can make a user interface feel faster to
the user and reduce the amount of bandwidth that the application
uses.
For most requests, the data will not have changed, so using some
form of ETag for the data makes a great deal of sense. The data is
requested from the server with an HTTP GET request
and an If-None-Match header. If the
server determines that the data has not changed, it can send back a
304 Not Modified response. If the
data has changed, the server sends back the new data, and the
application loads it into both the Ext.data.JsonStore
object and the sessionStorage
object.
The Ext.data.PreloadStore
object (see Example 4-6) stores data
into the session cache as one large JSON object (see Example 4-5). It further wraps the data that the
server sends back in a JSON envelope, which allows it to store some
metadata with it. In this case, the ETag data is stored as well as the date when the data is
loaded.
Example 4-5. Ext.data.PreloadStore offline data format
{
"etag": "25f9e794323b453885f5181f1b624d0b",
"loadDate": "26-jan-2011",
"data": {
"root": [{
"code": "us",
"name": "United States"
},
{
"code": "ca",
"name": "Canada"
}]
}
}Note
When building an ETag, make sure to use a good hash function. MD5 is probably the best choice. SHA1 can also be used, but since it produces a much longer string it is probably not worthwhile. In theory, it is possible to get an MD5 collision, but in practice it is probably not something to worry about for cache control.
Data in the localStorage object
can be changed in the background. As I already explained, the user can change the data
from the Chrome Developer Tools or from the Firebug command line. Or it
can just happen unexpectedly because the user has two browsers open to
the same application. So it is important for the store to listen for an
update event from the localStorage
object.
Most of the work is done in the beforeload event
handler. This handler checks the data store for a
cached copy of the data, and if it is there, it loads it into the store.
If there
is data present, the handler
will reload the data as well, but will use the Function.defer()
method to delay the load until a time when the system has hopefully
finished loading the web page so that doing the load will be less likely
to interfere with the user.
The store.doConditionalLoad() method makes an
Ajax call to the server to load the data. It includes the
If-None-Match header so that, if the
data has not changed, it will load the current data. It also includes a
force option that will cause the
beforeload handler to actually load
new data and not try to refresh the store from the localStorage cached version of the
object.
I generally define constants for SECOND, MINUTE, and HOUR simply to make the code more
readable.
Example 4-6. Ext.data.PreloadStore
Ext.extend(Ext.data.PreloadStore, Exta.data.JsonStore, {
indexKey: '',
//default index key
loadDefer: Time.MINUTE,
// how long to defer loading the data
listeners: {
load: function load(store, records, options)
{
var etag = this.reader.etag;
var jsonData = this.reader.jsonData;
var data =
{
etag: etag,
date: new date(),
data: jsonData
};
sessionStorage[store.indexKey] = Ext.encode(data);
},
beforeload: function beforeLoad(store, options)
{
var data = sessionStorage[store.indexKey];
if (data === undefined || options.force)
{
return true; // Cache miss, load from server
}
var raw = Ext.decode(data);
store.loadData(raw.data);
// Defer reloading the data until later
store.doConditionalLoad.defer(store.loadDefer, store, [raw.etag]);
return false;
}
},
doConditionalLoad: function doConditionalLoad(etag)
{
this.proxy.headers["If-None-Match"] = etag;
this.load(
{
force: true
});
},
forceLoad: function ()
{
// Pass in a bogus ETag to force a load
this.doConditionalLoad('');
}
});In the event that an application may be used offline, or with a flaky connection to the Internet,
it can be nice to provide the user a way to save her changes without
actually needing the network to be present. To do this, write the
changes in each record to a queue in the localStorage object.
When the browser is online again, the queue can be pushed to the server.
This is similar in intent to a transaction log as used in a
database.
A save queue could look like Example 4-7. Each record in the queue represents an action to take on the server. The exact format will of course be determined by the needs of a specific application.
Example 4-7. Save queue data
[
{
"url": "/index.php",
"params": {}
},
{
"url": "/index.php",
"params": {}
},
{
"url": "/index.php",
"params": {}
}
]Once the web browser is back online, it will be necessary to process the items in the queue. Example 4-8 takes the queue and sends the first element to the server. If that request is a success, it will take the next element and continue walking down the queue until the entire queue has been sent. Even if the queue is long, this process will execute it with minimal effect on the user because Ajax processes each item in an asynchronous manner. To reduce the number of Ajax calls, it would also be possible to change this code to send the queue items in groups of, say, five at a time.
If the uncertainty of all the client-side storage options is enough to drive you crazy, you have other options. As with many things in JavaScript, a bad and inconsistent interface can be covered up with a module that provides a much better interface. Here are two such modules that can make life easier.
DSt (http://github.com/gamache/DSt)
is a simple library that wraps the localStorage object. DSt can be a freestanding
library or work as a jQuery plug-in. It will automatically convert any complex object
to a JSON structure.
DSt can also save and restore the state of a form element or an
entire form. To save and restore an element, pass the element or its ID
to the DSt.store() method. To
restore it later, pass the element to the DSt.recall()
method.
To store the state of an entire form, use the DSt.store_form()
method. It takes the ID or element of the form itself. The data can be
restored with the DSt.populate_form()
method. Example 4-9 shows the basic use of DSt.
If you don’t want to venture to figure out which storage
engines are supported on which browsers and create different code for
each case, there is a good solution: the jStore plug-in for
jQuery. This supports localStorage,
sessionStorage, Gears SQLite, and
HTML5 SQLite, as well as Flash Storage and IE 7’s proprietary
solution.
The jStore plug-in has a simple interface that allows the programmer to store name/value pairs in any of its various storage engines. It provides one set of interfaces for all the engines, so the program can degrade gracefully when needed in situations where a storage engine doesn’t exist on a given browser.
The jStore plug-in provides a list of engines that are available in the jQuery.jStore.Availability instance variable.
The program should select the engine that makes the most sense. For
applications that require multibrowser support, this can be a useful
addition. See the jStore web page for more details.
Get Programming HTML5 Applications now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
