Chapter 19. Testing
THANKS TO THE AGILE MOVEMENT, TESTING IS NO LONGER SEEN AS THE BIT AT THE END WHERE QA gives its stamp of approval. It’s a core development technique, and if you’re creating complex web apps, it’s worth testing within and across all tiers.
Because of remoting, Ajax browser scripts usually rely on web services. But it’s not much fun working on both sides at the same time, so Simulation Service explains how to create a “dummy” or “stub” web service to call from the browser. You can use it for tests or for prototyping. Browser-Side Test is simply a test of the JavaScript code sitting in the browser. The hardest thing is to exercise it in an automated fashion; the pattern also looks at that issue. Service Test involves no JavaScript, because it actually takes the place of the browser; an HTTP client is used to test the web service. System Test effectively simulates a user to exercise the application across all layers.
Simulation Service
○ Canned, Double, Dummy, Fake, Masquerade, Mock, Service, Shunt, Sim, Simulation, Stub
Developer Story
Dave is beginning to work on a new email application. Eventually, the server will need to expose a mail API, so the browser can query for recent messages and upload new ones. However, that will require email expertise, and Dave wants to focus on the user interface in the browser for now. Thus, he hardcodes the responses he’d like in a “fake” server-side service. This allows him to proceed with user-interface development while his colleague takes care of the real server implementation.
Problem
How can you develop the browser script in parallel with the services it uses?
Solution
Develop the browser application against “fake” web services that simulate the actual services used in production. A Simulation Service is typically a hardcoded response that ignores any input parameters. So instead of:
$taxRate = calculateTaxRate($userId, $year); print $taxRate;
a Simulation Service would just say:
print "0.1235";
It’s also possible to do all this in a browser. If you use a library to perform Web Remoting (Chapter 6), you could make the implementation produce a hardcoded response, and ignore the server altogether.
In some cases, you might make the service more flexible; e.g., it can expose an operation to let the browser tell it what to output later on.
Simulation Service decouples the server and browser components of the application. Once the interface is defined, each can be developed in parallel. Moreover, this pattern paves the way for an agile process for building the application, based loosely on concepts such as Test-Driven Design (TDD) and lean manufacturing.
Conventional web development is usually a bottom-up approach: first comes the database, then the data layer to access it, then the business components on top of that, and finally the user interface. This sequence might seem like the most logical thing to do, given the dependency relationships involved. But the problem is that the user interface is really the whole point of the application, so there’s a good argument for starting there instead. For a given feature, flesh out the user interface and use that work to drive the service interface. So instead of designing the server side first and forcing the browser script to work with whatever interface arises, let the needs of the browser script drive the interface design. Consequently, the service will not only contain the precise functionality required by the browser, but also expose it with the most suitable interface.
Starting development in the browser sounds like a good idea until you realize the browser depends on the service, which doesn’t yet exist! That’s where a Simulation Service comes in: it lets you define what the service offers, not how the service achieves it. As you develop the browser script, you’ll continue to come across refinements to the service—an extra operation here, a different response format there. In each case, changing the Simulation Service is usually as simple as editing the hardcoded response. Once you’re confident with the requirements for the Service, you—or someone else—can go build the real thing.
Another application is in Browser-Side Tests (see later in this chapter). When testing a browser component, it’s important to test only the component itself, and not the services it depends on. Imagine you’re testing a browser component that renders news arriving from an RSS web service. An automated test could then interrogate the DOM and run a bunch of assertions to ensure it was formatted correctly. But if the component uses the real service, the resulting DOM will reflect the news of the hour. A human could manually verify it, but automation wouldn’t be possible. A hardcoded response would eliminate that problem, allowing the assertions to be consistent with the simulated RSS. Furthermore, there’s no dependence on the real implementation—no matter how complex the actual search algorithm, the rendering component will always be tested on its own merit.
Decisions
How will you switch between different services?
It’s easy enough to create a Simulation Service, but how do you switch between the simulation and the real thing? And, how do you switch between different simulation services, since you might prefer one or the other for different tasks. Right now, there’s no easy answer to that because there’s no real support in the way of tools and libraries. Where the practice of Simulation Service is followed, most developers probably switch over to a real service in the most brute-force way possible: they edit the code. But doing so means there’s a manual step involved in order to test or deploy. Here are a few ways to make the process more manageable:
Keep the JavaScript pointing at the same service URL, but have your server-side build tool (e.g.,
ant) decide what implementation resides there.Build some configuration into the JavaScript to help it point to the correct service. This would be useful for an automated JavaScript-based test. You could even create a simple browser control to let the user choose which service is executed, which is helpful during development.
Instead of hosting JavaScript in a static file, dynamically generate it from the server, so that variable URLs can be set on the fly.
Code Example: Simulation Service
The Ajax Pattern Reader (http://ajaxify.com/run/reader/) runs against a couple of Cross-Domain Proxy (Chapter 10) services. They fetch information from the AjaxPatterns.org web site and return them in a format easily parsed by the browser. For development purposes, the Basic version (http://ajaxify.com/run/reader/) uses Simulation Service.
The first service produces a Plain-Text Message (Chapter 9) containing the patterns in a pipe-separated list. A hand-created simulation (http://ajaxify.com/run/reader/patternList.phtml) is used instead of fetching the page from AjaxPatterns.org, parsing the HTML, and reformatting it.
Periodic Refresh|||One-Second Spotlight|||Slider
For comparison, the real service outputs a list that looks similar, but is much longer, and, of course, not hardcoded (http://ajaxify.com/run/reader/logging/realService/patternList.phtml).
Ajax Stub|||Browser-Side Cache|||Browser-Side Templating||| ... |||XML
Message|||XMLHttpRequest CallSimilarly, there’s a simulation of the summary service (http://ajaxify.com/run/reader/patternSummary.phtml?patternName=Slider), whose job is to produce an HTML Message summarizing a given pattern. The simulation includes some rudimentary logic to make the response vary according to the input:
<? $patternName = urlDecode($_GET["patternName"]); ?>
<h1><?= $patternName ?></h1>
<? if ($patternName=="Periodic Refresh") { ?>
With periodic refresh, it's all about the timer.
<? } else if ($patternName=="Slider") { ?>
<p>The slider slides from left to right.</p>
...
<? } else {
print ("Text goes here.");
}
?>The output from the real service is in the same format (http://ajaxify.com/run/reader/logging/realService/patternSummary.phtml?patternName=Slider), but as with the list service, it’s fetched from AjaxPatterns.org in real-time.
Related Patterns
Want to Know More?
In Lean IT Processes terminology, the Simulation Service pattern facilitates a “Pull System,” where design is driven by customer demands rather than technical capability (http://abc.truemesh.com/archives/000093.html).
Categorization of Test Doubles, distinguishing between mocks, stubs, and so on. Under that categorization, the pattern here describes a “Fake” (http://tap.testautomationpatterns.com:8080/Using%20Test%20Doubles.html).
Browser-Side Test
⊙ Automated, IntegrationTest, JavaScript, Regression, UnitTest, TDD, Test
Developer Story
Devi is about to embark on upgrading a blog reader and realizes she will be tinkering with the JavaScript RSS parser. The parser will need some refactoring, but fortunately there’s already a JavaScript unit test for it. She proceeds to refactor the parser, ensuring the test still passes, and continues to enhance the test as she builds in further functionality.
Problem
How can you test JavaScript components?
Solution
Create automated tests of browser-side JavaScript components. This pattern is a mapping from automated testing in other languages, where frameworks like the xUnit series (http://en.wikipedia.org/wiki/XUnit—e.g., JUnit; see http://junit.org—are now popular. Several frameworks do the same for JavaScript. As with other languages, you can unit test an individual JavaScript component (e.g., a function) you’ve created, and also perform integration tests and system tests that work across several objects. And also like other languages, you can accumulate tests into a test suite object, so it’s possible to run a full regression test.
The test is usually some JavaScript that performs some task and includes assertion to check the state is as expected. Usually, JavaScript frameworks include a Test Runner that will run a given set of tests, then morph the DOM to show results in a table. You usually build an HTML page, with test code embedded, and perhaps some inputs to control the test; e.g., Start and Stop buttons.
So the test is automated insofar as the programmer’s not manually interacting with the application and verifying results. That’s good enough while developing, but the demands of continuous integration suggest we need to go beyond that. How do you allow tools like Make, Ant, or Rake start and stop a JavaScript test?
One way to coordinate a Browser-Side Test is to simulate browser behavior. Using a JavaScript engine, you can programmatically execute some JavaScript code. For example, Java programmers have the Rhino library (a standard component of Java 6). If there’s some DOM manipulation involved as well, an approach like Browser Simulation, discussed in System Test can be used. A second approach is to fire up a real browser using a shell command, specifying the test page as an argument. The test framework will then run in the browser, and the page must be configured to upload results to a service somewhere, which will expose them to the test process. The test process, upon detecting new results, shuts down the browser with an interprocess communication protocol or, failing that, kills the process.
Tool Support
Scriptaculous testing framework
The Scriptaculous framework (http://script.aculo.us)
includes a unit-testing framework. It works in a similar manner as
JsUnit frameworks (discussed next) and the nUnit frameworks they are modelled on. A Test Case is an object
containing a sequence of methods beginning with “test”; e.g.,
testStringMatches( ), and
optional setup( ) and teardown( ). The test is passed to a
testrunner object. For each
test method, the runner executes setup(
), then the test method, and then teardown( ). A library of assertions are
available to verify state throughout the tests. As the tests
proceed, the runner updates the UI to report on the
results.
As with many Scriptaculous classes, the Runner object accepts an options argument. A particularly
interesting option is resultsURL, which the runner will post
results to. This is a facility that makes automated testing
possible, as discussed earlier in the "Solution.”
Scriptaculous also contains some functions that let you simulate mouse and keyboard; see System Test .
JsUnit framework (Hieatt)
Edward Hieatt’s JsUnit (http://www.edwardh.com/jsunit/) is a testing framework that supports standard xUnit-like functionality. Each test is actually an HTML page containing the test in a script, and the HTML page is passed as an argument to a test runner page. There’s a demo page available (http://www.edwardh.com/jsunit/runner/testRunner.html?testpage=www.edwardh.com/jsunit/runner/tests/jsUnitTestSuite.html).
JsUnit Framework (Schaible)
Jorg Schaible’s JsUnit (http://jsunit.berlios.de/index.html) works in a similar manner as the preceding frameworks.
Code Example: Using Scriptaculous Unit-Testing
The Basic Ajax Pattern Reader (http://ajaxify.com/run/reader/) uses a queue abstraction to represent the playlist. The queue itself was developed with a Scriptaculous test case (http://script.aculo.us; see Figure 19-3). The setup method establishes a new queue:
setup: function( ) { with(this) {
q = new Queue( );
}},
An initial test method verifies the queue’s initial state:
testEmptyQueue: function( ) { with(this) {
assertEqual(0, q.size);
var items = q.items( );
assertEqual(0, items.length);
}},Subsequent methods examine the queue as it’s built up:
testSequence: function( ) { with(this) {
q.admit("first");
q.admit("second");
assertEqual(2, q.size);
assert(util.membersEqual(["second","first"], q.items( )));
...
}},All of this is embedded an HTML file (http://ajaxify.com/run/reader/queueTest.html), which, if you visit it, will run the test and report the results. Here’s the structure of the HTML file:
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Queue Test</title>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<script src="/run/Lib/js/util.js" type="text/javascript"></script>
<script src="prototype.js" type="text/javascript"></script>
<script src="unittest.js" type="text/javascript"></script>
<script src="queue.js" type="text/javascript"></script>
<link rel="stylesheet" href="test.css" type="text/css" />
</head>
<body>
...
<!-- Log output -->
<div id="testlog"> </div>
<!-- Tests follow -->
<script type="text/javascript" language="javascript" charset="utf-8">
new Test.Unit.Runner({
q: null,
setup: function( ) { with(this) {
q = new Queue( );
}},
testEmptyQueue: function( ) { with(this) {
assertEqual(0, q.size);
var items = q.items( );
assertEqual(0, items.length);
}},
...
});
</script>
</body>
</html>Service Test
⊙ Automated, JavaScript, Remoting, UnitTest, TDD, Test, WebService, XMLHttpRequest
Developer Story
Dave is creating a trading application. On the server, there are services to give quotes and accept orders. To help develop and test the web service, he builds a suite of Service Tests. One test, for example, submits an order and checks that the database registered it. These tests are automated and re-executed every time the project changes.
Problem
How can you test web services?
Solution
Build up automated tests of web services, using HTTP clients to interact with the server as the browser normally would. The browser-server interaction of an Ajax App is usually a sequence of XMLHttpRequest Calls (Chapter 6). That’s fortunate, because testing the service is usually as easy as simulating the browser—passing a certain request and checking the response. Sometimes, verification might also involve checking what happened on the server, e.g., a database state change or a message to an external system.
The test itself can be built in any language and testing framework you like, because it’s completely decoupled from both the browser and the server. The only practical constraint is that you should be able to run it from a script, to allow for continuous integration. Many languages provide an HTTP client library, so it’s easy to build a test pass request to the server and make assertions against the response. Indeed, there are client frameworks like HTTPUnit (http://httpunit.sourceforge.net/) that are specifically for the purposes of testing web apps. They’re typically used for testing conventional applications, where the response contains an entire page, but they work fine for the terse responses usually provided by web services.
A RESTful Service (Chapter 9), in
particular, should be straightforward to test because it will
usually output structured responses which are easily parsed; and
there won’t be any conversational state to set up and track. A
service that outputs an HTML Message (Chapter 9) makes tests a bit more
fragile; passing in 5+5 will always let you assert 10 as the Plain-Text
Message (Chapter 9),
but not always The <span
class="strong">answer</class> is <span
class="surprising">10<span>!!! if an HTML Message
is used.
Tool Support
Ruby Net::HTTP
Net::HTTP (http://www.ruby-doc.org/stdlib/libdoc/net/http/rdoc/) is an HTTP client library for Ruby, shipping with the standard distribution. It’s used in the refactoring illustration below.
Jakarta HTTPClient
Jakarta HTTPClient (http://jakarta.apache.org/commons/httpclient/) is a richly featured HTTP client framework for Java.
PHP HTTPClient
Simon Willison’s HTTPClient (http://scripts.incutio.com/httpclient/) is a basic HTTP client framework for Perl.
Code Example: Service Test
The Basic Ajax Pattern Reader (http://ajaxify.com/run/reader/) uses some Simulation Services to get information from the server. A Service Test was first created against the Simulation Service, and once the Simulation Service passed, the test was used to drive the design of the real services. Earlier, the Solution mentioned that the programming language of the HTTP client is completely independent of the server or browser languages. That’s not just theoretical, since some languages (e.g., Ruby and Perl) are particularly well-suited to string manipulation. To prove the point, the example here uses Ruby’s Net::HTTP library.
The service being tested is patternSummary.phtml (http://ajaxify.com/run/reader/patternSummary.phtml),
a pattern text provider that accepts a pattern name and outputs its
solution. The test case is very basic—it just requests the
Slider pattern and checks that the output
matches a simple regular expression:
require "test/unit"
require "net/http"
class PatternServiceTest < Test::Unit::TestCase
def test_slider
sliderResponse = grabPattern("Slider");
regex = /^\s*<h1><a\ href=\".*?\">Slider<\/a>
<\/h1>\s*\
<span\ class=\"summary\">.+?<\/span>.+
/mx;
assert_match(regex, sliderResponse.body);
end
...
def grabPattern(patternName)
session = Net::HTTP.new('ajaxlocal', 80);
response = session.get("/run/reader/logging/realService/" +
"patternSummary.phtml?patternName=" + patternName);
assert_equal("200", response.code);
return response;
end
endSystem Test
⊙ Automated, FunctionalTest, JavaScript, PerformanceTest, Regression, Robot, UnitTest, TDD, Test
Developer Story
Devi has completed the first cut of a new Ajax App and wants to build an automated test of the whole thing. With the help of a testing application, she records a typical usage so it can be played back later. The software takes note of the final state, which will be used for verification during automated playback.
Solution
Build automated tests to simulate user behavior and verify the results. A System Test exercises a complete system in much the same way as a user would. The most common category of system test is a functional test—verifying the functionality—but system testing also encompass qualities such as performance and robustness. While they are often conducted manually, with users clicking and mousing around, it’s highly preferable to automate as much as possible in order to promote continuous integration.
Automated system tests remain somewhat difficult, but tools are improving. They can be broadly categorized as follows:
- Browser Controller (Figure 19-6)
A specialized, “robot"-style testing tool will fire up the browser, then use the underlying operating system’s API to create the effect of a user interacting with the browser. Several commercial controllers are able to record user actions for later playback.
- Embedded Browser Script (Figure 19-7)
Run the application in a standard browser and embed a Browser-Side Test on the page. The test can use some JavaScript trickery to simulate user activity. Don’t let the browser aspect put you off, because it’s feasible to have a continuous integration tool like Ant coordinate Browser-Side Tests, as pointed out earlier in this chapter.
- Browser Simulation (Figure 19-8)
The application is executed by a model of a real browser. Typically, the simulation pulls a page from a URL somewhere and builds up a DOM to represent it. The DOM can then be manipulated; e.g. the simulation allows a button click operation to occur or a form to be submitted. While several frameworks like this exist, the important issue for Ajax developers is how well they simulate JavaScript. JavaScript interpreters are implemented in several languages, so it’s possible for Browser Simulations to incorporate them in order to simulate a realistic Ajax interaction.
The first two approaches involve a real browser, which is a more realistic test, but it does imply that the test must run on a machine capable of launching the browser, or of somehow controlling it remotely.
System Tests require not just automation of user behavior, but also automation of result verification. As with other tests, results can be verified with a combination of assertions against the DOM and against server-side state.
Tool Support
GhostTrain
Scriptaculous’s GhostTrain (http://wiki.script.aculo.us/scriptaculous/show/GhostTrain)
is an Embedded Browser Script with recording
capability. When you start the recording mode, a Scriptaculous
test case begins to fill up inside the div. Each mouse and key action becomes a
new line of the test case. Later on, you can add your own
assertions about the application state. GhostTrain is a very
promising work in progress.
Watir
Watir (http://wtr.rubyforge.org) follows the Browser Controller approach, enabling a Ruby programmer to script browser behavior and interrogate the DOM at any stage. You can easily integrate Watir commands into a standard test case—for example:
ie = Watir::IE.start("http://ajaxpatterns.org")
ie.button(:name, "Search").clickWatir is IE-specific right now.
Selenium
Selenium (http://selenium.thoughtworks.com/) is an open source tool by Thoughtworks developers, an Embedded Browser Script framework that works in IE, Mozilla, Firefox, and Safari. A JavaScript Selenium component is embedded inside the browser application under test, and it communicates with a server-side component. Interesting, there’s flexibility on the location of the server-side component. It can be placed in the web server, to provide a quick summary in the web page being developed. Alternatively, it can be run as a separate server, where the server controls the browser (like a Browser Controller would), and the embedded browser component merely transmits server commands to the application.
HTTPUnit
HTTPUnit (http://httpunit.sourceforge.net/index.html)
is an open source Java testing framework, built on JUnit, and
specifically designed for testing web apps. It works as a
Browser Simulator, converting an HTML page
into a Java HTMLPage object
that wraps a Java DOM model of the original HTML. HTTPUnit
simulates JavaScript by delegating to the Rhino JavaScript engine,
although support is incomplete.
DojoToolkit
Dojo Toolkit (http://dojotoolkit.org/docs/compressor_system.html) is a Java-based Browser Simulator that uses Rhino to let you run tests from an Ant task. Given the project’s momentum and focus on Ajax, it’s likely to evolve into a very solid product.
Code Example: Using Scriptaculous GhostTrain
The Ajax Patterns Reader (http://ajaxify.com/run/reader/logging/realService/ghost/)
is refactored here to include Scriptaculous’s GhostTrain
(http://wiki.script.aculo.us/scriptaculous/show/GhostTrain),
discussed earlier. You can hit Escape anywhere in the document to open it
up, and then use the buttons to record a test case, which can then
be played back (Figure
19-9).
To hook up GhostTrain, the Scriptaculous library files must be included (quite a lot to download, but keep in mind they don’t need to be included in production):
<script type="text/javascript" src="/run/Lib/js/sc/prototype.js"></script> <script type="text/javascript" src="/run/Lib/js/sc/controls.js"></script> <script type="text/javascript" src="/run/Lib/js/sc/effects.js"></script> <script type="text/javascript" src="unittest.js"></script> <script type="text/javascript" src="ghosttrain.js"></script>
The only other requirement is to register the GhostTrain object as a keypress handler. The handler will take care of any other initialization:
window.onload = function( ) {
Event.observe(document, 'keypress', GhostTrain.intercept, true);
...
}Related Patterns
Want to Know More?
A Summary of Functional Testing Frameworks (http://www.softwareqatest.com/qatweb1.html#FUNC).
Get Ajax Design Patterns now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
