Using WebPageTest

Chapter 1. How Fast Is My Page?

The first question to pop into the minds of most people tasked with optimizing the performance of a web page is probably “How fast is it?” As in the story about the factory, understanding the current state of a system is an important first step on the road to optimization. Determining the current speed of a page helps to dictate the severity of the performance problem and sets a baseline from which to improve.

Before diving into WebPageTest for the purpose of getting the one golden number that represents the true speed of your page, take a step back and consider two cautionary points. First, the golden number you seek may not be the metric provided by the tool. If you want to know the speed of a page, you should define exactly what you’re trying to measure. Second, even if the tool did report on the metric you care about, it is not necessarily representative of the page’s true speed. The true speed is that which the real users of the page experience. Real users live all over the world, use different technologies like device type or browser, and connect to the Internet differently. The true speed of this amalgamation is extremely hard to reflect in a single test.

Measure What Matters

WebPageTest measures the speed of a web page based on the amount of time that has elapsed from the initial page request until the browser fires the load event, sometimes referred to as the document complete time. This is the time at which the Document Object Model (DOM) has been created and all images have been downloaded and displayed. For most traditional web pages, the load time is a suitable metric for representing how long a user must wait until the page becomes usable.

One misconception about WebPageTest is that the default metric, load time, is always applicable to the page being tested. Not all web pages are created equal, however, and this metric may not accurately represent the true amount of time users wait.

For example, think about the last time you read a news article online. As the page loaded, what were you waiting for? The most probable answer is that you were waiting for the text content of the article itself. The DOMContentLoaded event, also reported by WebPageTest, is like the load event except that it doesn’t wait for images to be displayed. The timing of this event may be a more appropriate metric to track because the time to load the ancillary images should not necessarily be taken into account. The default metric is not always the most relevant to the page in test.

For one final example, consider the YouTube video player page. This is not a traditional web page and its content is more elaborate than just text and images. Sometimes, as in this case, custom application-specific metrics are needed to represent the true load time. Users of this page are waiting for the video to play, as opposed to the text appearing or images loading. The page itself needs to tell the browser when the wait is actually over, because the built-in events aren’t designed for this particular use case of reporting when the video starts playing.

Application-Specific Metrics

You can log custom application-specific metrics to WebPageTest with the User Timing API. Using the YouTube example, when the video starts to play, that moment in time can be marked with a line of JavaScript:

performance.mark('playback-start');

WebPageTest will capture these marks and make them available in test results.

Synthetic Versus RUM

Web performance tools tend to be divided based on which big question they answer: “How fast is it?” or “How can it be made faster?” The two classifications of tools are commonly referred to as synthetic and real-user monitoring (RUM). WebPageTest falls under the synthetic category.

There’s a saying that when you have a hammer, all of your problems start to look like nails. Similarly, no one type of web performance tool can answer all of your questions. It’s important to distinguish what each type does and how it can be used so that you know when to use the right tool for the job:

Synthetic	RUM
Laboratory-like testing	Measures performance of real users
Low variability, controlled	High variability, unrestricted
Ad hoc tests	Continuous data collection

Tools like WebPageTest are considered to be synthetic because of their artificial testing environments. Akin to a clean room in a laboratory, WebPageTest gives its testers granular control over many of the variables that contribute to performance changes, such as geographic location and type of network connection. By making these variables constant, the root causes of poor frontend performance can be more easily identified and measured.

Unlike the control afforded by synthetic testing, RUM does what its name implies and measures the actual performance real users are experiencing in the wild. The unbridled variations in browsers and bandwidth are all accounted for in the reporting so that each and every user’s unique environment is represented. By looking at the raw data, you can draw definitive statistical conclusions. For instance, with access to the performance results, you are able to determine the page-load time for any given percentile. RUM is also considered to be monitoring because data tends to be continuously recorded and streamed to a dashboard. By monitoring performance, developers are able to get instant notification when page speed takes an unexpected turn; a decline in speed could theoretically page an engineer immediately if necessary. This is especially useful for mission-critical applications for which performance is just as important as availability.

For attempting to determine the overall speed of a page, it’s clear that RUM is the appropriate solution because it accurately represents the performance of actual users. When starting out with WebPageTest, one pitfall is to assume that the synthetic results are like real-user metrics. The reality, however, is that synthetic tools are deliberately designed to focus on the performance of a web page under strict conditions that are otherwise highly volatile in real-user performance.

To help illustrate this pitfall, imagine that you run a synthetic test of your home page and come to find that the load time is 10 seconds. “That’s crazy,” you think, because it never feels that slow to you. Your real-world experience does not coincide with the test results. It’s not that the test is necessarily wrong. The test configuration is meant to represent one particular use case. If it isn’t set up to match your browser, in your city, over your connection speed, you’re unlikely to get comparable results. The test is only an artificial representation of what someone under similar conditions might experience. It’s up to you to configure the test in a way that mimics the conditions that you want to compare. Throughout the book, we’ll look at different use cases that each has its own unique conditions, which in turn have corresponding configurations.