Web Site Measurement Hacks

(Don’t) Benchmark Your Site

Except in very controlled situations, attempting to compare your conversion rate information to published data almost always creates more problems than it solves and should be avoided.

Many business people seem to have an intense desire to benchmark themselves against market leaders, even if they aren’t selling to the same market. Especially for a relatively complex key performance indicator like conversion rate, one dependent on so many variables (for example, marketing strategy and spend, product assortment and pricing, usability of site, or audience), making any comparison except in the most controlled situations is an exercise in futility. I’ll explain two situations where you can compare your data to that gathered from another site in a minute, but first here are a few reasons to avoid benchmarking yourself.

First, unless you know exactly where the data is coming from, you might be comparing apples to bowling balls. Given the differences in accuracy observed when comparing different data sources, are you sure you want to make business decisions if you’re not sure about the accuracy of someone else’s data? If my key comparison is “page views per visit,” but I have no idea whether a competitor is rigorous in excluding robot and spider activity [Hack #23] from their data set or whether they use accepted standards to define a visit [Hack #1] , how can I be sure their number is any good? The easy answer is that you can’t, so comparisons may be misleading. ...