Search: How the Data Explosion Makes Us Smarter

CHAPTER 1

What Search Will See

There was a notion, when the Internet was first gaining consumer mindshare, that with it the infinite monkey theorem would be borne out much faster than had been previously thought. The theorem, developed by Èmile Borel, posited that a monkey randomly hitting keys on a typewriter would, over an infinite period of time, almost surely produce a work of Shakespeare. The capacity of the Internet was akin to multiplying the monkey by a million times; the underlying structure and massive scale of the web would enable anything to be created and described.

And, to an extent, that happened. The web was focused on text and images, and later, with YouTube, on video. By most accounts, the web in 1996 consisted of around 100,000 ...