Follow-up to "Quality of O'Reilly books"

I would like to thank everyone who responded to my Frankly Speaking column about the quality of our books. I counted 46 responses in all (of course, if that's the wrong number, someone will write and tell me), including one from Tim O'Reilly, one of my faithful readers. The comments (still available for your review) range from highly complimentary to critical, but all were helpful. Well, there was that one. But more about that one later. They covered a wide range of topics, all related to quality. I'll try to address as many as I can in this article.

Mostly, I'd like to say that it is a sobering responsibility to realize that our readers believe us when we say that we value high quality in our books. Our quality seems to be why readers start to buy our books, and, in some cases, the perceived decline of quality is the reason readers give for moving on to other sources of technical information. We're not always the first to publish on a topic, and we're not always the least expensive; but our readers cut us some slack in those two areas. When we are seen to compromise on quality, though, our readers give up on us. So we take these comments seriously. Every comment that contained a criticism was sent to the person responsible for that area, or to the editor responsible for that topic.

Some people who responded to my column agreed with Steve Adams' initial comment that our books currently contain more errors (typos, programming errors) than in the past. I said that I didn't believe that to be true, but I must admit that we have not kept statistics on the number of errors found in each printing, so my statement is based on anecdotal evidence only. (I'd like to begin to collect such data; I'm talking to our technology team about how to do so.) I did speak with current and previous reprint editors, and I looked at some books we printed and published from three, five, and seven years ago. Every book contained errors, some substantial, some minor. I haven't seen any period of time, including the present, when the likelihood of errors is substantially different from any other time.

Here are some reasons that might explain the perception that there are more errors in our books:

• We sell more copies of each book than ever before. To paraphrase Eric Raymond speaking about open source software in The Cathedral and the Bazaar, more eyes find more mistakes. (He actually said "Given enough eyeballs, all bugs are shallow," but the points are close.) We are thankful that we have readers who tell us when they find mistakes.

• We have more titles, so there are more opportunities for errors.

• If a reader goes from reading a number of O'Reilly books with relatively few errors, to reading an O'Reilly book that has more errors, that reader may assume that the quality of O'Reilly books is going down.

Our reprint policy determines how many errors a reader is likely to find in our books. We fix errors in nearly every reprint we send out; so a mistake that someone finds and reports in January may be fixed in the books printed in April, even though it's the same edition of the book; that is, you can't tell from the cover that we've changed anything. As a book goes through more reprints, it becomes a better book.

(I'm going to reveal a little-known fact here. We don't note every printing in the Printing History on the copyright page of our books. We note a printing in the printing history only if a significant number of corrections has been made. If you want to know when your copy was printed, look at the date in the bottom right-hand corner of the copyright page. There, in mm/yy form, is the printing date for your copy.)

In our early years, we reprinted often. For a few years (1993, say, to 1996), we increased our initial printing of a book to reduce our manufacturing costs. After 1996, we returned to printing smaller quantities to permit us to manage our inventory and fix our mistakes more quickly. On average, we print new versions of books every six months. Because we keep our backlist books in print longer than other publishers, and because most of those books have long ago had their errors fixed, you can conclude that our more recent books are reprinted every three or four months. Our most popular titles are reprinted more often than that. So a mistake, found by anyone, is unlikely to stay in a book more than a few months.

We take fixing our errors more seriously than any other publisher I know. We have a customer service department, of course; but we also have , a group that answers (or finds answers for) your technical questions about a book. We have three Production Editors dedicated just to fixing errors in books going to reprint. And, on every books' catalog page, we provide a way for readers to report and check errata by printing and edition. There isn't another publisher who does what we do.

I won't discuss binding in any depth here; I'm going to ask someone from Manufacturing to provide that material. We rely on a few close partners for our printing, in order to keep our quality high. Even so, from time to time, we get a bad batch of bindings. If your binding comes loose or pages fall out, take it back to where you bought it and they'll replace it. If they don't, please contact our Customer Service department. They will replace your book with a copy from the newest printing.

We use lay-flat bindings on every book we can. However, once a book exceeds 600 pages we have to use a more traditional perfect binding where the pages are glued directly to the cover. As Tim noted in his response, a lay-flat binding can look like it's coming loose from the cover, when, in fact, there is nothing wrong. If your cover comes off or pages fall off, that's a problem, and let us know. If the pages seem to be attached to an inner spine, that might be a feature rather than a bug.

Of most concern to me were questions of content quality. Readers mentioned several books that were not up to our standards. Of course, once again, there are instances in which a reader likes several books and then dislikes one. That reader assumes quality has dropped. But a number of books were mentioned more than once. A book mentioned more than once seemed to have one of three characteristics:

• The book was considered to be too close to the online documentation available for that technology. This is an interesting problem. It used to be that a Unix program had only a bare-bones manpage unless there was an O'Reilly book. Therefore, the release of certain O'Reilly books (like Sendmail) resulted in near riots. The improvement that the book represented over the manpage was dramatic. Now, online documentation is generally much better. Sun provides very good online reference documentation for Java, for example. With the Web, much more information is available about a technology, and with near-universal connectivity and high-speed connections, users can get lots of good online information quickly and easily from their vendors and their peers. Our books have to work harder to provide a significant improvement over the online sources.

  Another part of the problem is that we attempt to sign the key person in a technical community to write our book. It is sometimes the case with open source technologies that this same person wrote the online documentation. Larry Wall and Tom Christiansen, for example, write many of the Perl manpages and also our Camel book. Matt Welsh, a member of the Linux Documentation Project, also wrote Running Linux for us. These authors aren't stealing from the online sources; they're responsible for them. Nevertheless, I see a considerable difference between the level of information in our Running Linux book and the online material for Linux. I get the point, though: online documentation is often good nowadays, and we have to be a lot better or readers will resent paying us for our books.

• Our first edition of a book wasn't strong enough. Apache: The Definitive Guide and Virtual Private Networks are two such books. These were new technologies when we published these books, and we just didn't know what we didn't know. All our books are reviewed by outside, independent reviewers, usually five or more. In the case of the first editions of these books, we seemed to miss a part of the community in our reviews. So, while these books did well from a revenue standpoint, they did not do well with our regular readers, who wanted more. In both cases, we brought out new versions quickly that addressed the problems we heard about.

  In the case of VPN, we put the new material up on our Web site even before it was published, to supplement the edition we had in the bookstores. We believe that the current versions are better than the initial ones, and, in both cases, third editions are underway. But we have learned that our readers don't want to get an inferior version with the promise of a better one when we know more. They want quality from the start, and that's what they'll get from now on.

  UML in a Nutshell is the only book I'm very concerned about that fits in this category. We don't have a new edition planned for it, and it represents a technology that we have not followed as a company.

• We've done the introductory book first. Learning Perl/Tk, for example, is popular with those readers who are unfamiliar with Perl/Tk, but it disappointed our hardcore Perl audience, which expected a technical book that took them beyond their current knowledge of Perl/Tk. It is this deeper version that our readers seem to expect of us. They accept an introductory book from us only if we've published the more technical book first. We now know that we have to satisfy the more technical reader first. To put it another way, our first book on a technology should satisfy even the current users of that technology. As David Brickner pointed out, at O'Reilly, our books compete not with our competition, but with our other books and our reputation.

I should also note that we've had to adjust to increased competition from other publishers. Not so very long ago, many topics would lie in slumber until O'Reilly published a book on it. Now our competitors are publishing books on our traditional topics, and, sometimes, we wish we had their books. Overall, the quality of technical trade books is a lot better than when I started with O'Reilly six years ago. This change makes life harder for us, but it increases your choices.

We know we're in a dogfight for your attention and your loyalty. If we fall down on the job from time to time, we apologize. We certainly are now very aware that the major characteristic you look for from us is quality, and we'll take that seriously. Or Tim will fire our sorry behinds.

I have one last point. I noted at the top of this article that there was one comment that was not helpful. One person said that my earlier article had left a "fecal taste" in his mouth because I had blamed our errors on trying to provide "the best information out at the lowest price." He might reread my article; I never said any such thing. We don't trade off quality for price. (I said I was sometimes tempted to trade fit and finish at the end of a project for timeliness, which I consider a matter of quality as well.) So that reader might want to get some reading glasses and a new mouthwash.

Frank Willison
Editor-in-Chief

Return to: Frankly Speaking