From scientific applications to transaction processing, clustering technology provides an affordable, scalable computing solution. One of the hottest topics in Linux today is the ability to cluster Linux machines to mimic the performance of supercomputers (costing hundreds of thousands of dollars) for a fraction of the cost.
"Computing problems always seem to meet or exceed available computing resources. There is always a need for more processing power, more network bandwidth, and greater I/O capabilities," says Spector. "This need for speed has prompted a whole new class of machines: supercomputers. In fact, supercomputer class machines-the construction of which is the topic of my book-influence almost every aspect of modern life."
"The Linux operating system has given rise to a host of interesting and unique computing possibilities that would have been impossible, and perhaps even unthinkable, in a pre-Linux era," says Spector. Some examples that Spector gives, include:
- A single cross-platform operating system
- An open source operating system whose entire code base is freely available for research, commercial use, and improvement
- A system that has the potential to displace both Unix variants and Windows NT as a server operating system
- A scaleable parallel processing system that allows performance comparable to traditional supercomputers (IBM SP/2 or Cray), but at a fraction of the cost
"Since the ground-breaking work was done at NASA, Beowulf-style clusters have become their own well-defined genre of high-performance computing systems," explains Spector. "By October 1999, several of the systems in the top 200 supercomputers in the world were made up of Linux cluster systems."
Spector's first Linux cluster, and ultimately his new book Building Linux Clusters, arose from his need for an affordable home supercomputer for his research. Building it was a challenge. "I noticed that a lot of people talk about it, but the documentation is really terse. There's no way for someone to start who doesn't have all the knowledge already." Spector wrote up an outline and realized it would make a great O'Reilly book. And, as it turned out, O'Reilly was looking for just such a book.
Building Linux Clusters is a hands-on guide for people new to clusters. "Like most O'Reilly books," said Spector, "it's to get people going. You can build a cluster right out of the box: You pop the CD in, follow the instructions, and in ten or twenty minutes you have a fully functional cluster."
Spector likes to say that the book tells a story. And indeed, the first chapter is a brief, lively history of the evolution of supercomputers and, eventually, of clusters, including Don Becker's first Beowulf cluster. Built in 1994 to meet NASA's need for supercomputer performance on a shoestring budget, it used Linux and off-the-shelf hardware, saving NASA a couple of million bucks, according to Spector.
Spector thinks that the most exciting future applications of clusters are the least possible to predict. That's because learning to work with parallel computing changes the way people look at problems and opens new possibilities for applications. "People have to step out of the mindset of being serial and think about how to do everything at the same time. And that's a hard transition, but once you do that, suddenly new vistas open to you."
Online Resources:
- An interview with David HM Spector.
- Chapter 9, Application Examples.
- More information about the book.
- A cover graphic in jpeg format.
Scaling Linux for Scientific and Enterprise Applications
By David HM Spector
1st Edition, August 2000
1-56592-625-0, 352 pages, $44.95, Includes CD-ROM
order@oreilly.com
1-800-998-9938
About O’Reilly
O’Reilly, the premier learning platform for technology professionals, offers the industry’s most extensive catalog of high-quality technical and professional skills development courses. From AI, programming, and cloud technologies to essential business skills such as leadership training and critical thinking, O’Reilly delivers highly trusted content from its network of renowned experts that meets a diverse array of learning needs, with over 5,000 role-based on-demand courses, nearly 200 live events each month, access to interactive sandboxes and labs, and more. For more information, visit www.oreilly.com.