Clojure in Action

Chapter 12. Data processing with Clojure

 

This chapter covers

• The map/reduce pattern of data processing
• Analyzing log files using map/reduce
• Distributing the data processing
• Master/slave parallelization

 

A computer program accepts data that is given, manipulates it in some way, and provides some output. The growing volume of data collected every minute of every day is evidence that data processing is alive in most software today. This chapter is about writing such programs. Naturally, you’ll want to do this in as functional and as Clojure-esque a way as possible.

We’re going to examine two approaches to processing large volumes of data. The first is the approach known as map/reduce. We’ll show what it is, use it to parse log data, and ...