Book description
Frank Kane’s hands-on Spark training course, based on his bestselling Taming Big Data with Apache Spark and Python video, now available in a book. Understand and analyze large data sets using Spark on a single system or on a cluster.
About This Book
Understand how Spark can be distributed across computing clusters
Develop and run Spark jobs efficiently using Python
A hands-on tutorial by Frank Kane with over 15 real-world examples teaching you Big Data processing with Spark
Who This Book Is For
If you are a data scientist or data analyst who wants to learn Big Data processing using Apache Spark and Python, this book is for you. If you have some programming experience in Python, and want to learn how to process large amounts of data using Apache Spark, Frank Kane’s Taming Big Data with Apache Spark and Python will also help you.
What You Will Learn
Find out how you can identify Big Data problems as Spark problems
Install and run Apache Spark on your computer or on a cluster
Analyze large data sets across many CPUs using Spark’s Resilient Distributed Datasets
Implement machine learning on Spark using the MLlib library
Process continuous streams of data in real time using the Spark streaming module
Perform complex network analysis using Spark’s GraphX library
Use Amazon's Elastic MapReduce service to run your Spark jobs on a cluster
In Detail
Frank Kane’s Taming Big Data with Apache Spark and Python is your companion to learning Apache Spark in a hands-on manner. Frank will start you off by teaching you how to set up Spark on a single system or on a cluster, and you’ll soon move on to analyzing large data sets using Spark RDD, and developing and running effective Spark jobs quickly using Python.
Apache Spark has emerged as the next big thing in the Big Data domain – quickly rising from an ascending technology to an established superstar in just a matter of years. Spark allows you to quickly extract actionable insights from large amounts of data, on a real-time basis, making it an essential tool in many modern businesses.
Frank has packed this book with over 15 interactive, fun-filled examples relevant to the real world, and he will empower you to understand the Spark ecosystem and implement production-grade real-time Spark projects with ease.
Style and approach
Frank Kane’s Taming Big Data with Apache Spark and Python is a hands-on tutorial with over 15 real-world examples carefully explained by Frank in a step-by-step manner. The examples vary in complexity, and you can move through them at your own pace.
Table of contents
- Preface
- Getting Started with Spark
-
Spark Basics and Spark Examples
- What is Spark?
- The Resilient Distributed Dataset (RDD)
- Ratings histogram walk-through
- Key/value RDDs and the average friends by age example
- Running the average friends by age example
- Filtering RDDs and the minimum temperature by location example
- Running the minimum temperature example and modifying it for maximums
- Running the maximum temperature by location example
- Counting word occurrences using flatmap()
- Improving the word-count script with regular expressions
- Sorting the word count results
- Find the total amount spent by customer
- Check your results and sort them by the total amount spent
- Check your sorted implementation and results against mine
- Summary
-
Advanced Examples of Spark Programs
- Finding the most popular movie
- Using broadcast variables to display movie names instead of ID numbers
- Finding the most popular superhero in a social graph
- Running the script - discover who the most popular superhero is
- Superhero degrees of separation - introducing the breadth-first search algorithm
- Accumulators and implementing BFS in Spark
- Superhero degrees of separation - review the code and run it
- Item-based collaborative filtering in Spark, cache(), and persist()
- Running the similar-movies script using Spark's cluster manager
- Improving the quality of the similar movies example
- Summary
-
Running Spark on a Cluster
- Introducing Elastic MapReduce
- Setting up our Amazon Web Services / Elastic MapReduce account and PuTTY
- Partitioning
- Creating similar movies from one million ratings - part 1
- Creating similar movies from one million ratings - part 2
- Creating similar movies from one million ratings – part 3
- Troubleshooting Spark on a cluster
- More troubleshooting and managing dependencies
- Summary
- SparkSQL, DataFrames, and DataSets
- Other Spark Technologies and Libraries
- Where to Go From Here? – Learning More About Spark and Data Science
Product information
- Title: Frank Kane's Taming Big Data with Apache Spark and Python
- Author(s):
- Release date: June 2017
- Publisher(s): Packt Publishing
- ISBN: 9781787287945
You might also like
book
Apache Spark for Data Science Cookbook
Over insightful 90 recipes to get lightning-fast analytics with Apache Spark About This Book Use Apache …
book
Scala and Spark for Big Data Analytics
Harness the power of Scala to program Spark and analyze tonnes of data in the blink …
book
Scala Programming for Big Data Analytics : Get Started With Big Data Analytics Using Apache Spark
Gain the key language concepts and programming techniques of Scala in the context of big data …
book
Spark for Data Science
Analyze your data and delve deep into the world of machine learning with the latest Spark …