Python Parallel Programming Cookbook

Python Parallel Programming Cookbook

by Giancarlo Zaccone
Released August 2015
Publisher(s): Packt Publishing
ISBN: 9781785289583

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Master efficient parallel programming to build powerful applications using Python

About This Book

  • Design and implement efficient parallel software
  • Master new programming techniques to address and solve complex programming problems
  • Explore the world of parallel programming with this book, which is a go-to resource for different kinds of parallel computing tasks in Python, using examples and topics covered in great depth

Who This Book Is For

Python Parallel Programming Cookbook is intended for software developers who are well versed with Python and want to use parallel programming techniques to write powerful and efficient code. This book will help you master the basics and the advanced of parallel computing.

What You Will Learn

  • Synchronize multiple threads and processes to manage parallel tasks
  • Implement message passing communication between processes to build parallel applications
  • Program your own GPU cards to address complex problems
  • Manage computing entities to execute distributed computational tasks
  • Write efficient programs by adopting the event-driven programming model
  • Explore the cloud technology with DJango and Google App Engine
  • Apply parallel programming techniques that can lead to performance improvements

In Detail

Parallel programming techniques are required for a developer to get the best use of all the computational resources available today and to build efficient software systems. From multi-core to GPU systems up to the distributed architectures, the high computation of programs throughout requires the use of programming tools and software libraries. Because of this, it is becoming increasingly important to know what the parallel programming techniques are. Python is commonly used as even non-experts can easily deal with its concepts.

This book will teach you parallel programming techniques using examples in Python and will help you explore the many ways in which you can write code that allows more than one process to happen at once. Starting with introducing you to the world of parallel computing, it moves on to cover the fundamentals in Python. This is followed by exploring the thread-based parallelism model using the Python threading module by synchronizing threads and using locks, mutex, semaphores queues, GIL, and the thread pool.

Next you will be taught about process-based parallelism where you will synchronize processes using message passing along with learning about the performance of MPI Python Modules. You will then go on to learn the asynchronous parallel programming model using the Python asyncio module along with handling exceptions. Moving on, you will discover distributed computing with Python, and learn how to install a broker, use Celery Python Module, and create a worker.

You will also understand the StarCluster framework, Pycsp, Scoop, and Disco modules in Python. Further on, you will learn GPU programming with Python using the PyCUDA module along with evaluating performance limitations. Next you will get acquainted with the cloud computing concepts in Python, using Google App Engine (GAE), and building your first application with GAE. Lastly, you will learn about grid computing concepts in Python and using PyGlobus toolkit, GFTP and GASS COPY to transfer files, and service monitoring in PyGlobus.

Style and approach

A step-by-step guide to parallel programming using Python, with recipes accompanied by one or more programming examples. It is a practically oriented book and has all the necessary underlying parallel computing concepts.

Table of contents

  1. Python Parallel Programming Cookbook
    1. Table of Contents
    2. Python Parallel Programming Cookbook
    3. Credits
    4. About the Author
    5. About the Reviewers
    6. www.PacktPub.com
      1. Support files, eBooks, discount offers, and more
        1. Why Subscribe?
        2. Free Access for Packt account holders
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Sections
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
        5. See also
      5. Conventions
      6. Reader feedback
      7. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    8. 1. Getting Started with Parallel Computing and Python
      1. Introduction
      2. The parallel computing memory architecture
        1. SISD
        2. MISD
        3. SIMD
        4. MIMD
      3. Memory organization
        1. Shared memory
        2. Distributed memory
          1. Massively parallel processing
          2. A cluster of workstations
          3. The heterogeneous architecture
      4. Parallel programming models
        1. The shared memory model
        2. The multithread model
        3. The message passing model
        4. The data parallel model
      5. How to design a parallel program
        1. Task decomposition
        2. Task assignment
        3. Agglomeration
        4. Mapping
          1. Dynamic mapping
            1. Manager/worker
            2. Hierarchical manager/worker
            3. Decentralize
      6. How to evaluate the performance of a parallel program
        1. Speedup
        2. Efficiency
        3. Scaling
        4. Amdahl's law
        5. Gustafson's law
      7. Introducing Python
        1. Getting ready
        2. How to do it…
      8. Python in a parallel world
      9. Introducing processes and threads
      10. Start working with processes in Python
        1. Getting ready
        2. How to do it…
        3. How it works…
      11. Start working with threads in Python
        1. How to do it…
        2. How it works…
    9. 2. Thread-based Parallelism
      1. Introduction
      2. Using the Python threading module
      3. How to define a thread
        1. How to do it…
        2. How it works…
      4. How to determine the current thread
        1. How to do it…
        2. How it works…
      5. How to use a thread in a subclass
        1. How to do it…
        2. How it works…
      6. Thread synchronization with Lock and RLock
        1. How to do it…
        2. How it works…
        3. There's more…
      7. Thread synchronization with RLock
        1. How to do it…
        2. How it works…
      8. Thread synchronization with semaphores
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
      9. Thread synchronization with a condition
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
      10. Thread synchronization with an event
        1. How to do it…
        2. How it works…
      11. Using the with statement
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
      12. Thread communication using a queue
        1. How to do it…
        2. How it works…
      13. Evaluating the performance of multithread applications
        1. How to do it…
        2. How it works…
          1. The first test
          2. The second test
          3. The third test
          4. The fourth test
        3. There's more…
    10. 3. Process-based Parallelism
      1. Introduction
      2. How to spawn a process
        1. How to do it...
        2. How it works...
        3. There's more...
      3. How to name a process
        1. How to do it...
        2. How it works...
      4. How to run a process in the background
        1. How to do it...
        2. How it works...
        3. There's more...
      5. How to kill a process
        1. How to do it...
        2. How it works...
      6. How to use a process in a subclass
        1. How to do it...
        2. How it works...
      7. How to exchange objects between processes
        1. Using queue to exchange objects
        2. How to do it...
        3. How it works...
        4. There's more...
        5. Using pipes to exchange objects
        6. How to do it...
        7. How it works...
      8. How to synchronize processes
        1. How to do it...
        2. How it works...
      9. How to manage a state between processes
        1. How to do it...
        2. How it works...
      10. How to use a process pool
        1. How to do it…
        2. How it works…
      11. Using the mpi4py Python module
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
      12. Point-to-point communication
        1. How to do it…
        2. How it works…
        3. There's more…
      13. Avoiding deadlock problems
        1. How to do it…
        2. How it works…
        3. There's more…
      14. Collective communication using broadcast
        1. How to do it…
        2. How it works…
        3. There's more…
      15. Collective communication using scatter
        1. How to do it…
        2. How it works…
        3. There's more…
      16. Collective communication using gather
        1. How to do it…
        2. How it works…
        3. There's more…
      17. Collective communication using Alltoall
        1. How to do it…
        2. How it works…
        3. There's more…
      18. The reduction operation
        1. How to do it…
        2. How it works…
      19. How to optimize communication
        1. How to do it…
        2. How it works…
        3. There's more…
    11. 4. Asynchronous Programming
      1. Introduction
      2. Using the concurrent.futures Python modules
        1. Dealing with the process and thread pool
        2. Getting ready
        3. How to do it…
        4. How it works…
        5. There's more…
      3. Event loop management with Asyncio
        1. What is an event loop
        2. Getting ready
        3. How to do it…
        4. How it works…
      4. Handling coroutines with Asyncio
        1. Getting ready
        2. How to do it…
        3. How it works…
      5. Task manipulation with Asyncio
        1. Getting ready
        2. How to do it…
        3. How it works…
      6. Dealing with Asyncio and Futures
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
    12. 5. Distributed Python
      1. Introduction
      2. Using Celery to distribute tasks
        1. How to do it…
        2. See also
      3. How to create a task with Celery
        1. How to do it…
        2. How it works…
        3. There's more…
      4. Scientific computing with SCOOP
        1. Getting ready
        2. How to do it…
        3. How it works…
      5. Handling map functions with SCOOP
        1. Getting ready
        2. How to do it…
        3. How it works…
      6. Remote Method Invocation with Pyro4
        1. Getting ready
        2. How to do it…
        3. How it works…
      7. Chaining objects with Pyro4
        1. How to do it…
        2. How it works…
      8. Developing a client-server application with Pyro4
        1. How to do it…
        2. How it works…
      9. Communicating sequential processes with PyCSP
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
      10. Using MapReduce with Disco
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
      11. A remote procedure call with RPyC
        1. Getting ready
        2. How to do it…
        3. How it works…
    13. 6. GPU Programming with Python
      1. Introduction
      2. Using the PyCUDA module
        1. A hybrid programming model
        2. The kernel and thread hierarchy
        3. Getting ready
        4. How to do it…
        5. How it works…
        6. See also
      3. How to build a PyCUDA application
        1. How to do it…
        2. How it works…
        3. There's more…
      4. Understanding the PyCUDA memory model with matrix manipulation
        1. How to do it…
        2. How it works…
      5. Kernel invocations with GPUArray
        1. How to do it…
        2. How it works…
        3. There's more…
      6. Evaluating element-wise expressions with PyCUDA
        1. How to do it…
        2. How it works…
        3. There's more…
      7. The MapReduce operation with PyCUDA
        1. How to do it…
        2. How it works…
      8. GPU programming with NumbaPro
        1. Getting ready
        2. How to do it…
        3. How it works…
      9. Using GPU-accelerated libraries with NumbaPro
        1. How to do it…
        2. How it works…
        3. There's more…
      10. Using the PyOpenCL module
        1. Getting ready
        2. How to do it…
        3. How it works…
      11. How to build a PyOpenCL application
        1. How to do it…
        2. How it works…
      12. Evaluating element-wise expressions with PyOpenCl
        1. How to do it…
        2. How it works…
      13. Testing your GPU application with PyOpenCL
        1. How to do it…
        2. How it works…
    14. Index

Product information

  • Title: Python Parallel Programming Cookbook
  • Author(s): Giancarlo Zaccone
  • Release date: August 2015
  • Publisher(s): Packt Publishing
  • ISBN: 9781785289583