Python in a Nutshell, 4th Edition

Chapter 15. Concurrency: Threads and Processes

Processes are instances of running programs that the operating system protects from one another. Processes that want to communicate must explicitly arrange to do so via interprocess communication (IPC) mechanisms, and/or via files (covered in Chapter 11), databases (covered in Chapter 12), or network interfaces (covered in Chapter 18). The general way in which processes communicate using data storage mechanisms such as files and databases is that one process writes data, and another process later reads that data back. This chapter covers programming with processes, including the Python standard library modules subprocess and multiprocessing; the process-related parts of the module os, including simple IPC by means of pipes; a cross-platform IPC mechanism known as memory-mapped files, available in the module mmap; 3.8+ and the multiprocessing.shared_memory module.

A thread (originally called a “lightweight process”) is a flow of control that shares global state (memory) with other threads inside a single process; all threads appear to execute simultaneously, although they may in fact be “taking turns” on one or more processors/cores. Threads are far from easy to master, and multithreaded programs are often hard to test and to debug; however, as covered in “Threading, Multiprocessing, or Async Programming?”, when used appropriately, multithreading may improve performance in comparison to single-threaded programming. This chapter covers ...