High Performance Computing

High Performance Computing

by Thomas Sterling, Maciej Brodowicz, Matthew Anderson
Released December 2017
Publisher(s): Morgan Kaufmann
ISBN: 9780124202153

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

High Performance Computing: Modern Systems and Practices is a fully comprehensive and easily accessible treatment of high performance computing, covering fundamental concepts and essential knowledge while also providing key skills training. With this book, domain scientists will learn how to use supercomputers as a key tool in their quest for new knowledge. In addition, practicing engineers will discover how supercomputers can employ HPC systems and methods to the design and simulation of innovative products, and students will begin their careers with an understanding of possible directions for future research and development in HPC.

Those who maintain and administer commodity clusters will find this textbook provides essential coverage of not only what HPC systems do, but how they are used.

  • Covers enabling technologies, system architectures and operating systems, parallel programming languages and algorithms, scientific visualization, correctness and performance debugging tools and methods, GPU accelerators and big data problems
  • Provides numerous examples that explore the basics of supercomputing, while also providing practical training in the real use of high-end computers
  • Helps users with informative and practical examples that build knowledge and skills through incremental steps
  • Features sidebars of background and context to present a live history and culture of this unique field
  • Includes online resources, such as recorded lectures from the authors’ HPC courses

Table of contents

  1. Cover image
  2. Title page
  3. Table of Contents
  4. Copyright
  5. Dedication
  6. Foreword
  7. Preface
  8. Acknowledgments
  9. Chapter 1. Introduction
    1. 1.1. High Performance Computing Disciplines
    2. 1.2. Impact of Supercomputing on Science, Society, and Security
    3. 1.3. Anatomy of a Supercomputer
    4. 1.4. Computer Performance
    5. 1.5. A Brief History of Supercomputing
    6. 1.6. This Textbook as a Guide and Tool for the Student
    7. 1.7. Summary and Outcomes of Chapter 1
    8. 1.8. Questions and Problems
  10. Chapter 2. HPC Architecture 1: Systems and Technologies
    1. 2.1. Introduction
    2. 2.2. Key Properties of HPC Architecture
    3. 2.3. Parallel Architecture Families—Flynn's Taxonomy
    4. 2.4. Enabling Technology
    5. 2.5. von Neumann Sequential Processors
    6. 2.6. Vector and Pipelining
    7. 2.7. Single-Instruction, Multiple Data Array
    8. 2.8. Multiprocessors
    9. 2.9. Heterogeneous Computer Structures
    10. 2.10. Summary and Outcomes of Chapter 2
    11. 2.11. Questions and Problems
  11. Chapter 3. Commodity Clusters
    1. 3.1. Introduction
    2. 3.2. Beowulf Cluster Project
    3. 3.3. Hardware Architecture
    4. 3.4. Programming Interfaces
    5. 3.5. Software Environment
    6. 3.6. Basic Methods of Use
    7. 3.7. Summary and Outcomes of Chapter 3
    8. 3.8. Questions and Exercises
  12. Chapter 4. Benchmarking
    1. 4.1. Introduction
    2. 4.2. Key Properties of an HPC Benchmark
    3. 4.3. Standard HPC Community Benchmarks
    4. 4.4. Highly Parallel Computing Linpack
    5. 4.5. HPC Challenge Benchmark Suite
    6. 4.6. High Performance Conjugate Gradients
    7. 4.7. NAS Parallel Benchmarks
    8. 4.8. Graph500
    9. 4.9. Miniapplications as Benchmarks
    10. 4.10. Summary and Outcomes of Chapter 4
    11. 4.11. Exercises
  13. Chapter 5. The Essential Resource Management
    1. 5.1. Managing Resources
    2. 5.2. The Essential SLURM
    3. 5.3. The Essential Portable Batch System
    4. 5.4. Summary and Outcomes of Chapter 5
    5. 5.5. Questions and Problems
  14. Chapter 6. Symmetric Multiprocessor Architecture
    1. 6.1. Introduction
    2. 6.2. Architecture Overview
    3. 6.3. Amdahl's Law Plus
    4. 6.4. Processor Core Architecture
    5. 6.5. Memory Hierarchy
    6. 6.6. PCI Bus
    7. 6.7. External I/O Interfaces
    8. 6.8. Summary and Outcomes of Chapter 6
    9. 6.9. Questions and Exercises
  15. Chapter 7. The Essential OpenMP
    1. 7.1. Introduction
    2. 7.2. Overview of OpenMP Programming Model
    3. 7.3. Parallel Threads and Loops
    4. 7.4. Synchronization
    5. 7.5. Reduction
    6. 7.6. Summary and Outcomes of Chapter 7
    7. 7.7. Questions and Problems
  16. Chapter 8. The Essential MPI
    1. 8.1. Introduction
    2. 8.2. Message-Passing Interface Standards
    3. 8.3. Message-Passing Interface Basics
    4. 8.4. Communicators
    5. 8.5. Point-to-Point Messages
    6. 8.6. Synchronization Collectives
    7. 8.7. Communication Collectives
    8. 8.8. Nonblocking Point-to-Point Communication
    9. 8.9. User-Defined Data Types
    10. 8.10. Summary and Outcomes of Chapter 8
    11. 8.11. Exercises
  17. Chapter 9. Parallel Algorithms
    1. 9.1. Introduction
    2. 9.2. Fork–Join
    3. 9.3. Divide and Conquer
    4. 9.4. Manager–Worker
    5. 9.5. Embarrassingly Parallel
    6. 9.6. Halo Exchange
    7. 9.7. Permutation: Cannon's Algorithm
    8. 9.8. Task Dataflow: Breadth First Search
    9. 9.9. Summary and Outcomes of Chapter 9
    10. 9.10. Exercises
  18. Chapter 10. Libraries
    1. 10.1. Introduction
    2. 10.2. Linear Algebra
    3. 10.3. Partial Differential Equations
    4. 10.4. Graph Algorithms
    5. 10.5. Parallel Input/Output
    6. 10.6. Mesh Decomposition
    7. 10.7. Visualization
    8. 10.8. Parallelization
    9. 10.9. Signal Processing
    10. 10.10. Performance Monitoring
    11. 10.11. Summary and Outcomes of Chapter 10
    12. 10.12. Exercises
  19. Chapter 11. Operating Systems
    1. 11.1. Introduction
    2. 11.2. Operating System Structures and Services
    3. 11.3. Process Management
    4. 11.4. Threads
    5. 11.5. Memory Management
    6. 11.6. Summary and Outcomes of Chapter 11
    7. 11.7. Exercises
  20. Chapter 12. Visualization
    1. 12.1. Introduction
    2. 12.2. Foundational Visualization Concepts
    3. 12.3. Gnuplot
    4. 12.4. Matplotlib
    5. 12.5. The Visualization Toolkit
    6. 12.6. ParaView
    7. 12.7. VisIt
    8. 12.8. Summary and Outcomes of Chapter 12
    9. 12.9. Exercises
  21. Chapter 13. Performance Monitoring
    1. 13.1. Introduction
    2. 13.2. Time Measurement
    3. 13.3. Performance Profiling
    4. 13.4. Monitoring Hardware Events
    5. 13.5. Integrated Performance Monitoring Toolkits
    6. 13.6. Profiling in Distributed Environments
    7. 13.7. Summary and Outcomes of Chapter 13
    8. 13.8. Questions and Problems
  22. Chapter 14. Debugging
    1. 14.1. Introduction
    2. 14.2. Tools
    3. 14.3. Debugging OpenMP Example: Accessing an Unprotected Shared Variable
    4. 14.4. Debugging MPI Example: Deadlock
    5. 14.5. Compiler Flags for Debugging
    6. 14.6. System Monitors to Aid Debugging
    7. 14.7. Summary and Outcomes of Chapter 14
    8. 14.8. Exercises
  23. Chapter 15. Accelerator Architecture
    1. 15.1. Introduction
    2. 15.2. A Historic Perspective
    3. 15.3. Introduction to Graphics Processing Units
    4. 15.4. Evolution of Graphics Processing Unit Functionality
    5. 15.5. Modern Graphics Processing Unit Architecture
    6. 15.6. Heterogeneous System Architecture
    7. 15.7. Summary and Outcomes of Chapter 15
    8. 15.8. Problems and Questions
  24. Chapter 16. The Essential OpenACC
    1. 16.1. Introduction
    2. 16.2. OpenACC Programming Concepts
    3. 16.3. OpenACC Library Calls
    4. 16.4. OpenACC Environment Variables
    5. 16.5. OpenACC Directives
    6. 16.6. Summary and Outcomes of Chapter 16
    7. 16.7. Questions and Problems
  25. Chapter 17. Mass Storage
    1. 17.1. Introduction
    2. 17.2. Brief History of Storage
    3. 17.3. Storage Device Technology
    4. 17.4. Aggregated Storage
    5. 17.5. Summary and Outcomes of Chapter 17
    6. 17.6. Questions and Problems
  26. Chapter 18. File Systems
    1. 18.1. Role and Function of File Systems
    2. 18.2. The Essential POSIX File Interface
    3. 18.3. Network File System
    4. 18.4. General Parallel File System
    5. 18.5. Lustre File System
    6. 18.6. Summary and Outcomes of Chapter 18
    7. 18.7. Questions and Problems
  27. Chapter 19. MapReduce
    1. 19.1. Introduction
    2. 19.2. Map and Reduce
    3. 19.3. Distributed Computation
    4. 19.4. Hadoop
    5. 19.5. Summary and Outcomes of Chapter 19
    6. 19.6. Exercises
  28. Chapter 20. Checkpointing
    1. 20.1. Introduction
    2. 20.2. System-Level Checkpointing
    3. 20.3. Application-Level Checkpointing
    4. 20.4. Summary and Outcomes of Chapter 20
    5. 20.5. Exercises
  29. Chapter 21. Next Steps and Beyond
    1. 21.1. Introduction
    2. 21.2. Expanded Parallel Programming Models
    3. 21.3. Extended High Performance Computing Architecture
    4. 21.4. Exascale Computing
    5. 21.5. Asynchronous Multitasking
    6. 21.6. The Neodigital Age
    7. 21.7. Exercises
  30. Appendix A. Essential C
  31. Appendix B. Essential Linux
  32. Glossary
  33. Index

Product information

  • Title: High Performance Computing
  • Author(s): Thomas Sterling, Maciej Brodowicz, Matthew Anderson
  • Release date: December 2017
  • Publisher(s): Morgan Kaufmann
  • ISBN: 9780124202153