Scalable Computing and Communications: Theory and Practice

34

–––––––––––––––––––––––

Parallel Programming Models for Scalable Computing

James Dinan and Pavan Balaji

34.1   INTRODUCTION TO PARALLEL PROGRAMMING MODELS

A programming model can be thought of as the abstract machine that a programmer is writing instructions for. Such models form a rich topic for computer science research since programmers prefer them to be (1) expressive (capable of expressing any abstract algorithm), (2) portable (capable of being used on any computer architecture), (3) efficient (capable of delivering performance commensurate with that of the underlying hardware), and (4) easy to use. Often, simultaneously meeting all four of these requirements is hard, so different programming models make different trade-offs. This is one of the primary reasons several dozen scientific programming models exist today.

At the core of parallel computing is the broad space of languages and libraries used to craft parallel programs. Recent changes in system architecture have led to renewed interest in programming models as a means for mapping computation to complex hardware architectures and for enhancing the productivity of parallel programming on these platforms. Deepening memory hierarchies and nonuniformity in both the communication and the memory subsystems has led to an increased interest in models that can effectively incorporate the cost of data access, allowing programmers to exploit data locality in their application.

A parallel programming model defines the methods ...