Declarative Parallel Programming

A major difficulty in parallel programming is getting the communication right—ensuring correctness while getting the best possible parallel performance and keeping the code maintainable. Unfortunately, These are conflicting requirements. As a result, high performance programmers are often forced to tradeoff code readability and maintainability in favor of performance. For example, asynchronous MPI calls result in “spaghetti” code where communication primitives may occur far from where the communicated data are used or produced. A more detailed motivation for the project provides other examples.

This research explores adding an embedded domain-specific language (EDSL) within a host language, such as C++, to express communication steps as collectives using set-based notation. This approach is especially well-suited to Bulk Synchronous Parallel (BSP)-style programs, which cover a vast majority of scientific parallel programs. We have developed an EDSL, called Kanor, hosted within C++, and an accompanying compiler based on ROSE.

Kanor has also been extended with an EDSL to express GPU-bound computations with a language called, Harlan. Kanor and Harlan complement each other and make it possible to write scalable parallel programs for heterogeneous targets.

Funded by NSF, award CNS-0834722

Related publications:

1. Nilesh Mahajan, Uday Pitambare and Arun Chauhan. Globalizing Selectively: Shared-memory Efficiency with Address-space Separation. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), 2013.
   [Article DOI]
2. Joseph A. Cottam, Eric Holk, William E. Byrd, Arun Chauhan and Andrew Lumsdaine. High-level Coordination Specification: Operational Semantics for Kanor. In Workshop on Leveraging Abstractions and Semantics for High Performance Computing (LASH-C), 2013. Held in conjunction with the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP).
3. Eric Holk, William Byrd, Nilesh Mahajan, Jeremiah Willcock, Arun Chauhan and Andrew Lumsdaine. Declarative Parallel Programming for GPUs. In Koen De Bosschere, Erik H. D'hollander, Gerhard R. Joubert, David Padua, Frans Peters and Mark Sawyer, editors, Applications, Tools and Techniques on the Road to Exascale Computing, volume 22 in Advances in Parallel Computing, pages 297–304. IOS Press, Amsterdam, Netherlands, 2012. Proceedings of the 14th biennial ParCo Conference, 2011.
   [Full text]
4. Nilesh Mahajan, Sajith Sasidharan, Arun Chauhan and Andrew Lumsdaine. Automatically Generating Coarse Grained Software Pipelining from Declaratively Specified Communication. In Proceedings of the 18th International Conference on High Performance Computing (HiPC), 2011. Short paper in the Student Research Symposium.
   [Full text]
5. Fangzhou Jiao, Nilesh Mahajan, Jeremiah Willcock, Arun Chauhan and Andrew Lumsdaine. Partial Globalization of Partitioned Address Spaces for Zero-copy Communication with Shared Memory. In Proceedings of the 18th International Conference on High Performance Computing (HiPC), 2011.
   [Article DOI]
6. Eric Holk, William Byrd, Nilesh Mahajan, Jeremiah Willcock, Arun Chauhan and Andrew Lumsdaine. Declarative Parallel Programming for GPUs. In Proceedings of the International Conference on Parallel Computing (ParCo), 2011.
   [Full text]
7. Eric Holk, William E Byrd, Jeremiah Willcock, Torsten Hoefler, Arun Chauhan and Andrew Lumsdaine. Kanor: A Declarative Language for Explicit Communication. In Proceedings of the Thirteenth International Symposium on the Practical Aspects of Declarative Languages (PADL), 2011.
   [Article DOI]
8. Torsten Hoefler, Jeremiah Willcock, Arun Chauhan and Andrew Lumsdaine. The Case for Collective Pattern Specification. In Proceedings of the First Workshop on the Advances in Message Passing (AMP), 2010. Held in conjunction with the ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI).
   [Full text]