Array-level Collective Communications

Abstract

The Message Passing Interface(MPI) provides some well-defined collective interfaces for a set of messages between multiple processes. However, MPI can be improved. The collective interface in MPI is too specialized, that many collective communication patterns still require instantiation by programmers. In this paper, we describe a novel tool Parallel Dimension(PADM). PADM is defined to extend MPI for a more general class of collective communications. Therefore, in PADM, both the MPI interfaces and other collective communications are performed with the same general solution. PADM provides array style abstraction for both process topology and data distribution from the global view. Unlike the highly Partitioned Global Address Space(PGAS) model, PADM allows programmer to explicitly control the distribution and communication. And PADM provides more concise communication sentences, and allows greater scope for optimization.