Design and evaluation of primitives for parallel I/O

It is shown that the performance of parallel file systems can vary greatly as a function of the selected data distributions, and that some data distributions can not be supported. The authors have devised an alternative scheme for conducting parallel I/O, the two-phase access strategy, which generates higher and more consistent performance over a wider spectrum of data distributions. They have designed and implemented runtime primitives that make use of the two-phase access strategy to conduct parallel I/O, and facilitate the programming of parallel I/O operations. They describe these primitives in detail and provide performance results which show that I/O access rates are improved by up to several orders of magnitude. Further, they show that the variation in performance over various data distributions is restricted to within a factor of 2 of the best access rate.