Premium
Work stealing for GPU‐accelerated parallel programs in a global address space framework
Author(s) -
Arafat Humayun,
Dinan James,
Krishnamoorthy Sriram,
Balaji Pavan,
Sadayappan P.
Publication year - 2016
Publication title -
concurrency and computation: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.309
H-Index - 67
eISSN - 1532-0634
pISSN - 1532-0626
DOI - 10.1002/cpe.3747
Subject(s) - computer science , parallel computing , task (project management) , graphics processing unit , runtime system , graphics , central processing unit , parallelism (grammar) , distributed memory , load balancing (electrical power) , task parallelism , data parallelism , computation , general purpose computing on graphics processing units , distributed computing , shared memory , operating system , programming language , geometry , mathematics , management , economics , grid
Summary Task parallelism is an attractive approach to automatically load balance the computation in a parallel system and adapt to dynamism exhibited by parallel systems. Exploiting task parallelism through work stealing has been extensively studied in shared and distributed‐memory contexts. In this paper, we study the design of a system that uses work stealing for dynamic load balancing of task‐parallel programs executed on hybrid distributed‐memory CPU‐graphics processing unit (GPU) systems in a global‐address space framework. We take into account the unique nature of the accelerator model employed by GPUs, the significant performance difference between GPU and CPU execution as a function of problem size, and the distinct CPU and GPU memory domains. We consider various alternatives in designing a distributed work stealing algorithm for CPU‐GPU systems, while taking into account the impact of task distribution and data movement overheads. These strategies are evaluated using microbenchmarks that capture various execution configurations as well as the state‐of‐the‐art CCSD(T) application module from the computational chemistry domain. Copyright © 2016 John Wiley & Sons, Ltd.