Premium
The case for colocation of high performance computing workloads
Author(s) -
Breslow Alex D.,
Porter Leo,
Tiwari Ananta,
Laurenzano Michael,
Carrington Laura,
Tullsen Dean M.,
Snavely Allan E.
Publication year - 2016
Publication title -
concurrency and computation: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.309
H-Index - 67
eISSN - 1532-0634
pISSN - 1532-0626
DOI - 10.1002/cpe.3187
Subject(s) - data striping , computer science , supercomputer , parallel computing , heuristics , throughput , job scheduler , efficient energy use , mimd , disjoint sets , load balancing (electrical power) , distributed computing , computation , massively parallel , scheduling (production processes) , operating system , algorithm , cloud computing , mathematical optimization , geometry , mathematics , grid , combinatorics , electrical engineering , wireless , engineering
Summary The current state of practice in supercomputer resource allocation places jobs from different users on disjoint nodes both in terms of time and space. While this approach largely guarantees that jobs from different users do not degrade one another's performance, it does so at high cost to system throughput and energy efficiency. This focused study presents job striping, a technique that significantly increases performance over the current allocation mechanism by colocating pairs of jobs from different users on a shared set of nodes. To evaluate the potential of job striping in large‐scale environments, the experiments are run at the scale of 128 nodes on the state‐of‐the‐art Gordon supercomputer. Across all pairings of 1024 process network‐attached storage parallel benchmarks, job striping increases mean throughput by 26% and mean energy efficiency by 22%. On pairings of the real applications Gyrokinetic Toroidal Code (GTC), Large‐scale Atomic/Molecular Massively Parallel Simulator (LAMMPS), and MIMD Lattice Computation (MILC) at equal scale, job striping improves average throughput by 12% and mean energy efficiency by 11%. In addition, the study provides a simple set of heuristics for avoiding low performing application pairs. Copyright © 2013 John Wiley & Sons, Ltd.