English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

There is increasing interest in the use of HPC machines for urgent workloadsto help tackle disasters as they unfold. Whilst batch queue systems are notideal in supporting such workloads, many disadvantages can be worked around byaccurately predicting when a waiting job will start to run. However there arenumerous challenges in achieving such a prediction with high accuracy, notleast because the queue's state can change rapidly and depend upon manyfactors. In this work we explore a novel machine learning approach forpredicting queue wait times, hypothesising that such a model can capture thecomplex behaviour resulting from the queue policy and other interactions togenerate accurate job start times.  For ARCHER2 (HPE Cray EX), Cirrus (HPE 8600) and 4-cabinet (HPE Cray EX) weexplore how different machine learning approaches and techniques improve theaccuracy of our predictions, comparing against the estimation generated bySlurm. We demonstrate that our techniques deliver the most accurate predictionsacross our machines of interest, with the result of this work being the abilityto predict job start times within one minute of the actual start time foraround 65\% of jobs on ARCHER2 and 4-cabinet, and 76\% of jobs on Cirrus. Whencompared against what Slurm can deliver, this represents around 3.8 timesbetter accuracy on ARCHER2 and 18 times better for Cirrus. Furthermore ourapproach can accurately predicting the start time for three quarters of all jobwithin ten minutes of the actual start time on ARCHER2 and 4-cabinet, and for90\% of jobs on Cirrus. Whilst the driver of this work has been to betterfacilitate placement of urgent workloads across HPC machines, the insightsgained can be used to provide wider benefits to users and also enrich existingbatch queue systems and inform policy too.

Predicting batch queue job wait times for informed scheduling of urgent HPC workloads