Modeling and Simulation for Exploring Power/Time Trade-off of Parallel Deep Neural Network Training | Zendy

Paweł Rościszewski | Zendy

AI Assistant Blog Pricing

Home ZAIA Blog

Open Access

Modeling and Simulation for Exploring Power/Time Trade-off of Parallel Deep Neural Network Training

Author(s) -

Paweł Rościszewski

Publication year - 2017

Publication title -

procedia computer science

Language(s) - English

Resource type - Journals

SCImago Journal Rank - 0.334

H-Index - 76

ISSN - 1877-0509

DOI - 10.1016/j.procs.2017.05.214

Subject(s) - computer science , artificial neural network , discrete event simulation , power (physics) , power consumption , training (meteorology) , deep learning , real time computing , event (particle physics) , parallel computing , artificial intelligence , simulation , physics , quantum mechanics , meteorology

In the paper we tackle bi-objective execution time and power consumption optimization problem concerning execution of parallel applications. We propose using a discrete-event simulation environment for exploring this power/time trade-off in the form of a Pareto front. The solution is verified by a case study based on a real deep neural network training application for automatic speech recognition. A simulation lasting over 2 hours on a single CPU accurately predicts real results from executions that take over 335 hours in a cluster with 8 GPUs. The simulations allow also estimating the impact of data package imbalance on the application performance.

The content you want is available to Zendy users.

Already have an account? Click here to sign in.

Having issues? You can contact us here

Accelerating Research