Premium
An efficient concurrent implementation of a neural network algorithm
Author(s) -
Andonie R.,
Chronopoulos A. T.,
Grosu D.,
Galmeanu H.
Publication year - 2006
Publication title -
concurrency and computation: practice and experience
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.309
H-Index - 67
eISSN - 1532-0634
pISSN - 1532-0626
DOI - 10.1002/cpe.987
Subject(s) - computer science , benchmark (surveying) , artificial neural network , backpropagation , computation , parallel computing , heuristic , set (abstract data type) , speedup , process (computing) , heterogeneous network , time complexity , distributed computing , algorithm , artificial intelligence , telecommunications , wireless network , geodesy , wireless , programming language , geography , operating system
The focus of this study is how we can efficiently implement the neural network backpropagation algorithm on a network of computers (NOC) for concurrent execution. We assume a distributed system with heterogeneous computers and that the neural network is replicated on each computer. We propose an architecture model with efficient pattern allocation that takes into account the speed of processors and overlaps the communication with computation. The training pattern set is distributed among the heterogeneous processors with the mapping being fixed during the learning process. We provide a heuristic pattern allocation algorithm minimizing the execution time of backpropagation learning. The computations are overlapped with communications. Under the condition that each processor has to perform a task directly proportional to its speed, this allocation algorithm has polynomial‐time complexity. We have implemented our model on a dedicated network of heterogeneous computers using Sejnowski's NetTalk benchmark for testing. Copyright © 2005 John Wiley & Sons, Ltd.