English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper investigates the stochastic optimization problem with a focus ondeveloping scalable parallel algorithms for deep learning tasks. Our solutioninvolves a reformation of the objective function for stochastic optimization inneural network models, along with a novel parallel strategy, coined weightedaggregating stochastic gradient descent (WASGD). Following a theoreticalanalysis on the characteristics of the new objective function, WASGD introducesa decentralized weighted aggregating scheme based on the performance of localworkers. Without any center variable, the new method automatically assesses theimportance of local workers and accepts them according to their contributions.Furthermore, we have developed an enhanced version of the method, WASGD+, by(1) considering a designed sample order and (2) applying a more advanced weightevaluating function. To validate the new method, we benchmark our schemesagainst several popular algorithms including the state-of-the-art techniques(e.g., elastic averaging SGD) in training deep neural networks forclassification tasks. Comprehensive experiments have been conducted on fourclassic datasets, including the CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST.The subsequent results suggest the superiority of the WASGD scheme inaccelerating the training of deep architecture. Better still, the enhancedversion, WASGD+, has been shown to be a significant improvement over its basicversion.

Weighted Aggregating Stochastic Gradient Descent for Parallel Deep Learning