English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

As the need for edge computing grows, many modern consumer devices nowcontain edge machine learning (ML) accelerators that can compute a wide rangeof neural network (NN) models while still fitting within tight resourceconstraints. We analyze a commercial Edge TPU using 24 Google edge NN models(including CNNs, LSTMs, transducers, and RCNNs), and find that the acceleratorsuffers from three shortcomings, in terms of computational throughput, energyefficiency, and memory access handling. We comprehensively study thecharacteristics of each NN layer in all of the Google edge models, and findthat these shortcomings arise from the one-size-fits-all approach of theaccelerator, as there is a high amount of heterogeneity in key layercharacteristics both across different models and across different layers in thesame model.  We propose a new acceleration framework called Mensa. Mensa incorporatesmultiple heterogeneous ML edge accelerators (including both on-chip andnear-data accelerators), each of which caters to the characteristics of aparticular subset of models. At runtime, Mensa schedules each layer to run onthe best-suited accelerator, accounting for both efficiency and inter-layerdependencies. As we analyze the Google edge NN models, we discover that all ofthe layers naturally group into a small number of clusters, which allows us todesign an efficient implementation of Mensa for these models with only threespecialized accelerators. Averaged across all 24 Google edge models, Mensaimproves energy efficiency and throughput by 3.0x and 3.1x over the Edge TPU,and by 2.4x and 4.3x over Eyeriss v2, a state-of-the-art accelerator.

Mitigating Edge Machine Learning Inference Bottlenecks: An Empirical Study on Accelerating Google Edge Models