Open AccessGenSynth: a generative synthesis approach to learning generative machines for generate efficient neural networks
Author(s) -
Wong Alexander,
Javad Shafiee Mohammad,
Chwyl Brendan,
Li Francis
Publication year - 2019
Publication title -
electronics letters
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.375
H-Index - 146
eISSN - 1350-911X
pISSN - 0013-5194
DOI - 10.1049/el.2019.1719
Subject(s) - computer science , deep learning , artificial intelligence , artificial neural network , generative grammar , machine learning , generator (circuit theory) , computer engineering , power (physics) , physics , quantum mechanics
The tremendous potential exhibited by deep learning is often offset by architectural and computational complexity, making widespread deployment a challenge for edge scenarios such as mobile and other consumer devices. To tackle this challenge, we explore the following idea: Can we learn generative machines to automatically generate deep neural networks with efficient network architectures? In this study, we introduce the idea of generative synthesis, which is premised on the intricate interplay between a generator‐inquisitor pair that work in tandem to garner insights and learn to generate highly efficient deep neural networks that best satisfies operational requirements. Experimental results for image classification, semantic segmentation, and object detection tasks illustrate the efficacy of generative synthesis (GenSynth) in producing generators that automatically generate highly efficient deep neural networks (which we nickname FermiNets with higher model efficiency and lower computational costs (reaching > 10 × more efficient and fewer multiply‐accumulate operations than several tested state‐of‐the‐art networks), as well as higher energy efficiency (reaching > 4 × improvements in image inferences per joule consumed on a Nvidia Tegra X2 mobile processor). As such, GenSynth can be a powerful, generalised approach for accelerating and improving the building of deep neural networks for on‐device edge scenarios.