Open AccessDeveloping an Algorithm for Fast Performance Estimation of Recurrent Memory Cells
Author(s) -
Sergiu Cosmin Nistor,
Mohammad Jaradat,
Razvan Liviu Nistor
Publication year - 2023
Publication title -
ieee access
Language(s) - English
Resource type - Journals
ISSN - 2169-3536
DOI - 10.1109/access.2023.3322367
Subject(s) - aerospace , bioengineering , communication, networking and broadcast technologies , components, circuits, devices and systems , computing and processing , engineered materials, dielectrics and plasmas , engineering profession , fields, waves and electromagnetics , general topics for engineers , geoscience , nuclear engineering , photonics and electrooptics , power, energy and industry applications , robotics and control systems , signal processing and analysis , transportation
We propose a novel graph-oriented machine learning algorithm which we use for estimating the performance of a recurrent memory cell on a given task. Recurrent neural networks have been successfully used for solving numerous tasks and usually, for each new problem, generic architectures are used. Adapting the architecture could provide superior results, but would be time-consuming if it would not be automated. Neural architecture search algorithms aim at optimizing the architectures for each specific task, but without a fast performance estimation strategy it is difficult to discover high-quality architectures, as evaluating each candidate takes a long period of time. As a case study, we selected the task of sentiment analysis on tweets. Analyzing the sentiments expressed in posts on social networks offers important insights into what are the opinions on different topics and this has applications in numerous domains. We present the architecture of the estimation algorithm, discussing each component. Using this algorithm, we were able to evaluate one million recurrent memory cell architectures and we discovered novel designs that obtain good performances on sentiment analysis. We describe the discovered design that obtains the best performances. We also describe the methodology that we designed, such that it can be applied to other tasks.