Open AccessA BAYESIAN APPROACH BASED ON ACQUISITION FUNCTION FOR OPTIMAL SELECTION OF DEEP LEARNING HYPERPARAMETERS: A CASE STUDY WITH ENERGY MANAGEMENT DATA
Author(s) -
Mohammad Akbar Ali,
Krishneel Prakash,
Hemanshu R. Pota
Publication year - 2020
Publication title -
science proceedings series
Language(s) - English
Resource type - Journals
eISSN - 2663-9467
pISSN - 2663-9459
DOI - 10.31580/sps.v2i1.1232
Subject(s) - hyperparameter , computer science , leverage (statistics) , machine learning , artificial intelligence , bayesian optimization , data mining , model selection , process (computing) , data acquisition , bayesian probability , selection (genetic algorithm) , deep learning , function (biology) , evolutionary biology , biology , operating system
With the recent rollout of smart meters, huge amount of data can be generated on hourly and daily basis. Researchers and industry persons can leverage from this big data to make intelligent decisions via deep learning (DL) algorithms. However, the performance of DL algorithms are heavily dependent on the proper selection of parameters. If the hyperparameters are poorly selected, they usually lead to suboptimal results. Traditional approaches include a manual setting of parameters by trial and error methods which is time consuming and difficult process. In this paper, a Bayesian approach based on acquisition is presented to automatic selection of optimal parameters based on provided data. The acquisition function was established to search for the best parameter from the input space and evaluate the next points based on past observations. The tuning process identifies the best model parameters by iterating the objective function and minimizing the loss for optimizable variables such as learning rate and Hidden layersize. To validate the presented approach, we conducted a case study on real-life energy management datasets while constructing a deep learning model on MATLAB platform. A performance comparison was drawn with random parameters and optimal parameters selected by presented approach. The comparison results illustrate that the presented approach is effective as it brings a notable improvement in the performance of learning algorithm.