Open AccessFeature selection in deterministic policy gradient
Author(s) -
Li Luntong,
Li Dazi,
Song Tianheng
Publication year - 2020
Publication title -
the journal of engineering
Language(s) - English
Resource type - Journals
ISSN - 2051-3305
DOI - 10.1049/joe.2019.1193
Subject(s) - reinforcement learning , computer science , dependency (uml) , task (project management) , feature (linguistics) , mathematical optimization , range (aeronautics) , selection (genetic algorithm) , algorithm , function (biology) , gradient method , artificial intelligence , mathematics , linguistics , philosophy , materials science , management , evolutionary biology , economics , composite material , biology
The authors consider the task of learning control problem in reinforcement learning (RL) with continuous action space. Policy gradient, and in particular the determinist policy gradient (DPG) algorithm, provides a method for solving learning control problem with continuous action space. However, when the RL task is complex enough so that tuning of the function approximation is necessary, hand‐tuning for the features is infeasible. In order to solve this problem, the authors extend DPG algorithm by adding an approximate‐linear‐dependency‐based sparsification procedure, which makes DPG algorithm to automatically select the useful and sparse features. As far as the authors know, this is the first time to consider the feature selection problem in DPG. Simulation results illustrate that (i) the proposed algorithm can find the optimal solution of the continuous version of mountain car problem; (ii) the proposed algorithm achieves good performance over a large range of the approximate linear dependency threshold parameter settings.