Premium
AUTOMATIC COMPLEXITY REDUCTION IN REINFORCEMENT LEARNING
Author(s) -
Chiu ChungCheng,
Soo VonWun
Publication year - 2010
Publication title -
computational intelligence
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.353
H-Index - 52
eISSN - 1467-8640
pISSN - 0824-7935
DOI - 10.1111/j.1467-8640.2009.00350.x
Subject(s) - dimensionality reduction , reinforcement learning , curse of dimensionality , computer science , dimension (graph theory) , artificial intelligence , reduction (mathematics) , graph , relevance (law) , representation (politics) , theoretical computer science , machine learning , algorithm , mathematics , politics , geometry , political science , pure mathematics , law
High dimensionality of state representation is a major limitation for scale‐up in reinforcement learning (RL). This work derives the knowledge of complexity reduction from partial solutions and provides algorithms for automated dimension reduction in RL. We propose the cascading decomposition algorithm based on the spectral analysis on a normalized graph Laplacian to decompose a problem into several subproblems and then conduct parameter relevance analysis on each subproblem to perform dynamic state abstraction. The elimination of irrelevant parameters projects the original state space into the one with lower dimension in which some subtasks are projected onto the same shared subtasks. The framework could identify irrelevant parameters based on performed action sequences and thus relieve the problem of high dimensionality in learning process. We evaluate the framework with experiments and show that the dimension reduction approach could indeed make some infeasible problem to become learnable.