Genetic Network Programming-Sarsa with Subroutines for Trading Rules on Stock Markets

The purpose of this paper is to propose a new approach to generate effective subroutines, which are automatically discovered by the GNP-Sarsa programs combining evolution and reinforcement learning. We name it asGNP-Sarsa with subroutines (GNP sb -Sarsa) and apply it to the trading rules on stock markets. In the proposal method, GNP sb -Sarsa offers an alternative population, where individuals are represented by subroutines. Each main program of GNP sb -Sarsa can refer to an individual in the subroutine population, after adding a new kind of node, namely subroutine node, in the graph network structure of GNP-Sarsa. GNP sb -Sarsa containing a main program and a subroutine evolves by natural selection and genetic operations, where the gene of GNP sb -Sarsa is the pair of the main GNP and its subroutine. That is, the genetic operations on GNP sb -Sarsa are constrained by the gene structure on which they can operate. In the simulations, the stock prices of different brands from 2001 to 2004 are used to test the effectiveness of the GNP sb -Sarsa. The results show that the proposed approach can provide reasonable opportunities for evolving complex solutions.