A New Energy Prediction Algorithm for Energy-Harvesting Wireless Sensor Networks With Q-Learning

Traditional wireless sensor networks (WSNs) face the problem of a limited-energy source, typically batteries, resulting in the need for careful and effective utilization of the energy source. However, inevitable energy depletion will eventually disturb the operation of a WSN. Energy harvesting (EH) technology is acquiring particular interest, because it has the potential to provide a continuous energy supply in battery-powered WSNs. Solar energy is the most effective environmental energy for EH-WSNs because of its high energy intensity, which comes from a non-controllable source. Therefore, the prediction of future energy availability is a critical issue, as the amount of the harvestable energy may vary over time. In this paper, a novel solar energy prediction algorithm with Q-learning, called Q-learning-based solar energy prediction (QL-SEP), is proposed. Q-learning is an effective way of predicting future actions based on past observations. The distinctive feature of QL-SEP is that not only past days' observations but also the current weather conditions are considered for prediction. The performance of QL-SEP is simulated in this paper using real-world measurements obtained from a solar panel in comparison with the state-of-art approaches.