Open AccessOptimised Q‐learning for WiFi offloading in dense cellular networks
Author(s) -
Fakhfakh Emna,
Hamouda Soumaya
Publication year - 2017
Publication title -
iet communications
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.355
H-Index - 62
eISSN - 1751-8636
pISSN - 1751-8628
DOI - 10.1049/iet-com.2017.0213
Subject(s) - computer science , reinforcement learning , handover , cellular network , computer network , q learning , base station , interference (communication) , markov decision process , constraint (computer aided design) , quality of service , channel (broadcasting) , real time computing , distributed computing , artificial intelligence , markov process , mechanical engineering , statistics , mathematics , engineering
WiFi traffic offloading is becoming especially appealing because of the upcoming ultra‐dense cellular networks. However, WiFi offloading decision as well as WiFi‐Access Points (W‐AP) selection should be carefully studied in order not to affect the offloaded users’ experience. Here, a new reinforcement‐learning framework is presented. The authors propose a distributed Q‐learning algorithm in which each cellular user learns about his local environment and selects the best base station (macro‐BS or W‐AP) after reaching convergence. They introduce a new reward parameter which takes into account the load of each detected W‐AP, the duration of the vertical handover, the offered gain, as well as the achieved signal‐to‐interference‐plus‐noise ratio. With the Q‐learning scheme, each user decides to join the WiFi offloading or not, depending on the received reward from his environment and from his previous learning. In addition, since the AP's load value is very important in the reward parameter, an optimal value of the weight given to the channel load is solved under quality of service constraint. Simulation results showed the effectiveness of the proposed Q‐learning‐based scheme when compared with common WiFi offloading scheme in terms of cellular residence time.