Using a Semantic Simulation Framework for Teaching Machine Learning Agents

Autonomous virtual agents that operate in complex IoT environments and apply machine learning algorithms face two fundamental challenges: (i) they usually lack sufficient start-up knowledge and (ii) hence are incapable to adequately adjust their internal knowledge base and decision-making policies during runtime to meet specific user requirements and preferences. This is problematic in Ambient Assisted Living (AAL) and Health-Care (HC) scenarios, since an agent has to expediently operate from the beginning of its lifecycle and adequately address the target users’ needs; without prior user and environmental knowledge, this is not possible. The presented approach addresses these problems by providing a semantic use-case simulation framework that can be tailored to specific AAL and HC use cases. It builds upon a semantic knowledge representation framework to simulate device events and user activities based on semantic task and ambient descriptions. Through such a simulated environment, agents are provided with the ability to learn the best matching actions and to adjust their policies based on generated datasets. We demonstrate the practical applicability of the simulation framework through the evaluation of the chronic kidney disease pathway from the vCare EC project. Thereby, we proof that an agent that uses reinforcement learning (RL) is able to improve its performance during and after the training and thus makes optimal (activity) recommendations to a prospective patient.