Analytics for Action: Assessing effectiveness and impact of data informed interventions on online modules

Investigating effectiveness of learning analytics is a major topic of research, with a recent systematic review finding 689 papers in this field (Larrabee Sonderlund et al., 2019). Few of these (11 out of 689) highlight the potential of interventions based on learning analytics. The Open University UK (OU) is one of few institutions to systematically develop and implement a learning analytics framework at scale. This paper reviews the impact of one part of this framework - the Analytics for Action (A4A) process, focusing on the 2017-18 academic year and reviewing both feedback from module teams and interventions coming out of the process. The A4A process includes hands-on training for staff, followed by data support meetings with educators when the course is live to students. The aim being to help educators with making informed, evidence-based interventions to aid student retention and engagement.  Findings from this study indicate that participants are satisfied with the training and that the data support meetings are helping in providing new perspectives on the data. The scope and nature of actions taken by module teams varies widely, ranging from no intervention at all to interventions spanning over multiple presentations. In some cases, measuring the impact of the actions taken will require data analysis from further presentations. The paper also presents findings indicating room for improvement in the follow up of the actions agreed, support given to module teams to implement such actions and final evaluation of impact on student outcomes.