Data Analysis for Evaluation on Course Design and Improvement of “Cyberethics” Moodle Online Courses

This study evaluates our security awareness education online course on GakuNinMoodle, which is a Shibboleth-federated Moodle platform used in Japan, via an analysis of learning data from more than 20,000 users in the 2013–2015 academic years. We develop a customized data collection method to aggregate the duration, completion status, and score of each learning object each year, which we use to evaluate changes in course design for the 2015 academic year. We further use text mining of user comments from the Moodle assignment modules to extract user opinions. We found an improvement in the completion rate and final test score obtained in the 2015 academic year compared with 2013–2014. We also found that users became polarized between those who allotted sufficient time for the course and those who did not. Additionally, we obtained user complaints about the duration required to complete the course and problems with the user interface of the Moodle SCORM module. This data analysis will be useful because Moodle is a global-standard learning management system. Further work is underway to analyze learning data and feedback in near-real time.