Using Learning Analytics to Characterize Student Experimentation Strategies in the Context of Engineering Design

Engineering design is a complex process. The design process cannot be assessed based solely on a product or as a simple test because there is no single perfect design for a problem. An important design strategy is the conduction of experiments. Informed designers carry out experiments and use their outcomes to inform their next steps. On the other hand, beginning designers do little or no experiments, and the few experiments they do involve confounding variables. These behaviours that differentiate beginning and informed designers are not easy to assess in educational settings because they occur throughout the design process. This paper proposes and evaluates a model to analyze student interactions with a CAD tool in order to identify and characterize the different strategies students use to conduct experiments. A two-fold study is carried out to validate the model. The first phase uses the clickstream data of 51 middle school students working on a design project to create a net-zero energy house. The analysis of clickstream data is compared to a qualitative analysis of an open-ended posttest. The second phase correlates the number of experiments students did to the student prototype quality. The results suggest that the proposed model can be used to identify, characterize, and assess student strategies to conduct experiments.