Board # 38 : Work in Progress: Quantification of Learning through Learning Statements and Text Mining

How can instructors leverage assessment instruments in design, build, and test courses to simultaneously improve student outcomes and assess student learning well enough to improve courses for future students? We recognize the need for a framework for shaping and assessing student learning. In that spirit, we contend that in design, build, and test courses students learn when they are required to reflect on their experiences and identify their learning explicitly. Further, we posit that utilization of an assessment instrument, the learning statement (LS), can be used to both enable and assess student learning. In our course, AME4163: Principles of Engineering Design, a senior-level, pre-capstone, engineering design course, students learn by reflecting on doing by writing statements anchored in Kolb’s experiential learning cycle. In Fall 2016 we collected over 11,000 learning statements from over 150 students. To address the challenge of analyzing and gleaning knowledge from the large number of learning statements we resorted to text mining to analyze student-submitted learning statements. We assert that text mining empowers instructors to better assess student learning in design, build, and test courses. Utilizing current algorithms for identifying writing patterns, we form a picture of learning over the course of the semester. Employing this tool, we can quantify student learning through reflection on doing and improve the course for future offerings. We find that student insight is characterized by focusing on the future utility of learning, particularly in areas such as planning a design process and evaluating design concepts. In this paper, we cover the salient features of the course, the learning statements, text mining and initial findings.