Premium
Automated assessment of learners' understanding in complex dynamic systems
Author(s) -
Kopainsky Birgit,
PirnayDummer Pablo,
Alessi Stephen M.
Publication year - 2012
Publication title -
system dynamics review
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.491
H-Index - 57
eISSN - 1099-1727
pISSN - 0883-7066
DOI - 10.1002/sdr.1467
Subject(s) - computer science , coding (social sciences) , system dynamics , artificial intelligence , quality (philosophy) , machine learning , measure (data warehouse) , human–computer interaction , data science , data mining , philosophy , statistics , mathematics , epistemology
Research on learning via system‐dynamics based learning environments depends on accurate measurement of learning. Most such research considers at least two aspects of learning: participants' understanding of the models and problems; and participants' performance in the environment, i.e. quality of decision making. Understanding is much more difficult to measure than performance. Measurement of understanding is often done by eliciting verbal protocols from participants. Coding and analysis of verbal protocols are subjective and time consuming. To facilitate measurement and analysis of understanding via verbal protocols, we investigate the utility of a software application which performs such analysis automatically. We assess this automated analysis methodology using data from a system‐dynamics based learning environment. The results from the automated analysis are comparable to the results from a manual analysis when we analyze how participants' understanding compares to experts and how it changes over time. Additionally, the automated analysis facilitates investigating relationships between understanding and performance. Copyright © 2012 System Dynamics Society.