Applying Natural Language Processing Techniques to an Assessment of Student Conceptual Understanding

This work-in-progress briefly surveys a selection of open-source Natural Language Processing (NLP) tools and investigates their utility to the qualitative researcher. These NLP tools are widely used in the field of lexical analysis, which is concerned with automating the generation of useful information from human language using a variety of machine processes. Recent research shows that the statistical analysis of software recognized linguistic features can benchmark certain mental processes, such as cognitive load. This investigation generates those linguistic indicators using transcripts from a multi-year, interview based study and compares them to a qualitative analysis of a subject’s conceptual understanding of various engineering topics. Our intermediary findings indicate a correlation between changes in the linguistic indicators introduced in this paper and a qualitatively coded analysis of conceptual understanding over time. Future work will involve increasing the breadth of the dataset to further establish the fidelity of this approach and expand on the premise of using automatically generated linguistic indicators to aid the qualitative researcher.