Premium
Validating process variables of sourcing in an assessment of multiple document comprehension
Author(s) -
Hahnel Carolin,
Kroehne Ulf,
Goldhammer Frank,
Schoor Cornelia,
Mahlow Nina,
Artelt Cordula
Publication year - 2019
Publication title -
british journal of educational psychology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.557
H-Index - 95
eISSN - 2044-8279
pISSN - 0007-0998
DOI - 10.1111/bjep.12278
Subject(s) - operationalization , competence (human resources) , psychology , comprehension , sample (material) , process (computing) , test (biology) , computer science , social psychology , paleontology , philosophy , epistemology , biology , programming language , operating system , chemistry , chromatography
Background With digital technologies, competence assessments can provide process data, such as mouse clicks with corresponding timestamps, as additional information about the skills and strategies of test takers. However, in order to use variables generated from process data sensibly for educational purposes, their interpretation needs to be validated with regard to their intended meaning. Aims This study seeks to demonstrate how process data from an assessment of multiple document comprehension can be used to represent sourcing, which summarizes activities for the consideration of the origin and intention of documents. The investigated process variables were created according to theoretical assumptions about sourcing, and systematically tested for differences between persons, units (i.e., documents and items), and properties of the test administration. Sample The sample included 310 German university students (79.4% female), enrolled in several bachelor's or master's programmes of the social sciences and humanities. Methods Regarding the hierarchical data structure, the hypotheses were analysed with generalized linear mixed models ( GLMM ). Results The results mostly revealed expected differences between individuals and units. However, unexpected effects of the administered order of units and documents were detected. Conclusions The study demonstrates the theory‐informed construction of process variables from log‐files and an approach for empirical validation of their interpretation. The results suggest that students apply sourcing for different reasons, but also stress the need of further validation studies and refinements in the operationalization of the indicators investigated.