Premium
Special Considerations When Using Statistical Analysis in Engineering Education Assessment and Evaluation
Author(s) -
Larpkiataworn Siripen,
Muogboh Obinna,
BesterfieldSacre Mary,
Shuman Larry J.,
Wolfe Harvey
Publication year - 2003
Publication title -
journal of engineering education
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 3.896
H-Index - 108
eISSN - 2168-9830
pISSN - 1069-4730
DOI - 10.1002/j.2168-9830.2003.tb00761.x
Subject(s) - bonferroni correction , table (database) , computer science , diagram , statistical analysis , type i and type ii errors , tree (set theory) , tree diagram , machine learning , statistics , data mining , artificial intelligence , mathematics , mathematical analysis , bayesian probability , posterior probability
Two special considerations are discussed which frequently arise when conducting statistical analyses for the assessment and evaluation of engineering education programs. The first concerns the multiple comparison problem and Type I errors, specifically when should the significance level be adjusted and which adjustment procedure is most appropriate? Three scenarios are presented to illustrate three different applications of the classical Bonferroni procedure, one of the most extensively used adjustment procedures. A scenario is also presented for when an adjustment is not necessary. The second consideration is: when evaluating a predictive model should a tree diagram be used as an alternative to a classification table? For example, how does one assess a model's predictions when certain of its “recommendations” are not followed? For this type of case, a classification table may yield incomplete information. The use of a tree diagram to present more information on model performance is discussed.