Consistency in Assessment of Pre-Engineering Skills

Assessment tools are often used in a predictive way to gauge the overall skills of first-year engineering students as they begin their engineering education. They are also useful in setting interventions in terms of tutorials, as well as providing self-improvement motivation for the students who achieve scores that are not consistent with earlier high school performance. Previous research has demonstrated that the academic averages obtained in high school, may not necessarily reflect the skill level (competency) of the students entering first-year, especially in mathematics. In addition, a longitudinal study over more than ten years has also indicated that the averages from the math advisory and engineering assessment (Force Concept Inventory) exams did not show a statistically significant decline during that time period. In this study, both the math and engineering assessment results were further analyzed on a per question basis to determine whether or not there were any observable trends in the student responses. The results for math assessment exams, taken over thirteen years, indicated that the average performance on each question every year is statistically very consistent. The questions that the majority of the students got right each year, and those that the majority got wrong each year showed very little variation in the standard deviation (typically < 5%), which was used as the measure in variability of the mean. The results were further analyzed by categorizing the questions according to three classifications: algebra, trigonometry and geometry. Typically, the questions with the best overall performance were simple algebra questions, and the questions with the worst overall performance involved trigonometric concepts. Moreover, as the complexity of the algebra questions increased, the success rate on those questions diminished as expected. Both assessment exams were time limited and students were not allowed to use calculators. In the high school curriculum in our region, students use calculators regularly in their high school math courses. As a result, their inherent competency in trigonometric functions is lacking, as the average scores (typically less than 30%) on these questions would indicate. Engineering assessment (Force Concept Inventory) exam results collected over a slightly shorter duration (six years) were also analyzed. The same trends in student responses were observed, but in this case the results were somewhat less striking than the results obtained from the math assessment. It is clear, however, that there is a consistency in the success rate for individual exam questions that test both math and engineering concepts. These results support the anecdotal contention that students collectively have competency in certain areas (algebra) but lack competency in others (trigonometry). It further demonstrates that students often come into firstyear engineering with common misconceptions and common math deficiencies. The results from this study are useful from several perspectives. They can provide a focus for interventions that might address both competency and misconceptions. Secondly, the consistency and repeatability of this data may provide an impetus to work with K-12 educators to address these issues before the students reach university. The consistency of this data also implies that pre-engineering skills are somewhat predictable from year to year. P ge 24315.2