The Engineering Education Assessment Process - A Signals and Systems Perspective

In this work in progress, a signal model is suggested for the knowledge acquired by engineering students during their study of a specific engineering subject, based on Signals and Systems theory. This model is illustrated using three courses typically taught in modern American-style engineering schools, namely Circuit Analysis, Signals and Systems, and Feedback Control Systems. Additionally, a signals-and-systems-based model is suggested for the assessment process that inputs the acquired knowledge signal, mentioned above, and produces the so-called assessed (engineering) knowledge signal. Based on the largely acknowledged continuous nature of the brain activity, the cumulative nature of the acquired subject-specific engineering knowledge, and the discrete nature of assessment schemes typically administered in engineering schools, it is argued that the acquired engineering knowledge could appropriately be modeled by a continuous-time signal, however the assessed engineering knowledge could be more realistically modeled by a discrete-time signal. As such, the assessment process could then be modeled as a sampling process, where samples of the acquired knowledge signal are captured at various periods of time for the purpose of reconstructing a good image of the student acquired knowledge. It is reasoned that the perceived level of student learning does depend to a large extent on the successful reconstruction of the (continuous) acquired knowledge signal from the (discrete) assessed knowledge signal, i.e. in order for the evaluation of acquired knowledge to be satisfactory, the reconstruction process needs to be error-free! Towards that end, Nyquist-Shannon’s sampling theorem is used to place a condition on the minimum assessment frequency, in order to avoid aliasing errors in the assessment/sampling process. Based on that theorem, a minimum of two samples (assessments) are necessary in the smallest period of the continuous signal (i.e. the acquired knowledge signal). Accordingly, and based in part on previous works in this domain, a preliminary figure of merit is suggested as a necessary minimum assessment frequency, below which the assessment process, and consequently the validity of perceived learning may be questionable. This work is in line with recent studies by educational experts/psychologists advocating the switch from a 2-or-3-assessment-per-semester assessment scheme to a more frequent assessment scheme. The present work seeks the feedback of the engineering education community regarding the validity and implications of the proposed models and the potential of benefiting engineering schools in their pursuit of quality education by drawing their attention not only to the quality (level) of their learning and assessment activities, but also to the frequency of these activities and to the importance of appropriately weighing each learning and assessment activity, including homework assignments and quizzes. I – Introduction When it comes to improving the effectiveness of engineering education, much research has been published and numerous methods and techniques have been suggested. Active Learning, for example, is a popular learning methodology advocating the initiation of classroom activities, such as group discussions, interactions and/or short quizzes, aiming to improve long-term knowledge retention [1]. Cooperative, Collaborative, and Problem-based Learning (PBL) are other methodologies that have been proposed to improve learning [2], [3], [4]. A more recent approach, the Flipped Classroom, proposes replacing the traditional classroom lecture by a teambased active classroom discussion session, following preparatory work outside the classroom [5]. Most of these suggested methodologies, however, focus on the modes and techniques of knowledge delivery. Other suggested methodologies focus on knowledge assessment and on the implicit relationship between assessment and learning. Scenario-based Assessment and Assessment Based on Learning Outcomes are two of these suggested methodologies [6], [7]. Most of these assessment-related approaches focus on the modes and techniques of assessment, without giving much attention to the frequency of assessment and its relationship to effective learning. Paced Active Learning (PAL) is one variant of Active Learning that looks at the importance of increasing the frequency of assessment, as part of a more comprehensive educational approach incorporating two other techniques, namely Regularly Assessed Performance (RAP) and Computer Assisted Presentation (CAP). However, PAL addresses the issue of assessment frequency in a rather informal and empirical fashion [8]. Other studies have suggested that administering frequent, low-stakes exams has the potential of improving learning effectiveness while reducing dishonest behavior. These studies partly originated in the cognitive psychology field [9], [10]. The present work, however, looks at the issue of assessment frequency from a mathematical/Signals and Systems perspective. Given the measurability and continuity of acquired engineering knowledge, in addition to its cumulative nature, this knowledge is first modeled as a continuous-time signal and the assessment process is modeled as a sampling system, where assessment outcomes are simply seen as discrete samples of the continuous acquired engineering knowledge signal. Because these samples are normally captured with the purpose of reconstructing the acquired engineering knowledge, a minimum sampling/assessment frequency is required to validate the reconstruction process, based on Nyquist-Shannon’s sampling theorem. As such, engineering assessments are seen as processes designed to check whether a certain path has been followed rather than whether a certain point in time has been reached. Towards that end, some of the limitations of individual assessments (points in time) are stated, including breadth and depth limitations, coverage limitations, and circumstantial limitations. Since Nyquist-Shannon’s sampling theorem uses ideal sampling to address the question of sampling frequency, and because ideal sampling is based on the so-called unit impulse function, the widely accepted one-or-two-hour exam is suggested as a practical approximation of the unit impulse function. Under this assumption, it is argued that an adequately weighted homework assignment could also be considered as a practical approximation of the unit impulse function, provided a high ethical standard is adhered to. This brings up the issues of ethics and plagiarism in modern engineering schools, and the need to formally systemize acceptable approximations of the unit impulse function, perhaps by assigning appropriate weights for each of the widely known assessment activities, such as exams, homework assignments, quizzes, and projects, as part of modeling the learning assessment process. Finally, it is suggested that the modeled knowledge assessment/sampling system could be complemented by two other systems, one preceding it, designated as the student learning system, and another one following it, and called the learning evaluation system. More importantly, the knowledge assessment system could be used to close the learning loop of the student, who may use the assessment output to “correct the error” between the desired engineering knowledge signal and his/her acquired engineering knowledge signal. As part of this engineering knowledge modeling exercise, a signal noise could be added to the combined model to account for potentially misleading information received by the student. Towards that end, the engineering education community is kindly solicited to give feedback on a number of issues including: 1. The soundness and validity of the proposed models of the acquired engineering knowledge signal, the assessed knowledge signal, and the learning assessment process. 2. If the models are acceptable, what are the implications of the results suggested in this work on the currently adopted engineering education assessment schemes in various parts of the world, in terms of evaluation accuracy and validity? 3. Given the increasing engineering educators’ workloads and the varying scales of student plagiarism in contemporary engineering schools, would it be realistic to switch from an assessment scheme consisting of 2-to-3 major assessments per semester to a more frequent assessment scheme, reaching one formal exam per week as suggested in this work? And would it be possible to rely on the rapidly emerging online learning management systems to facilitate this task? II – Modeling Acquired Engineering Knowledge Given the nature and characteristics of engineering knowledge disseminated in modern engineering schools, and the way in which the human brain learns and retains information, it is hereby suggested that, despite being abstract rather than physical, engineering knowledge could be modeled by a continuous-time (or simply continuous) signal [11]. This model is largely based on the following three properties: 1. Engineering Knowledge is Measurable In an engineering education environment, knowledge measurement is one of the most commonly performed tasks. A knowledge measurement is performed every time a test, quiz, or homework assignment is graded, a project is rated, a presentation is evaluated, or a GPA is calculated! This grading/evaluation process is actually a way of quantifiably measuring the level of acquisition of specialized engineering knowledge at a certain point in time, in comparison with the desired knowledge level. As stated in Section IV, the smaller the “error” between the acquired and the desired knowledge, the higher is the grade. Since, by definition, a signal is a model of a measurable variable, we can simply state that specialized engineering knowledge could be modeled by a signal. It could be argued, however, that knowledge measurement may differ from one engineering school/institution to another or eve