Development Of Performance Criteria For Assessing Program Outcomes In Engineering, Engineering Technology & Computer Science Programs

This paper presents the development and the use of performance criteria that could be used for detailed assessment of specific students’ performance in the program outcomes listed for Engineering programs (EAC Criterion 3, a-k outcomes), Engineering Technology programs (TAC Criterion 2, a-k outcomes) and Computer Science programs (CAC criterion 1, a-i outcomes). Performance criteria have been used to break down each program outcome into concrete measurable actions students are expected to be able to perform to demonstrate proficiency in the outcome. For each of the listed outcomes for the ABET Accreditation bodies, detailed performance criteria are presented in this paper. Suggestions on how the performance criteria can be used in a program are described in detail to allow selective adoption of the performance criteria for different programs and for different courses. The methodology for defining and using the performance criteria enables faculty to (1) fully understand the outcomes, (2) understand a range of performance criteria that need to be measured for each outcome, and (3) remove any ambiguity in the interpretation of the outcomes. In addition, it makes it possible to identify the critical skill-sets to measure for each outcome and makes assessment meaningful to the various programs. Introduction In the advent of EC 2000, Engineering, Engineering Technology and Computer Science programs have grappled with methods for assessing the ABET outcomes, especially those skills which are not taught in the traditional programs. Even though several assessment methods have been published in the literature (1, 2, 3, 4) for assessing outcomes, there is still a need to establish concrete performance criteria for the outcomes to make the interpretation of assessment results meaningful. Richard Felder and Rebecca Brent 5 have provided useful references that provide additional suggestions for defining performance criteria for the outcomes discussed in this paper. Performance criteria are specific measurable statements that indicate the actions or competencies students should be able to perform or possess at the end of the measurement period. Defining performance criteria for each program outcome is important because it (1) delineates specific statements that identify concrete measurable actions students should be able to perform to meet the outcome, (2) clearly states what needs to be measured, (3) provides common understanding among the faculty on the interpretation of an outcome, thereby removing any ambiguity in the interpretation of an outcome, (4) informs students of the expectations from the outcome, (5) provides focus on the type of data to be collected, (6) provides validity to the assessment results, (7) clearly identifies specific problem areas to be addressed as a result of the assessment process. To ensure that the performance criteria developed can be used by different programs, they were developed based on the program outcomes for Engineering (ABET Criterion 3, a-k outcomes), Engineering Technology (TAC Criterion 2, a-k outcomes) and Computer Science (CAC criterion 1, a-i outcomes). The program outcomes from the three ABET Accreditation Commissions were analyzed and grouped based on similarities. The performance criteria were developed for each similar group of outcomes. In the description provided in this paper, the outcomes were put into twelve groups consisting of outcome group 1 to outcome group 12. Performance Criteria for Outcome Group 1 Outcome group 1 consists of (1) EAC Criterion 3, outcome a, “ability to apply the knowledge of mathematics, science, and engineering”, (2) TAC Criterion 3, outcome b, “ability to apply current knowledge and adapt to emerging applications of mathematics, science, engineering and technology”, and (3) CAC criterion 3, outcome a, “ability to apply knowledge of computing and mathematics appropriate to the discipline”. For this outcome group, the performance criteria were based on knowledge and application of mathematics, science, and pre-requisite courses. The six performance criteria developed for use in measuring this outcome are based on determining whether: 1. Students have the knowledge and the ability to apply basic mathematics involving algebra, geometry, and trigonometry 2. Students have the knowledge and the ability to apply intermediate mathematics involving differential calculus, integral calculus, and probability & statistics 3. Students have the knowledge and the ability to apply advanced mathematics including complex analysis, numerical analysis, Fourier series, Laplace transforms, and linear algebra 4. Students demonstrate the knowledge and the ability to apply chemistry 5. Students demonstrate the knowledge and the ability to apply physics 6. Students have the knowledge and the ability to apply named prerequisite courses By measuring the performance of students in these six areas, it is possible to determine, in finer detail, specific areas that may need improvement. Faculty and students are also made aware, and a common ground is created for assessing this outcome. Performance Criteria for Outcome Group 2 Outcome group 2 consists of (1) EAC Criterion 3, outcome b, “Ability to design and conduct experiments, as well as to analyze and interpret data”, and (2) TAC Criterion 2, outcome c, “ an ability to conduct, analyze and interpret experiments and apply experimental results to improve processes”. Four performance criteria were developed for this outcome. For each performance criterion, detailed guides for assessing the performance criterion were also provided. The six performance criteria developed for this outcome, are based on determining whether: 1. Students have the ability to design experiments This performance criteria is assessed by determining whether given a phenomenon to be experimentally investigated, the students are able to (i) identify the measurable parameters of the phenomenon, (ii) identify different methods for measuring the phenomenon, (iii) identify and select or design appropriate equipment or components for measuring the parameters, (iv) provide steps for setting up and conducting the experiment, and (v) identify the relationship between the phenomenon and the measured parameters. 2. Students have the ability to conduct experiments To assess this performance criteria, it is necessary to determine if students are able to (i) demonstrate general lab safety, (ii) follow experimental procedures for the experiment , while maintaining all safety precautions, (iii) demonstrate knowledge of how equipment functions and their limitations, (iv) complete pre-lab assignment before coming to the lab when required, and (v) collect and record data using appropriate units of measurement and identify the dependent and independent variables in the experiment 3. Students are able to analyze experimental data This performance criteria is assessed by students’ ability to (i) Analyze the data to generate the required parameters using appropriate units and significant figures, and (ii) use statistical analysis as needed. 4. Students are able to interpret data To measure this outcome, it is necessary to determine students’ ability to (i) present the data (raw /derived) in tabular or graphical form to meet the objectives and to aid in interpretation, (ii) discuss the raw and derived data/graphs and assess the validity of the results, (iii) demonstrate the ability to relate how experimental result can be used to improve a process, and (iv) draw appropriate or reasonable conclusions. Performance Criteria for Outcome Group 3 Outcome group 3 consists of (1) EAC Criterion 3, outcome c, “an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability”, (2) TAC Criterion 3, outcome d, “ an ability to apply creativity in the design of systems, components or processes appropriate to program educational objectives, and (3) CAC Criterion 3 c, “an ability to design, implement, and evaluate a computer-based system, process, component, or program to meet desired needs. Five performance criteria were developed for this outcome. For each performance criteria, detailed guide for assessing the performance criteria were also provided. The five performance criteria developed for this outcome, are based on 1. Ability to define the problem This performance criterion is assessed by determining if students are able to (i) identify the customer and the needs, (ii) identify and list the design objectives, and (iii) identify the design constraints. 2. Ability to plan the project This performance criterion is also assessed by determining if students are able to (i) define the design strategy and methodology, (ii) identify and break down work into tasks and subtasks, and identify the personnel and deliverables for each, (iii) develop a Gantt chart and critical path analysis for managing the project, (iv) establish major milestones for tracking progress and define performance metrics to measure success. 3. Ability to conduct a review of the literature This performance criterion determines the extent to which students are able to (i) identify the types of information needed for a complete understanding of all aspects of the project (based on tasks described in the project planning), (ii) gather information on relevant fundamentals, theory / concepts, similar existing systems (demonstrate technical competence) and relate them to the design, and (iii) provide the sources in a list of references properly cited in the literature review section and relevant sections of the report. 4. Ability to generate ideas and apply creativity This is assessed by determining ability of students to (i) define functional requirements for design (specific required actions needed to be performed for the design to be achieved), (ii) transform functional requi