Constructing Mathematical And Spatial Reasoning Measures For Engineering Students

Engineering students sometimes encounter difficulties in classes due to their ability to understand and interpret mathematical and visual representations of a problem. This paper describes tools to assess students’ abilities in four different constructs. The two mathematical constructs are: M1. Compare and contrast mathematical operations and M2. Express engineeringand physicsbased principles mathematically. The two spatial-reasoning constructs are: S1. Rotate and transform geometric objects in three-dimensional space and S2. Translate two-dimensional images to three-dimensional images and vice-versa when representing visually engineeringor physics-based principles. Examples are provided for each construct and assessment methods are also presented. Background and Motivation The purpose of this paper is to introduce mathematical and spatial-reasoning constructs that are keys to academic success in engineering. The term, “construct”, is defined as a latent, unobservable trait, such as an ability or skill that directs how students select or generate answers to test items. 1 Several constructs or latent traits have been identified as important in engineering education. The authors illustrate how test items can be designed given various classroom assessment goals (e.g., course examinations, homework assignments) for two sets of constructs that can result in reliable and valid scores. Specifically, two mathematical constructs and two spatial-reasoning constructs are the focus of this paper. The mathematical constructs represent students’ abilities to: (M1) compare and contrast mathematical operations (e.g., differentiation, integration, interpolation); and (M2) express engineeringand physics-based principles mathematically. Likewise, two spatial-reasoning constructs are of interest. These constructs represent students’ strategies to: (S1) rotate and transform geometric objects in three-dimensional space; and (S2) translate two-dimensional images to three-dimensional images and vice versa when representing visually engineeringor physics-based principles (e.g., acceleration, equilibrium, force). Each mathematical and spatial-reasoning measure individually has received attention in the literature because of its importance in defining academic success in engineering. Devon, Engel, and Turner 2 determined that the students’ ability to rotate and transform geometric objects in threedimensional space is predictive of graduation and retention in engineering programs. Similarly, knowing how forces are represented visually in diagrams commonly employed in statics and P ge 15313.2 thermodynamic courses is a skill that successful engineering students have. However, many college students have difficulty understanding how physics-based principles are represented visually. As a result, the types of problems assigned in courses like statics and thermodynamics that utilize these visual representations may be one reason these classes are perceived as challenging 3,4 and are sometimes called stumbling block courses. The challenge students encounter in engineering courses is escalated by the fact that no ability or skill acts in isolation. Research from cognitive psychology 5,6,7 provides ample evidence that constructs must be coordinated or integrated if students are to reach levels of competence or proficiency within their domain. Therefore, in this paper, the researchers advocate for designing classroom measures that represent construct sets required to solve problems effectively in areas of specialization such as statics and thermodynamics. The researchers also introduce psychometric questions to be addressed in the study concerning the reliability and validity of scores for the measures. These questions pertain to both dimensionality (i.e., how many constructs predict the response patterns for any given test) as well as how the scores are assigned for that test. Even for the well-known multiple-choice items, scores can be assigned in a variety of ways. For example, they can be scored dichotomously (i.e., correct versus incorrect) or polytomously (e.g., correct, partially correct, incorrect). Further, some of these multiple-choice items are constructed as testlets. 8 Testlets are groups of items that are dependent on the same stem or sets of tasks that must be solved within one problem space. 9 A reading comprehension passage followed by a set of multiple-choice items is a testlet. Determining the reliability and validity of testlet scores requires psychometric considerations that differ from those needed to analyze the data for a series of multiple-choice items that are independent and do not refer to a common stem or stimulus.. In the following sections, the investigators present items that represent each of the constructs of interest. The importance of these constructs within the coursework for engineering majors is described in the context of programs of study at one large university. Finally, a description of how the study of dimensionality and score assignment can lead to various reliability and validity analysis strategies is provided. Closing sections address statistical considerations and future directions in test and task development to study the academic development of students enrolled in undergraduate engineering programs. Mathematical Test Items: Examples M1 and M2 The use of mathematics in solving and communicating engineering analysis can be an obstacle for some students. In describing the use of mathematics in engineering, we have distinguished between two different constructs, listed above as: (M1) compare and contrast mathematical applications relevant to solving varied problems in engineering; P ge 15313.3 (M2) understand how the engineering quantities (e.g. force, work, power, and flow rate) are described by the mathematical representations (e.g. integration, differentiation, or interpolation) presented in statics, dynamics, thermodynamics, and fluid mechanics. Although these two constructs are similar, we have listed them separately to better define the particular usage of mathematics that a student finds challenging. The following two examples will better define these constructs. Construct (M1) refers to an understanding of the mathematical equations and solution methods without relating it to a physical quantity such as force, pressure, or power. An example of this type of problem is: M1.7. The function y = f(x) is shown on the graph. Circle all statements below that are true: a. 1 2 dy dy dx dx > b. 1 2 dy dy dx dx < c. location 1 is an inflection point