Effective and Adoptable Metacognitive Tools

This paper, an evidence-based practice paper, describes two metacognitive teaching tools that were tested in classroom environments for their efficacy and ease of adoption. Ease of adoption refers to the subjective measures of (a) ease of implementation, (b) minimal displacement of class time, and (c) no requirement for a change in pedagogy. The two tools promote metacognition, which has an extensive evidence base for promoting learning in a wide range of subjects and across grade levels. The first tool tested is the exam wrapper. Examinations or tests provide a measure of student performance and offer feedback to students of their learning and the need to perhaps adjust their learning strategies. Many students, however, focus on the grade rather than the comments or corrections. Even when students make the effort to look at the mistakes, they often miss the opportunity to reflect on the deeper root causes and instead focus on the superficial error. Without deep reflection students may not gain the awareness that they need to confront misconceptions or make strategic changes in their learning. The second tool tested is the assignment correction, a variant of exam wrappers but used for more frequently occurring activities such as homeworks or quizzes. The idea is that, perhaps, improving metacognition requires frequent practice. If the exam wrapper could be adapted for use with graded assignments, it would provide such practice. To remain a tool that is easy to use, however, assignment corrections must be briefer than an exam wrapper, easy to assign, collect and score, and continue to consume little to no class time for completion. The two tools were tested in various engineering courses and mixed results were found: While both tools were adoptable, only the exam wrapper appeared to be efficacious in this study. Introduction Metacognition, which has as its simplest definition thinking about one’s thinking, is the modern term used to capture the processes that learners use to reflect upon and take actions to improve their learning. The psychologist John Flavell introduced the term in the 1970’s while advancing research on the topic, but ideas about the usefulness of reflection in improving learning began much earlier, starting with John Dewey. Both Piaget and Vygotsky – both recognized widely for their theories in education – wrote of the role of metacognition in the cognitive development of children. Finally, in the much-heralded 2000 revision of the taxonomy of educational objectives, originally put forth by Benjamin Bloom and colleagues in 1956, metacognition was included as a fourth dimension of knowledge (alongside factual, procedural and conceptual knowledge). Despite this rich history and a wealth of research supporting its effectiveness in helping students learn, metacognition is not a teaching tool in wide use by engineering educators nor is it intentionally taught as a learning strategy to students. This paper describes the implementation and testing of two metacognitive tools for their efficacy in improving student learning and performance. Background Much effort and cost have been devoted to improving STEM learning over the past few decades. The majority of such projects has been focused on improving the learning of subject content and student success through, for example, improving classroom teaching [e.g., refs. ], curricular reform [e.g., ref.], or providing more and better design experiences [e.g., refs. ]. Little attention, however, has been given to the incorporation of reflection to promote learning and the experience of learning. The psychologist Ellen J. Langer exposes the pitfall of a lack of reflection: Learning without being mindful sometimes leads to rote exercising that could build bad habits and prevent learners from seeing how to apply knowledge learned in one context to other, very different ones. As Lang points out, “[s]tudents who have frequent opportunities to pause and reflect on what they’re practicing will develop deeper understanding and more transferable skills.” Mindfulness in learning, when combined with self-generated control of that learning, is synonymous with the modern psychological term metacognition. Pintrich defines metacognition as the awareness, knowledge and control of one’s thinking while learning. Experts across widely varied fields, including academic disciplines, chess and various sports, actively practice metacognition, demonstrating perhaps that metacognition is a trait important not only to learning and expertise, but also to the application of that knowledge. Moreover, adept problem-solvers possess strong metacognitive skills. They are aware of flaws or gaps in their knowledge, can readily describe their thought processes, and subsequently alter their solution strategies toward achieving a desired outcome. Novices and students often lack these skills or do not know to apply them when needed. As described by Pintrich, Flavell suggested a three-part framework for metacognitive knowledge: strategy, task and person. Strategic knowledge consists of knowledge of strategies for thinking, learning and problem solving. They are generally applicable to all or most academic disciplines, as opposed to being specific to one discipline. These may include, for example, rehearsal (e.g., repeated practice) or elaboration (e.g., providing detailed explanation or paraphrasing of a solution) or problem solving (e.g., working backward from a goal). Task knowledge, or knowledge about cognitive tasks, involves recognizing that various tasks can differ in difficulty and may therefore require different strategies. For example, a recognition task (recognizing a word in an unfamiliar language) is easier than a recall task (reproducing that word from memory). Learners must be aware of why and when to apply different skills in such situations. Finally, personor self-knowledge describes awareness of one’s strengths and weaknesses. If a learner prefers multiple-choice rather than short-answer tests, but is faced with the latter, she would recognize this challenge and alter her study strategies in preparation (e.g., start studying earlier and outline the chapters for review). Self-knowledge also includes aspects of motivation for learning. For example, is the learner pursuing the learning through an intrinsic (“this is interesting”) or extrinsic (“I want a good grade”) orientation, and what about the learner’s self-efficacy? Research over the past 40 years has conclusively demonstrated the effectiveness of learning accompanied by metacognition [see, for example: refs. 18, 19, ]. Although few of these studies have been based in engineering or science, the evidence seems clearly extendable to these learning environments. As Pintrich states, “Because metacognitive knowledge in general is positively linked to student learning, explicitly teaching metacognitive knowledge to facilitate its development is needed.” Furthermore, Bransford et al., in their synthesis of research on learning over the past few decades, declares the effectiveness of a metacognitive approach to instruction as one of its three key findings. An Internet search of the term “metacognitive teaching methods” returns several hundred thousand hits. A perusal of the first two pages of this reveals scholarly web sites or articles that include suggestions for various ways that educators could incorporate metacognition into instruction to promote learning. These include short activities such as predicting outcomes, selfquestioning, journaling and critiquing. Longer and more complex activities include using a rubric for self-evaluation, creating mind maps, and reflection writing. Finally, whole-class or extensive metacognitive teaching methods include student-developed tests or grading rubrics, self-assessment of assignment, and creating concept maps. Further study of the search results reveals that many metacognitive teaching methods have been the subject of scholarly study, including, for example, rubrics, self-assessment, student-written exam and concept map. These studies all demonstrate positive outcomes for student learning, attitude, or both. Given the overwhelming evidence of effectiveness, the question is why are metacognitive teaching methods not widely adopted in science and engineering disciplines? Given this nation’s need for more and better trained engineers and scientists 27, 28, 29, 30 and the greater attention given to improving undergraduate STEM teaching over the past few decades, it seems that more faculty should be incorporating metacognitive approaches in instruction. Susan Ambrose, the educator and co-author of How Learning Works, directly addressed this point to the engineering education community, saying “So, yes, students learn by doing, but only when they have time to reflect on what they are doing – the two go hand in hand. Why, then, don’t engineering curricula provide constant structured opportunities and time to ensure that continual reflection takes place?” The reason for the low use of metacognitive instruction is likely to be low awareness and the challenge of adoption. Metacognition is a relatively new construct and thus has low awareness among engineering and science faculty, whose graduate education included little or no educational theory or training. Even if faculty members were made aware of the importance and value of metacognitive instruction, we argue that its adoption would remain low. Many wellknown instructional approaches with overwhelming research supporting their effectiveness have yet to be adopted in engineering and science teaching . The most frequently cited reasons for non-adoption of these innovations include the displacement of course content, the fear of student resistance, and the alteration of the instructor’s preferred pedagogy. It is clear that overwhelming evidence is insufficient to convince faculty to adopt a new teaching method. It must also overcome the cited barri