Improving Writing In Civil And Environmental Engineering Courses Using Claqwa, An Online Tool For Writing Improvement

A required ABET student outcome of engineering programs is “communication” which, according to the American Society of Civil Engineers BOK means that a student can “Plan, compose, and integrate the verbal, written, virtual, and graphical communication of a project to technical and non-technical audiences.” The Civil and Environmental Engineering program at the University of South Florida, addresses this outcome over a student’s undergraduate career, however, tools for student improvement are typically not directly linked with the course syllabus and the actual assessment of skills. The Cognitive Level and Quality Writing Assessment (CLAQWA) instrument is a computer based assessment and feedback tool designed to improve the writing skills and raise cognitive levels necessary for a given writing assignment. It also allows faculty to assess, diagnose and grade a writing assignment and student peers to provide feedback to each other. CLAQWA provides guided, interactive examples for self instruction on writing improvement and is flexibly designed to emphasize the particular instructor’s grading priorities. CLAQWA was integrated into an upper level Environmental Engineering Systems course where it was used for a student term paper in both Fall 2008, the pilot, and Spring 2009. Working in groups of three (in Fall 2008 17 groups; Spring 2009 – 13 groups), students selected a topic relevant to environmental engineering and worked closely with the two course instructors in outlining the paper’s theme. A writing teaching assistant familiar with CLAQWA was assigned to the class and specially scheduled training sessions were provided to train students on the use of CLAQWA. Each group’s first paper draft received at least three peer reviews, all of which were compared for the purposes of this paper to determine any common issues amongst the class. The faculty and teaching assistant compared their own grading using CLAQWA in an effort to calibrate the process for the grading of the final term paper. For the Spring 2009 offering of the class the entire CLAQWA process was undertaken without the online component. This was done to determine whether the tool can be successful by use of traditional teaching techniques i.e. as a paper based tool. Student surveys were used to gain feedback on whether students felt CLAQWA was useful and helped them to improve their writing skills. CLAQWA provides effective tools that actually improve student writing skills and enables data collection to demonstrate this improvement. Introduction According to Flateby and Fehr 1 , “the inability to communicate effectively in writing seems to be a common deficiency amongst engineers. One can debate over the root cause of this deficiency, but most people agree that strengthening communication skills will increase the effectiveness of the engineer.” To address this issue amongst engineering graduates, ABET now requires BS graduates to be able to effectively communicate their ideas verbally and through writing 1 . This concept has been incorporated into the Civil Engineering Body of Knowledge (BOK) 2 . P ge 15705.2 Outcome 16 of the Civil Engineering BOK 2 states that undergraduates should be able to organize and deliver effective verbal, written, virtual, and graphical communications by the end of their bachelor’s degree. The Civil and Environmental Engineering program at the University of South Florida, addresses this outcome in different classes over a student’s undergraduate career, however, tools for student improvement are not directly linked with the course syllabus and the actual assessment of skills, e.g. writing, is difficult. For written communication, especially in the form of technical papers, students are directed to college or university wide help centers which many times means meeting with a writing tutor. At the University of South Florida this question is being addressed through a tool to assess written communication skills based on an assignment given in an Environmental Engineering Systems class that is required of all Civil and Environmental Engineering undergraduates. The departmental vision is to institute this tool in all writing assessments and possibly expanding to first year courses that are not housed in the department. These training sessions included both classroom type settings in both Fall 2008 and Spring 2009, and a virtual training environment using Elluminate TM software for the Fall 2008 offering. Using a cross-disciplinary example, students examined sixteen different writing and thinking elements that fell into the following five categories: assignment parameters, structural integrity, reasoning and development of ideas, language, and grammar and mechanics. A writing teaching assistant familiar with CLAQWA was assigned to the class and specially scheduled training sessions were provided to train students on the use of CLAQWA. About ENV4001 Environmental Systems Engineering This course is a senior level mandatory course for all students attempting a BS Civil & Environmental Engineering degree. Most students take this course in their senior year often in the semester of graduation. The writing intensive term paper provide a crashcourse in written communication before heading out into the world of work. This was deemed essential since most of the students would have had their only communication skills acquired in their freshman year during Gordon Rule 6A Communications and General Education English composition requirements. The term papers are done in groups of three to further foster the development of non-written communications skills, which is also important according to Outcome 16 of the Civil Engineering BOK 2 . The enrollment in this class is usually above 65 students. The term project is worth 15% of each student’s final course grade and it is broken into four parts. The first part is worth 10% of the project grade and is based on a group selection of topic and delivery of a list of 25 journal references for the paper in APA format. In about a month from then the rough drafts are due from all groups which are mainly graded on effort in contributing a possible 20% to the project grade. Each group has their rough drafts evaluated by at least three other groups constituting a peer review while the drafts are graded by the instructor and teaching assistant to give the students more insights into where their effort should be focused in the production of their final draft. Peers have two weeks to complete their reviews and their own review is worth 20% of the grade. Final drafts are due at the end of the semester and is worth 50% of the P ge 15705.3 project grade. It is evaluated by only the instructing faculty and the teaching assistant. The process implements an iterative review and assessment process which is considered to be instrumental in making assessment of writing successful 3,4 . Held 3 and Yin 4 corroborate that student participation in the assessment process is paramount to meeting the written communication objectives of any assignment. Hence, the incorporation or peer reviews into the term paper exercise; an experience that is not common among most engineering curricula. Background on CLAQWA Designed to help university instructors assess, diagnose, and grade student writing, Cognitive Level and Quality Writing Assessment (CLAQWA) conveys the writing skills and cognitive level necessary for a given writing assignment. CLAQWA is useful for the full range of writing and thinking assessment activities (from course to the institution), and helps guide the student peer review process. CLAQWA provides guided, interactive examples for self instruction on writing improvement and is flexibly designed to emphasize the particular instructor’s grading priorities. The root of CLAQWA stems from assessment at a defined cognitive level based on Bloom’s taxonomy of cognition 5 . Cognitive levels range from 1-5 hierarchically and are described below: Level 1: Knowledge. Accurately recalls or describes, identifies information which was presented in class or reading. Involves memorization. Level 2: Comprehension. Translates or rephrases known words, interprets or explains in a way that demonstrates understanding of the material. Level 3: Application. Uses what is learned in the assignment or in class. Level 4: Analysis, Synthesis. Evaluation. Makes a judgment of a work or plan based upon a given constructed set of specific criteria, not opinion. Organizes or reorganizes ideas or combines elements to make a whole. Distinguishes between fact and fiction. Compares and contrasts or deduces. Identifies relationships of parts to a whole. Level 5: Level 4 with superior, effective presentation style. Proper placement of material for greatest impact. Higher cognitive levels may include characteristics of lower levels and so half and quarter level points may be allocated. According to the creators of the CLAQWA system, any instructor that uses CLAQWA can determine prior the expected cognitive level of students. This needs to be selected depending on the complexity of the subject matter as well as the academic level of the class. Nevertheless, whatever the level, the assessment of writing with CLAQWA takes place under five distinct categories (i.e. assignment parameters, structural integrity, reasoning and development of ideas, language, and grammar and mechanics). Each category has particular analytical elements or traits for assessment (see Table 1). Each trait or element can be assessed at the five levels. This means that the overall level of operation can be assessed based on the number of problematic traits contained in a Page 15705.4 piece of writing. A generalized example of assessing a trait at the five cognitive levels is shown in Table 2. Table 1: CLAQWA analytical scale traits