Engaging Faculty in Continuous Improvement: The Context of an ABET Accreditation Process

Lessons learned. In this paper we present the lessons learned while implementing three strategies to promote faculty engagement in continuous improvement. These strategies were devised within the continuous improvement process that was established in the School of Engineering at Universidad Icesi. In this paper we will refer to full-time faculty members from two academic departments which serve three undergraduate programs, all of which were accredited by ABET ( Accreditation Board for Engineering and Technology) in 2017. At the beginning of the 2018 academic year, faculty members were asked about their perceptions of the devised strategies; their perceptions are presented as a new ABET assessment cycle is beginning. The contribution of this paper is twofold. First, it discusses the strategies designed by the School of Engineering at Universidad Icesi to effectively engage faculty members in the implementation of a sustainable and continuous improvement process. Second, it presents early results obtained from the implementation of these strategies, including the perceptions of faculty members about these changes. Background and supporting literature This paper presents a work in progress related to the consolidation of a continuous improvement process at the School of Engineering at Universidad Icesi. As presented in [1], the continuous improvement process of an academic program can be explained as a PDCA (Plan–Do–CheckAct) cycle. Similarly, the process of improving faculty teaching competences can be explained by following a PDCA cycle. The outcome-based program accreditation ABET led to the consolidation of this improvement process in the School of Engineering and at the departmental level. Two academic departments participated in this consolidation effort, the Information and Telecommunication Technologies Department (ICT) and the Industrial Engineering Department (IND), which consists of 13 and 12 full-time faculty members, respectively. The faculty member reflection is as important as the students’ reflection that extends over time after the courses have ended [2]. This reflection process is critical to course assessment and improvement in the quality of engineering education. Collaborative reflection has also been found to be useful for improving teaching practices [3], and as such this paper presents an initial step towards that direction. Currently, most reflection efforts at the School of Engineering are done by individual faculty members in isolation, yet moving towards collaborative reflection is a direction we intend to move toward. The Accreditation Board for Engineering and Technology (ABET) states in criterion 5 for accrediting engineering programs that faculty must ensure that the program devotes adequate attention and time to each curriculum component, which is consistent with the outcomes and objectives of the program and the institution [4]. Evidence of student work must be presented during the visit, and a binder for each course containing all evidence collected could be arranged in a display room for the program evaluators to review during their visit. The faculty portfolio described later in this work serves as part of that evidence. Continuous improvement process development and implementation At the departmental level, a continuous improvement process based on the strategies listed below has been proposed in order to engage faculty in continuous improvement. The proposal is based on the premise that after reflecting, a faculty member will improve her or his course and teaching practices. These strategies constitute part of the faculty continuous improvement process shown in Figure 1. Strategy #1 – Faculty Portfolio Strategy #2 – Peer discussion and learning Strategy # 3 – Improvement plan consolidation Figure 1. Faculty continuous improvement process These strategies were formulated during the 2015-2017 ABET accreditation process. To date, the three strategies have not been fully implemented, however a path for improvement has been devised. The strategies are discussed in chronological order of implementation, which is to say that the first strategy was the initial one implemented while the third is still a work in progress. Strategy #1 (Portfolio) Development and implementation The first strategy was to move away from course evaluations to what we call "faculty portfolios". A faculty portfolio is a set of artefacts that support the teaching process utilized by a faculty member, including course syllabi, evaluations, and a self-evaluation known as the reflective memo, among other documents. While course evaluations have traditionally played a role in establishing faculty classroom performance, in this paper we explain how in this new context faculty members have the opportunity not only to consider course evaluations with their portfolio evaluation, but also to reflect on and construct improvement plans based on their own portfolio evaluation results. The portfolio is developed by each faculty member during the semester and is housed in a digital repository managed by a centralized office (the Quality Assurance Office at the School of Engineering). The portfolio was found to have positive impact on faculty engagement since each faculty member has the opportunity to explain how her or his class was planned and delivered. This is a win-win situation since through this process each faculty member collects and analyses the evidence required (curriculum evidence by program, criterion 5) for any international accreditation project such as ABET. To better understand this strategy, the evolution of the portfolio will be explained below. Initial stage – before 2015 An instrument (a form to be filled out) called the faculty self-evaluation was used to record faculty members’ evaluations of the achievement of the objectives proposed in the course syllabus, as well as the effectiveness of the strategies and materials used in achieving them. In addition, the results of course evaluations completed by the students were given to the professor after the completion of the course, sometimes the following semester. The course evaluation is an anonymous, mandatory survey answered by students regarding various aspects of the course, such as instructor performance; the quality of the evaluations, including their alignment with the objectives of the course; and the effectiveness of teaching and learning strategies proposed by the instructor. With this information, the faculty member proposed improvement actions related to her or his course where necessary and may eventually make suggestions regarding the chain of pre-requisites. The instrument worked quite well at the course level, however, it did not have a wider impact or significance at the curricular level due to the fact that the results were not shared with other faculty members of the same curricular block or area. Neither were improvement actions shared, thus in this aspect, faculty efforts were autonomous and isolated. Beginning in 2011, we implemented curricular reform based on the CDIO Syllabus 2.0 [5] [6], which takes the evolution and consolidation of knowledge and skills along the curriculum building blocks into account. At that point, we used the CDIO Syllabus 2.0 (linked to Standard 3) as a resource for curriculum benchmarking. We did not use the complete CDIO framework, which includes twelve Standards focused on program improvement. As a result, the curricular content of the courses of a given block or area began to exhibit increased cohesion, and each area did make a clear compromise in the development of competences. In order to demonstrate this, it became necessary to implement measurement instruments and develop evaluation rubrics. Through these measures, it was then possible to establish the achievement of the learning objectives quantitatively. The instrument subsequently evolved into a reflective memorandum in which the faculty member made an objective reflection of the achievement of the learning objectives and of the teaching strategies employed, which could then be used to justify the levels of performance achieved by the students. The reflective memorandum was not mandatory, and faculty response were low. ABET accreditation 2015-2017 When the School of Engineering began the ABET accreditation process for three of its programs in 2015, the reflective memorandum evolved into a faculty portfolio, an instrument that could now be used not only for course improvement and curricular alignment, but also as objective and verifiable evidence of the professor’s work. It can additionally be used by each faculty member as a reflection about her or his teaching competences. The various programs followed a similar process in terms of documentation, and the academic department most closely related to the program implemented the strategies explained within this paper. This was done in order to take the documentation (in this case course evidence) a step further, using it as the basis for faculty reflection (both individual and collective). The faculty portfolio includes: a) Course card: Course cards are meso-curricular documents that allows the program to connect the macro-curriculum (curricular matrix) with the micro-curriculum (course syllabi). The course card is the mechanism we have developed in order to maintain consistency between course goals (alignment between course and student outcomes) and terminal learning objectives, as well as teaching and learning strategies that will be used to create course syllabi [7]. b) The syllabus of the proposed course is adjusted after a collaborative review carried out by the faculty members within the curricular area. The course planner makes up part of the syllabus and includes, for each session: the objectives for the session, the activities to be carried out before and during the class, the material that will be used, and the activities that must be carried out by the student after the session. c) Eval