Assessing the Impact of an Interdisciplinary First-Year Experience Program

This research paper builds upon an ongoing exploration of a large-scale, interdisciplinary course integration for first-year Technology majors. Our research begins to show that the program is making measurable differences to students’ learning, engagement, and sense of community. Administrators and instructors from 2 colleges within the university collaborated to organize and teach paired sections of Technology, English, and Communication courses. All 3 classroom instructors in each set of sections work together within the Integrated First-Year Experience, which ultimately aims to tie essential skills and concepts from the humanities and STEM fields to realistic global problems and contexts. The program was implemented for more than 500 firstyear students in each academic year (2015–16 and 2016–17). Our research responds to questions about the potential for integrated courses to improve students’ conceptual learning and engagement with the university, and about the most effective ways for instructors and administrators to plan, support, and implement this kind of integration. We demonstrate the potential for interdisciplinary pedagogy generally and STEM–Humanities integration in particular to improve students’ perceived learning transfer and sense of academic engagement. Introduction This research responds to questions about the potential for integrated courses to improve students’ conceptual learning and engagement with the university, outlines our methods of evaluating a large-scale course integration, and discusses effective ways for instructors and administrators to plan, support, and implement strong integration. With this paper, we build upon an ongoing exploration of a large-scale, interdisciplinary course integration for first-year Technology majors: the Integrated First-Year Experience. This course integration program grew out of the recognition that Technology students seem to struggle effectively expressing design ideas, while student work in English and Communication courses can sometimes seem to lack a meaningful context outside of the classroom. To address these concerns, and with a goal of enriching the first-year experience for students, administrators and instructors from Purdue’s Polytechnic Institute, English Department, and School of Communication all collaborated to organize and teach paired sections of Technology, English, and Communication courses. The resulting Integrated First-Year Experience brings humanities and STEM fields into cooperation, asking instructors from both to collaborate at the classroom level. The program’s overall goal has been to improve student learning, transfer, academic engagement, and sense of community. The following sections of this paper engage with existing literature and provide background on the program and its development since 2015. In order to document and measure the impact of our integrated pedagogy over the last 2 years, we have collected and analyzed student data from both integrated and non-integrated sections of the courses involved. Our analysis and comparisons encompass Fall 2015 and Fall 2016 sections as well as integrated and nonintegrated sections from Fall 2016. Following this analysis, we discuss the implications of our findings and plans for future research. Existing Research on Assessing Integrated Pedagogy Integration among and across engineering and other disciplines has been studied from many perspectives. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 Honey, Pearson, & Schweingruber summarized much of the current integration research and put forth concrete principles of effective integration. Their report also acknowledged the risks and challenges of implementing integrated pedagogy. Others’ reports on integration cover various program designs and strategies, and have outlined specific ways of assessing such programs. Kellam et al. described a curricular integration among design, engineering, and social sciences threaded through 4 years of their engineering program, noting that the goal was for disciplines to integrate “at both a content level (integration of content across courses) and a meta-level (integration of meta-learning and ways of thinking)” (p. 2). They implemented a studio requirement each year, where project based learning, community service, and reflection are highlighted. Kellam et al. drew from student reflections and focus group transcripts in their evaluation of this long-term integration. Guthrie et al. used quantitative student self assessment and collected student comments to gauge the effectiveness of their interdisciplinary capstone design course. Rhee et al. in “A Case Study of a Co-instructed Multidisciplinary Senior Capstone Project in Sustainability” discussed a senior capstone course where students engage together with specific shared projects, share classroom space and meeting times. Mentors from several engineering and non-engineering disciplines assist students. Rhee et al. assessed their program using surveys, focus groups, and interviews. There is strong potential for integrated and interdisciplinary curriculum to improve student learning and experience. Transforming engineering curricula to more thoroughly and consistently integrate core principles of engineering design with communication and critical thinking skills can make engineering education more valuable and more effective overall; however, such integration efforts may not always succeed. Often the work of integration is highly demanding in terms of logistical planning, institutional support, and instructor commitment, as Hirsch et al. recognized. Guthrie et al. noted that collaborative, integrated pedagogy “requires significant time, a flexible mindset and a commitment to collaborate.” Combining resources across colleges or schools provides great opportunities for interdisciplinary, cross-college instruction, but also comes with complexities and potential costs. Our project compares the implementation of this integration program across two semesters, documents the pedagogical and programmatic improvements being made, and begins to measure what difference the program has made in terms of students’ experiences. We describe specific pedagogical and logistical preparations and our preliminary quantitative assessment of the results. Background and Research Questions The Integrated First-Year Experience was implemented for more than 500 first-year students in each academic year (2015-16 and 2016-17). A description of the Integrated First-Year Experience and related assessment is described more fully in Chesley et al. Here we review the general logistics of the program for each year’s implementation. During both years, Technology instructors teaching Design were paired with English and Communications instructors to collaboratively connect and reinforce the skills of all three courses for students enrolled in the program. In each introductory Design course, 40-45 students are enrolled; of these, 20-25 are also enrolled together in an introductory Communication course, and 20 are enrolled together in an introductory English composition course. All students enrolled in the Integrated First-Year Experience shared instructors, classroom space, and class time with the same group of peers across two paired courses—either Design and Composition, or Design and Communication. All three classroom instructors in each set of sections worked together within the Integrated FirstYear Experience. At Purdue University, the introductory Communication course and the introductory English course are required of nearly all students. The introductory Design course is required of every student majoring in the college of Technology at this institution, and students are encouraged to take the course during their first year. For the first implementation of this course integration in Fall 2015, administrators and instructors taught 13 sections of integrated Technology courses and 3 non-integrated sections. In the spring, 1 integrated section was taught along with 9 nonintegrated sections. The following academic year, 12 integrated sections were taught, but split across Fall and Spring semesters evenly. Thus, 6 integrated and 6 non-integrated Technology sections were taught during Fall 2016. Data collection for this report is limited to fall terms only because the spring terms’ courses included a large number of upper level students who may be enrolled in one, but not all three courses. To measure the impact of this interdisciplinary integration, our larger research project has collected and will be analyzing qualitative and quantitative data from students, instructors, and administrators involved in the Fall 2015 and the Fall 2016 semesters. For the current study, we wanted to know whether students’ perceptions of their own learning and engagement with design thinking were higher in integrated sections of this Technology course. We also hypothesized that the refinements and changes made to the Integrated First-Year Experience would lead to higher perceived learning transfer and engagement. Our basic research questions are: 1. Did changes to our integrated pedagogy improve students’ learning experience, sense of engagement and transfer in integrated sections in year 2? 2. Does the integration program lead to a better learning experience and sense of transfer and engagement for students in integrated sections compared to non-integrated sections? To address these questions, we first describe the Integrated First-Year Experience program as it was implemented in Fall 2015 and then in Fall 2016. We then describe our data collection and analysis thus far. Year 1: Fall 2015 For this new course integration, 34 instructors from all 3 departments were grouped together in “trios” to teach and integrate their Design Thinking in Technology, English, and Communication courses. Each trio of course sections included one Design Thinking section, one English section, and one Communication section, all linked tog