An Objectives Based Approach To Assessment Of General Education

This paper describes the development of an innovative strategy to assess how students and faculty perceive and accomplish the objectives of general education at Penn State. The University’s general education curriculum is intended to achieve a number of educational goals, including the exploration and development of knowledge domains and skills that are consistent with, and complementary to, the learning outcomes associated with the students’ major programs of study. A diverse team was assembled to evaluate three crucial aspects of general education, namely, its design, delivery and reception. The collaboration began with examination of coursetaking patterns and framing of the University Faculty Senate’s expressed objectives for general education in the context of the program goals and learning outcomes for selected technical and non-technical majors. Focused interviews with students and information solicited from course instructors were then used to gain an understanding for how these stakeholders actually view their experiences and course goals/delivery mechanisms, respectively, in terms of this objectivebased matrix. A first attempt to implement an on-line methodology was made with limited success. The lessons learned shed light on the challenges and opportunities for scaling up a process that would allow efficient and widespread program assessment, across many disciplines of study, to facilitate academic advising and curricular improvement. I. Origins of General Education Assessment at Penn State Assessment of the general education program at Penn State has long been of interest at the University, owing to its prominence as a substantial component of the curriculum and degree requirements. The need for comprehensive assessment was articulated most specifically over a decade ago by a Task Force on Undergraduate Education charged by the provost in 1991. A Commission for Undergraduate Education subsequently developed a plan for assessment in 1993, 2 and other reports on curricular coherence and relevance and assessment of educational outcomes followed in 1995 and 1996, respectively. 3,4 These initiatives focused, however, on surveying and encouraging the various motivations and methods for assessment within the independent disciplines, and the first call for broader application across the entire curriculum was issued as a key recommendation of the University’s most recent general education reform effort in 1997. The Special Committee on General Education emphasized the imperative to “institutionalize a process for formative assessment that is based on measurable outcomes, recognizes the importance of learning processes and informs continuous curricular Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright 2005, American Society for Engineering Education P ge 10193.1 improvement.” Other important characteristics of the assessment strategy suggested in that study were that it should be founded on the goals for teaching and learning; owned and implemented at multiple levels – by major programs and the principal “delivery” units; and geared, not at improving specific courses, but at the general education curriculum as a whole and the multidimensional opportunities it affords to students in meeting the career and life goals that they, as well as other stakeholders – faculty, future employers, etc.– deem important. The American Association for Higher Education espouses similar best practices for assessing student learning, including involvement of “representatives from across the educational community,” and the use of approaches that “reveal change, growth and increasing degrees of integration,” by focusing on experiences that lead to the expressed learning outcomes. As will be seen later, our design for assessment draws heavily on these principles. Another fundamental change made to the general education program at Penn State was the emphasis on more active engagement of students in their own learning. The general education curriculum has always been and still is defined by skills and content areas or “knowledge domains,” constituting a substantial 45 credits of the university-wide degree requirements. These include writing and speaking, quantification, health and physical activity, natural sciences, arts, humanities, social and behavioral sciences, and international and U.S. cultures. The new, Faculty Senate-approved requirement, however, stipulated that active learning elements should be incorporated into the delivery of all courses carrying general education credit, namely active use of writing, speaking and other forms of self expression; opportunity for information gathering, synthesis and analysis in solving problems and critical thinking; engagement in collaborative learning and teamwork; application of intercultural and international competence; and dialogue pertaining to social behavior, community and scholarly conduct. It was this feature of the Penn State general education that opened the door for an assessment process that begins with students’ reflection on their experiences related to core competencies or involving inor out-of-class learning activities. II. Goals for General Education Assessment The Team for Assessing Student Learning was charged in February 2004 under the Teaching and Learning Consortium (TLC), an arm of the University’s Schreyer Institute for Teaching Excellence. The group’s formation coincided with the development of a self-study for the University’s accreditation review by the Middle States Commission on Higher Education, in which student learning outcomes were selected as an area for re-examination and strengthening. As part of this effort, a Web survey was conducted of all departments on their articulation of expected learning outcomes in undergraduate and graduate programs, as well as their assessment practices. Notably, responses were received for over half of the University’s 252 baccalaureate programs, and over two-thirds of the respondents reported that they have explicitly defined and written learning outcomes (in contrast to the 24% who reported that they had such defined outcomes in an earlier 1993 survey). However, only a third of the programs reported that measuring achievement of general education goals is attempted as part of the assessment activity. The kinds of assessment methods included a wide variety of student, alumni and employer surveys and interviews, and to a lesser extent, portfolios, capstone projects and practica, and standardized testing. The survey also found substantial variability in the extent to which program outcomes were mapped to course goals and outcomes. Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright 2005, American Society for Engineering Education P ge 10193.2 The assessment of general education curriculum has also been a stumbling block for even those programs subject to discipline-based accreditation review, such as Engineering, Business, Education and Nursing. For instance, while the College of Engineering has traditionally acquired course syllabi for many courses in the general education curriculum and mapped the course outcomes to the program educational objectives and outcomes articulated in the ABET criteria, the prospect of assessment is complicated by a number of factors. First, in most cases the students are selecting courses in the various knowledge domains from a suite of over 300 approved general education course offerings. Second, although course proposals and their subject outlines are approved centrally by the University Faculty Senate, the specific topical content and course objectives, even for different sections of the same course, may vary significantly. Third, there is no formal mechanism for exchange of information regarding the expected outcomes for various degree programs as they may be addressed through the general education curriculum. And fourth, formal entry to the engineering majors occurs in the third or fourth semester, after students have already taken a substantial portion of their required general education courses. The disconnect between general education and the major programs may be further complicated by the multi-campus configuration of the University, where nearly half of the engineering students typically spend their first two years at 21 campuses distributed throughout the Commonwealth before entering their majors offered at only three of those campus locations. The TLC Team was composed of representatives from a diverse mix of fields – faculty from engineering, education and liberal arts, instructional design and technology experts, statisticians, etc. – to tackle these inter-related problems. A first cut at the team charge was simply to assess how students meet the University’s general education goals and how these intersect with specific program objectives and outcomes, providing recommendations on how the learning outcomes can be more effectively met through the current course electives. The approach to be used involved mapping students’ experiences in the subset of courses most frequently taken to understand their motivations and the associations they make with defined program goals. To accomplish this, attention would be given to identifying the technology that would afford efficient and broad-based input from students in many different disciplines. The premise was that, by providing this information to course instructors, they will be able to evaluate their expected or desired course goals from the consumers’ perspectives – both the enrolled students and the programs that rely on general education to further their accomplishment of selected educational objectives. And finally, the resultant assessment process/instrument was intended to: (1) Encourage students to select courses and monitor their progress more coherently, with a better informed perspective of their programmatic, ca