Program Assessment Through Product-based Learning in Undergraduate Engineering Programs in India

Engineering education is one of the key enablers for sustainable growth of a nation’s economy. The exponential growth of engineering education in India has affected the quality of engineering graduates in terms of their employability. The National Board of Accreditation (NBA) accredits engineering programs using the Outcome-Based Education (OBE) framework. This framework has twelve graduate attributes of the ‘Washington Accord’ aligned with program outcomes. This paper proposes a systems approach which consists of input, transformation and output towards achieving employable skills in engineers. The program outcomes consisting of technical and professional skills are derived from the competencies required for the target roles in the industry and the graduates’ attributes. Keeping this in mind, a structured outcome-based curriculum was established for a mechanical engineering program in association with the industry. The courses needed towards achieving the program outcomes were identified and course outcomes have been established. While the course outcomes were assessed using formative and summative assessments, the engineering institutions had no proven mechanism to assess the program outcomes explicitly. In an attempt to resolve this issue, a thematic approach called the Product and System Based Learning (PSBL) was adopted in the lines of Product Oriented Learning (POL) and Conceive-Design-Implement-Operate (CDIO) approach in three stages. The three stages include Implement-Operate (Skills), Design-Implement-Operate (Design), and eventually Conceive-Design-Implement-Operate (Innovation). Program learning outcomes were established for each stage as competencies and performance indicators were developed for assessment in the form of a rubric. On completion of the first stage, the performance of the students using the course assessment was compared with the performance assessment using the indicators aligned to competencies. The results showed a very high level of academic performance at the course level assessment, but this result was not reflected at the performance level assessment. This indicates that a direct assessment of program outcomes is important to develop employable engineering graduates for the industry. This paper demonstrates the need for a direct assessment of program outcomes which will ensure the readiness of hands on, industry ready engineering graduates from the academic system and proposes to resolve the gap through an integrated framework. 1. Background Tertiary education, and in particular engineering education, is critical to India’s aspiration of becoming a competitive player in the globalized world [1]. Post the economic reforms beginning in the early nineties, the enrolment to engineering education has increased from a meager 200 thousand in 1947 to 34 million in 2017-18 [2]. Engineering institutions have mushroomed without adequate infrastructure, effective governance and good faculty, resulting in poor quality of education [3]. Thus, the exponential growth of engineering education has significantly affected the quality of engineering graduates in India. The All India Council for Technical Education (AICTE), the apex statutory body for governance of the engineering institutions, reported that the employable engineering talent available is 47%, [4] whereas a testing and certification agency for engineering graduates in India, reports that less than 10% of the engineering graduates are employable [5]. Fair access and affordable participation to quality engineering education are critical to empower its people that allow individual potential to be fulfilled with opportunities for employment [1]. Most of the students who opt for engineering education are driven by parental aspirations or peer group influence than by their own desires or their innate abilities. Hence, they are not fully engaged during their studies, career and life [3]. These issues affect the employability of the engineering graduates. 2. Initiatives at macro level to improve the quality of engineering education The National Board of Accreditation (NBA) was set up to assess the quality of programs offered by engineering institutions in India [2]. Outcome based education (OBE) [6] is targeted to achieve the desirable outcomes (in terms of knowledge, skills, attitudes, and behavior) [2]. NBA aligned its methodology to OBE using graduate attributes of the Washington Accord and started accrediting the engineering programs in India. The graduate attributes are generic to the education of professional engineers in all the engineering disciplines. They are categorized into what graduates should know, the skills they should demonstrate and the attitudes they should possess [6]. While these standards are adopted at a macro level for governance by the accreditation bodies of India, the understanding and deployment at the institutional level needs significant improvement. The authors are engaged in understanding and deploying outcome-based curriculum for few programs in the last eight years. The primary and secondary school education system to a large extent in India is unfortunately rote based, in terms of memorizing the learning content without an understanding of the concepts and context [3]. Engineering is about applied sciences and mathematics. Weak foundation in such subjects becomes a major constraint in learning engineering effectively. This needs to be addressed as this is a prerequisite to learn engineering. 3. Evolution of Outcome Based Education Outcome Based Education (OBE) addresses the features such as outcomes as observable competencies, workplace relevance, assessments of outcomes as judgments of competence, improved skills recognition [7]. The graduate attributes of the Washington Accord are a set of assessable outcomes that are indicative of the graduate’s potential and competence to practice at the appropriate level [8]. In addition, the institutions need to understand the job roles the students/graduates are expected to perform, and competencies required for such roles as an additional input for deciding the appropriate outcomes. This combined set of knowledge, skills, and attitudes is essential for strengthening productivity, entrepreneurship, and excellence in an environment that is increasingly based on technological complexity [9]. 4. Systems thinking The entire chain of education from schools to the university impacts the availability of employable talent for the industry. This chain is complex enough to consider leveraging the concept of systems thinking. Systems thinking is the discipline of seeing the whole and the patterns of change than the static snapshots. All the events are distant in time and space and yet all are connected within the same pattern [10]. A system can be represented as InputTransformation-Output relationship as shown in Fig. 1. Fig. 1: Systems thinking model (Source: [11]) Systems thinkers must be able to integrate ideas, concepts, knowledge, and evidence across disciplinary boundaries [12]. The transformation between the inputs and outputs is the ‘process’ of the enterprise. The ‘process’ is frequently recognized through a set of measures of the process called macroscopic process variables. The output of the process can be classified as technical output and system output. Technical outputs are managed by those Transformation Input Technical output System output who are internal to the process. System outputs are the outputs expected by the stakeholders external to the process [11]. 5. A framework for Product and System Based Learning (PSBL) PSBL framework was evolved by combining the concepts of Systems approach, OBE (Outcome Based Education), ADDIE (Analyze, Design, Develop, Implement, Evaluate) model of instructional design, POL (Product Oriented Learning) methodology and CDIO (Conceive, Design, Implement and Operate) to develop industry ready engineering graduates. The authors have established a system for outcome-based education for an undergraduate program in Mechanical Engineering using system approach as shown in Fig. 2. This model is evolved using systems thinking with clearly defined inputs, technical and system outputs. Outcome based education is considered for transforming the input into output. Fig. 2: Systems approach for education in engineering 5.1 Preparing inputs The job roles that the engineers are expected to play were identified and competencies required for such roles were defined using a structured survey and job evaluation. Using these competencies and graduate attributes of the Washington Accord, the program educational objectives and program outcomes were articulated. The program outcomes were classified as technical and professional skills. Foundation tests were conducted for established higher order skills in English, Mathematics, Physics, and Chemistry for the first-year students. Even Skills PEOs* & POs**: Skills for the target roles and Washington Accord Strong Foundation INPUT TRANSFORMATION • Courses aligned with POs • Electives courses aligned to roles • Course map and relationships • Course outcomes • Deign using IDM &Blooms • Delivery and assessment Outcome-based Education • Co-curricular and extra-curricular activities • Industry attachment program • Product and System Based Learning OUTPUT System Output Technical Output • Technical Skills • Professional Skills • Foundation tests in Maths, Physics, Chemistry, English • Foundation Courses • Academic performance • Qualification for a degree • Aspirational, Competent and engaged talent for Employers • Employment and Life skills for parents * PEOProgram Educational objectives ** PO Program Outcomes those who obtained high scores in the school ended up getting low scores in the foundation tests. This proved that the students coming out of school lack higher order skills in these courses. Foundation courses were developed on those themes studied in their school which gets applied in engineering. Three