Evaluating The Individual Scientist, Engineer, And Technologist: A Review Of Practices And Suggested Framework

Measuring the performance of scientists, engineers, and technologists is a daunting task due to the nature of the job they perform and the absence of common work standards. Presenting this material in an engineering management class can be, therefore, confusing and controversial. This paper will seek to develop a review of industry practice in using five present performance appraisal types relative to scientists, engineers, and technologists; review a new innovation in individual performance appraisals, the individual balanced scorecard; and identify factors important to a successful performance assessment program for scientists, engineers, and technologists. I. An Introduction to the Problem and Report Objectives It starts with the prescribed form sent to each engineering manager with the little check boxes and a No. 2 pencil. The manager begins, “On a scale from one to five evaluate the engineer’s problem-solving ability.” Immediately the manager struggles to remember key instances in the last year where the engineer exhibited good or poor problem-solving abilities. Remembering none, the manager decides to rate the engineer a 3.5 based on no recent memory of odious or pernicious errors in judgement. The performance appraisal process has begun. The performance appraisal is a well-recognized and established feature in modern corporate climates. Although many companies espouse a belief in systems thinking, the individual performance appraisal is a cornerstone of American management [1]. A survey by the Wyatt Company, a Washington consulting firm, found less than half of employees evaluated liked the process and the reviewers were even less satisfied [2]. Measuring the performance of scientists, engineers, and technologists is a more daunting task due to the nature of the job they perform and the absence of common work standards [3,4]. A. Schainblatt states, “There are no currently used systems for measuring the productivity of scientific and engineering groups without substantial flaws. Nor does the literature on productivity measurement offer encouragement that suitable systems will soon be available” [5]. This makes the topic of performance appraisal a particularly difficult and controversial portion of an engineering management course. This paper will seek to develop a review of industry practice in using five present performance appraisal types relative to scientists, engineers, and technologists; review a new innovation in individual performance appraisals, the individual balanced scorecard; and identify factors important to a successful performance assessment program for scientists, engineers, and technologists. The purpose of such a review is to P ge 733.1 “Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition Copyright © 2002, American Society for Engineering Education” provide material suitable for presentation in an engineering management course at the graduate level and to provide instructors and students of engineering management a framework to evaluate variations of performance evaluation formats. II. Performance Evaluation Definitions and Types The literature is imprecise concerning definitions of performance appraisals and for defining the technical function. It will be necessary to clarify these terms in a course of engineering management for discussion purposes before undertaking a review of relevant types of performance appraisals. II. A. Operational Definitions Before undertaking a discourse in the measurement of individual engineers and scientists, it will be beneficial to define the terms: performance assessment, performance appraisal, scientists, engineers, and technologists. II. A. 1. Performance Assessment According to Berk, “Performance assessment is the process of gathering data by systematic observation for making decisions about an individual” [6]. He notes five key elements in this definition: ∑ performance assessment is a process, not a single measure or event; ∑ an emphasis exists on data gathering with many instruments; ∑ the data represent the information from systematic observation; ∑ the data are used to make decisions that provide the form and substance of the assessment; and ∑ the focus of the assessment is the individual [6]. Performance evaluation is a synonymous term for performance assessment. The assessment of the individual is conducted in terms of the job requirements of knowledge, skills, and aptitudes gathered through job analysis [7]. II. A. 2. Performance Appraisal According to Cascio, “Performance appraisal is the systematic description of the job-relevant strengths and weaknesses with and between employees” [8]. Although similar to the definition above, this paper will use this definition of performance appraisal because of its focus on the systematic description of the employee. A performance appraisal is distinguished from performance assessment by its focus on the description and its omission of the job analysis and data gathering. Performance appraisal may be seen as the rating or measurement of the employee based upon a performance assessment system. II. A. 3. Scientists, Engineers, and Technologists For the purposes of this paper, scientists, engineers, and technologists are defined according to their roles in business and industry. Scientists are interested in the study of particular aspects of the universe. They may be involved in pure research into the nature of things or may have a directed interest into the innovation or breakthrough necessary to create a new product or service. Generally speaking, a scientist’s interests lie in the “why” of the world [9]. While a scientist is interested in the “why,” engineers and technologists are interested in the “how” of P ge 733.2 “Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition Copyright © 2002, American Society for Engineering Education” things. Engineers apply mathematical and scientific principles in creation of useful products and services [9]. Technologists use existing technologies and systems to build or refine products or services [9]. For all these groups, the creative and innovative process involved in doing their work causes difficulty in establishing standards of work performance. The technical function within an organization will encompass all the jobs defined as scientists, engineers, and technologists. II. B. Purpose and Types of Performance Assessments There are numerous ways to evaluate employee work performance discussed in the literature. Many graduate engineering management students will be familiar with Deming and his beliefs that the individual performance appraisal should be eliminated. The review of literature and practice tend to show the performance appraisal can serve many useful purposes. These purposes, as part of the total performance assessment, are discussed in the next section. The subsequent sections will explain and discuss five different types of performance appraisals. II. B. 1. Purposes for Performance Appraisals W. Edwards Deming and some of his followers have been enthusiastic about eliminating the individual performance appraisal, because they find it is detrimental to the system view o f the organization and does not properly consider individual variation in measurement data [10,11]. Deming even goes so far as to label appraisals as the third Deadly Disease of Companies of the Western World [10]. A summary of Deming’s charges is presented in Table 1. Appraisals nourish short-term focus and performance Appraisals feed rivalry and politics that are detrimental to organizational goals. Appraisals build fear of management into the system. Appraisals anger and embitter employees. Appraisals demolish teamwork. Appraisals and ratings are unfair as they ascribe system variation to individuals. Appraisals increase variability in job performance. Appraisal rewards performance within the system instead of improving the system. Table 1. Deming’s Exhortations Against Appraisals Cardy and others assert the performance appraisal can be designed without the problematic flaws that lead to Deming’s criticisms [12,13]. Such an appraisal, properly designed, can serve several purposes for the employer and employee [12]. These are summarized in Table 2. Employer’s Perspective Employee’s Perspective Individual differences in employees do affect organizational goals. Feedback is desired and needed. Documentation from appraisals may be needed for legal defense. Improvement requires feedback and appraisal. Appraisal help rationalize bonuses or merit. A sense of fairness demands individual differences be recognized and good performance rewarded. Appraisals can operationalize strategic goals and relate them to individual performance. Appraisal and recognition can motivate an employee to better performance. The appraisal is part of a valid performance assessment. Appraisals can include teams as its focus. Table 2. Cardy’s Purposes for Performance Appraisals P ge 733.3 “Proceedings of the 2002 American Society for Engineering Education Annual Conference & Exposition Copyright © 2002, American Society for Engineering Education” The performance appraisal can also be used in training and development efforts to assess potential and identify needs [14]. II. B. 2. Five Types of Performance Appraisals Creativity and individual application influence the specific content and method of the performance appraisal; however, there are five major types of appraisals relevant to scientists, engineers, and technologists found in the literature. II. B. 2. a. Employee Ranking Employee ranking is a popular method of evaluating employee performance by directly comparing all employees performing a similar job for a single unit or single supervisor [15]. Employees are simply given an ordinal rank relative to their co-workers. Jacobs notes two fundamental flaws with such a system: 1)it is inherently ordinal and 2)it compares over