Assessing Both Technical And Artistic Skills In Digital Media Courses Within A Technology Program

Digital media courses are usually found in art or communications departments where the instructional emphasis is weighted toward artistic and creative aspects of project development. In digital media courses offered in departments oriented to the technical professions, such as computer science and engineering technology, the instructional emphasis is usually on technical skills. Regardless of where these courses are housed, graduates working in animation, 3D visualization, and interactive media fields are finding that an ability to apply both technical and creative skills is necessary. This presents challenges in classrooms where students range from the "very artistic/somewhat technical" to the "very technical/somewhat artistic", not least of which is how to assess projects in which both technical and artistic skills must be demonstrated. There is often a negative perception of the critique process as being purely subjective. This “that’s-just-the-teacher’s-opinion” perception often becomes the stopping point in attempts to get students to recognize critique as part of an iterative development of a design solution. How we handle this assessment can be the difference between a student seeing critique as truly constructive criticism, or merely a matter of opinion. This researcher has developed a comprehensive method that addresses both objective and subjective criteria while giving students confidence in the validity of the critique. In this method, peer and instructor feedback is given informally as students develop their ideas, and formally at project completion with a traditional class critique. This is followed by completion of an on-line form incorporating Rikert scales and comment fields for specific criteria. Works being evaluated are also posted so students can view each piece as they complete the form. Results are compiled into a database, and a password protected report is automatically generated for each student showing the average ranking for each question and compiled comments. Anonymity is preserved, allowing students a comfortable way to provide honest feedback to classmates. Students can use this report to guide revisions to their work, and the instructor can use it to evaluate how the class, and each student, understands the principles being taught, and how their abilities to think critically are developing. By tracking averages of student evaluations over several years, it has been found that peer feedback and instructor feedback closely correlate, providing affirmation of the critique to the student, and support for the final grade. Creative Critique vs. Objective Scoring Critique is the traditional means used to provide constructive analysis and advice to students. The typical process involves in-class discussion where everyone present has an opportunity to offer verbal feedback intended to help each student understand how well their work demonstrates skills they are learning. i In many digital media courses offered in art departments, the instructional emphasis is weighted heavily toward the creative. In these courses, traditional in-class critique of creative assignments is expected by the student as this is the norm in every art class they take. Critique in this situation is subjective in that there may be many solutions to a design problem. Every student’s solution will be different, and there is no one right answer. Discussion in the critique is intended to help the students understand the relative strengths and weaknesses of how design principles were applied to the work presented, and to encourage them to explore options to improve the solution. Since instruction on software and technical skills in art departments is often limited, scoring on these skills is secondary to the creative in critique. The score on a project may include a small percentage for technical merit based less on whether the student can perform particular “tasks” and more on whether poor application of technical skills has negatively impacted the visual result. Scoring in this case can be very subjective and can differ somewhat from instructor to instructor. Even in art courses where the students are used to this kind of evaluation, students will still complain that the instructor is just offering opinion and not concrete evidence for assigning a grade—particularly if they do not agree with the instructor. Peer evaluation during the in-class critique will usually help to support the instructor’s comments in cases where the class is accustomed to the critique process and where everyone participates fully in the discussion. This becomes a bit more problematic in beginning courses, or when a particular class has a high percentage of “non-participators.” In courses offered in many technical programs, instruction is more heavily weighted to learning software and technical skills with less emphasis placed on creative application. Scoring in these courses are often more objective, with points given for accomplishing each particular task required by the assignment. Creative aspects of an assignment are evaluated by a traditional critique process in most cases, but the percentage of the score for design and creativity will not be as high. This percentage, even though smaller than the technical, may serve to adjust the final score enough so that students will “blame” this loss of a letter grade or more on the instructor’s opinion. This is particularly true in courses where design principles are not emphasized strongly enough to help students develop strong criticism skills, putting the burden of providing feedback on the instructor. Having taught in both types of programs, this researcher has observed even less acceptance of the “subjective critique” from students in the technical programs than from those in art departments, in part because it has not been as much a part of their educational experience. ii Digital media, as an industry, is relatively new, and instructional programs to train the artists and technical staff needed are even newer. As the industry has matured, it has become evident that a good balance between artistic and technical skills will be required from future digital media professionals. Instructional programs are responding to the needs of this growing industry and are beginning to develop and apply best practices for instruction that can provide this balance. In the Digital Media Program (DIGM) at East Tennessee University (ETSU), located in the Department of Technology & Geomatics, courses have been recently redesigned to include both creative and technical content in a more balanced proportion. Because we are in an engineering technology department rather than in an art department, many of the students who come to the program do not consider themselves to be “artists” first, and tend to put less value on the creative aspects of the courses than on the technical. The “that’s just the teacher’s opinion” complaint has been a common response to critique in this program. How we handle the less concrete parts of our assessment can be the difference between a student seeing a critique as merely a matter of opinion, or as truly constructive criticism. In an attempt to make the critique process more effective—and more evidently valid—for these students, a method of project assessment that addresses both objective and subjective criteria while giving students confidence in the validity of the critique has been developed This method combines traditional critique with objective scoring and peer corroboration. The Method In this method, critique becomes part of the evaluation process in an iterative way, similar to the process of creating and revising work in a work situation. Peer feedback is given informally in concept critiques as student develop their ideas, and formally at the end of the project with a traditional in-class critique and an online critique. Goals for critique throughout the process are as follows: • To identify aesthetic and creative strengths of the solution to the assigned problem, • give students practice identifying problems and proposing possible solutions, • help students develop critical analysis skills in evaluating their work and the work of others, • give students practice defending their work by using knowledge of appropriate principles of design and communication as support for their solutions, and • give students opportunities to apply feedback from critiques toward improving the work. When a project is assigned, students are given a detailed document describing project parameters (Appendix 1). Included is a “guidelines” page outlining considerations for assessing design principles and what to look for when evaluating technical aspects of the project (Appendix 2). They are also given a copy of the survey that they will complete as part of an online critique (Appendix 3) and a score sheet specifying point breakouts for the project criteria (Appendix 4). These will be somewhat different for each project depending upon the specific objectives for the project. Because some parameters of a project are more concrete, the score sheet can be used as a checklist by the student for confirming that all conditions have been met. Providing these documents at the beginning of an assignment lets students know up front how grades will be assigned. The iterative assessment and revision process throughout the assignment generally proceeds as follows: 1. Brainstorming and feedback on initial ideas, and on sketches, storyboards, or other previsualization 2. Informal feedback by the instructor and the class during production 3. In-class critique on the finished project 4. Online critique assigned as homework 5. Report generated from the online critique 6. Revision of the project based on feedback 7. Final grade and instructor feedback, including scores for both technical and creative criteria. In the “brainstorming and feedback” phase, students present conceptual ideas (usually more than one) for the