Sequential Unmasking: A Means of Minimizing Observer Effects in Forensic DNA Interpretation

Sir: Observer effects are rooted in the universal human tendency to interpret data in a manner consistent with one’s expectations (1). This tendency is particularly likely to distort the results of a scientific test when the underlying data are ambiguous and the scientist is exposed to domain-irrelevant information that engages emotions or desires (2). Despite impressions to the contrary, forensic DNA analysts often must resolve ambiguities, particularly when interpreting difficult evidence samples such as those that contain mixtures of DNA from two or more individuals, degraded or inhibited DNA, or limited quantities of DNA template. The full potential of forensic DNA testing can only be realized if observer effects are minimized. We met on December 1 and 2, 2007 in Washington, D.C. to discuss the implications of observer effects in forensic DNA testing and ways to minimize them. The interpretation of an evidentiary DNA profile should not be influenced by information about a suspect’s DNA profile (3–6). Each item of evidence must be interpreted independently of other items of evidence or reference samples. Yet forensic analysts are commonly aware of submitted reference profiles when interpreting DNA test results, creating the opportunity for a confirmatory bias, despite the best intentions of the analyst. Furthermore, analysts are sometimes exposed to information about the suspects, such as their history or motives, eyewitness identifications, presence or absence of a confession, and the like. Such information should have no bearing on how the results of a DNA test are interpreted, yet may compound an unintentional confirmatory bias. This bias can result in false inclusions under not uncommon conditions of ambiguity encountered in actual casework. It can also render currently used frequency statistics or likelihood ratios misleading. These problems can be minimized by preventing analysts from knowing the profile of submitted references (i.e., known samples) when interpreting testing results from evidentiary (i.e., unknown or questioned) samples. The necessary filtering or masking of submitted reference profiles can be accomplished in several ways, perhaps most easily by sequencing the laboratory workflow such that evidentiary samples are interpreted, and the interpretation is fully documented, before reference samples are compared. A simple protocol would dictate a separation of tasks between a qualified individual familiar with case information (a case manager) and an analyst from whom domain-irrelevant information is masked. Such a protocol would have the following steps. First, the analyst interprets the results of testing on the evidentiary samples. In this initial interpretation, the analyst would perform the following: