When is a replication not a replication? Or how to spot a good genetic association study

Published genetic association studies are becoming an epidemic. For example, whereas only 53 PubMed entries are found for the keywords “polymorphism” and “arthritis” in 1998, the number of entries is nearly 3-fold higher for 2004. With the recent completion of phase I of the haplotype map (HapMap) project and the continuing drop in genotyping costs, genome-wide association studies using up to 500,000 markers are becoming feasible. Soon we can expect a flourish of genetic association studies with positive findings to appear in the leading journals. Although the findings of these studies will be exciting at first glance, the vast amount of genes and polymorphisms tested can leave both readers and reviewers confused as to their interpretation. If already published studies serve as predictors of what is to come, a high percentage of the to-be-published genetic association studies may yield one-off results that will fail to be consistently replicated. In fact, according to recent metaanalyses, only 16–30% of initially reported significant associations have been consistently replicated (1,2). Encouragingly, we know some of the reasons for this inconsistency. First, the attitude of the modern gene hunter has not yet evolved to keep pace with the recent developments in genetic epidemiology, and the optimism is still largely based on findings for Mendelian disease (3). To combat this, useful guidelines that addressed this issue were produced nearly 2 years ago in Arthritis & Rheumatism (4) (see Table 1). Given the advances in technology and methodology, however, the current dilemma involves 3 questions: What do results actually mean? Can they be relied upon? And, most important, is it worth investing more funds to follow them up? A number of factors can explain the lack of confirmation of an initial positive finding. These include 1) overestimation of the genetic effect in the first report, 2) inconsistent coverage of genetic variation in the gene, 3) inherent genetic differences in the populations studied, and 4) differences in the clinical definition of phenotype.