Re: BAKER P.J. & HOEL D.G. (2007) European Journal of Cancer Care 16 , 355–363. Meta‐analysis of standardized incidence and mortality rates of childhood leukaemia in proximity to nuclear facilities

We have read the above paper with considerable interest. It is an important topic and it was about time someone tackled it. Since increased childhood leukaemia mortality was found in a 10-mile radius of the nuclear installation at Windscale (see e.g. Black 1984), the question has never left the public eye, and so a number of studies have been published. However, we see a number of limitations and flaws in this meta-analysis, of which we would like to point out the main points. There is the general problem of summarizing heterogeneous data, which all meta-analyses suffer from up to a point. The main sources of heterogeneity here are the different age groups, different types of nuclear facilities and different zone definitions. Beyond this, this meta-analysis has a number of rather specific problems. The authors seem to have identified studies until 1999 which approached the question by calculating Standardized Incidence Ratio/Standardized Mortality Ratio (SIRs/ SMRs) for childhood/adolescent leukaemia incidence or mortality in geographical zones around nuclear sites. The completeness of the publication search cannot be verified, as only the 17 studies which were used in the metaanalysis are referenced, but not the 20 studies which were identified but excluded: ‘Thirty-seven studies were identified for possible inclusion. Seventeen studies . . . met the criteria for at least one analysis’. The possibility of bias by selecting 17 out of 37 is not discussed in the paper. At least one reason given for this selection is questionable in our opinion: ‘The study must include . . . for individual nuclear sites, as opposed to a summarization that includes multiple sites’. Not publishing single site results, as they may be spurious and based on small numbers of cases, is good scientific practice in our opinion. Technically, it is perfectly possible to summarize summary SMRs/SIRs in a meta-analysis, even without having the information on the actual number of observed and expected cases. Furthermore, ‘Five sites in the USA were excluded due to zero observed deaths . . .’. Leaving a site out because of zero events is definitely a biased selection away from the null. The authors provided us with the funnel plots for the analyses which were conducted. Some of the funnel plots (SIR for age group of 0–9) show a telltale hole for the combination SIR close to 1 and 1/standard error small, that is, analyses with a small number of cases. This indicates a publication bias. Beyond this, there seems to be a general problem of understanding the methodology. For example, SIR/SMR is called a rate throughout the paper (including the title), it is however a ratio. The forest plots (figures 1 and 2) claim to show SIRs/SMRs and their confidence intervals on a log scale; however, a log scale and correctly calculated confidence intervals would lead to symmetric confidence intervals, while the ones shown are obviously skewed. We would very much like to quote a meta-analysis of such studies in our work no matter what the result, instead of single inconclusive studies, if only we were certain that it had been done properly. This is regrettably not the case in the paper presented here.