Watching the watchdogs

Recently, the journals Science and Nature have retracted several papers from the same research group, acknowledging that the research was fraudulent.1-4 This possibility is a nightmare for any editor. There are two traditional ways of avoiding the problem. The first depends on the rigour of the review process in which selected peers are asked to assess the quality of a paper and the contribution it makes to a field. The second requires that the authors provide a clear account of both the research methods used and the analysis of the data so that reviewers can assess the research process. These are standard procedures for any peer-reviewed journal. Both processes work well, although they are notably unsuccessful in identifying fraudulent research.1-4 While these standard journal processes are well understood, and are usually honoured, there is less consideration of an additional issue: the formal rules that govern decision making by journal editors. What processes should be in place to ensure that editors do not make biased decisions in accepting or rejecting papers for publication? In this editorial we set out the processes we use in this Journal. In a multidisciplinary field such as public health, it is, we believe, an advantage to have editors from different disciplinary backgrounds and our backgrounds in medicine (JL) and sociology (JD) could, arguably, hardly be more different. While we both have a broad interest in public health research methodology, one of us (JL) is more experienced in epidemiological research and the other (JD) is more experienced in research using qualitative methods. Because we make joint, face-to-face decisions about papers, both disciplinary and methodological preconceptions are tempered. If a diversity of views in editors is an advantage in reducing bias, then so is a diversity of reviewers. For each paper sent for review, we select from our database three reviewers from different backgrounds to represent the public health fields with an interest in the topic of the paper. This might include, for example, a reviewer to comment on clinical issues in a health field, a reviewer able to comment on relevant policy issues and a reviewer expert in the research methods used. It is worth noting here that our reviewers are exemplary in providing assessment of papers where the substantive concerns may fall outside their present research interests but where we require specific information about aspects of a paper. We place great weight on the views expressed by reviewers. These include the written comments for authors as well as the graded recommendation seen only by the editors. A paper may be classified as Accept, Minor revision, Major revision with or without re-review or Reject, but reviewers also indicate on a scale of 1 to 5 ratings of the contribution the paper makes to public health, aspects of the methods and the validity of the conclusions. Where reviewers are in agreement, we accept their decision. Where there is substantial disagreement, we are required to make an editorial judgement. Given the important issue of research bias, if there is damaging criticism of the methods used or the validity of the conclusions, we might over-ride a positive recommendation by a reviewer with a different perspective and reject the paper. More often, if the paper addresses an important area of public health or is the subject of current debate, we might seek additional reviews to ensure that we reach an informed decision. We lean towards allowing authors to revise a paper to address concerns raised by reviewers and we are diligent in checking that each issue has been addressed, preferably in the paper itself. If a reviewer has asked to re-review a paper, it is returned to this reviewer. There are issues that raise hackles and divide the public health community. In these (fortunately rare) cases the well-informed comments from one side may be strongly resisted by the other and it becomes very difficult to achieve a revision of a paper to address points raised by a reviewer. Such controversies are worthy of public debate and our preference then is to publish both opposing views. In these cases, we welcome further debate in the Letters section. So far, we have considered only issues where the editors have no vested interest. A different procedure is needed when one of the editors has a conflict of interest, either because of professional or personal connections or because she is an active participant in a research field. That leaves the remaining editor making decisions in an area where she may not be well informed of the niceties of the field or the research method used. Our database of reviewers lists areas of interest and this provides a good guide. If the authors are asked to revise a paper and the remaining editor has difficulty in assessing whether a reviewer’s comments have been addressed, she has the option of returning the revision to the original reviewer. Alternatively, and this applies to all difficult decisions, one or more members of the editorial board can be asked for their assessment. Membership of the editorial board is required to represent all States and Territories and to cover a diversity of disciplines and interests. Finally, editors have to be able to justify their editorial decisions when called to account, whether publicly or in private. We are aware that there are periodic concerns about the balance of views represented in the Journal, with suggestions that we are favouring epidemiological research and neglecting research relevant to practitioners and policy makers. Given the confidentiality of the review process, it is difficult to counter these claims. We depend substantially on the papers submitted to us and these tend to favour certain topics and approaches. We accept for publication about 40% of the papers submitted to the Journal and papers addressing neglected topics do not necessarily survive the review process. Often the concerns are methodological and during 2006 we will be devoting some of our editorials to discussion of methodological approaches that enhance the chance of acceptance of papers.