On balanced representation in research

After the December issue was published, this Journal received some negative comment on a paper by Steele et al on children in immigration detention centres. The most serious accusation was that the authors were over-sympathetic to their research participants, so that the children’s mental illness was exaggerated. Thus the scientific credibility of the paper is called into question. At a more personal level, a book on evidence-based medicine, by one of us (JD),1 has aroused negative comment for failing to represent fairly the contribution of women researchers. Underlying both criticisms is the problem of how to assess credible research. How should we reconcile different points of view about what should be addressed in research? How do we ensure balanced representation in the research published in the Journal? Evidence-based medicine has taught the clear lesson that there is unlikely to be a definitive study in health research, no matter how carefully it is designed and executed. The most rigorous trial will be subjected to review in the light of subsequent research and the conclusions may be revised or even rejected. This is even more likely when research addresses a context where there are numerous interacting factors operating, such as in a communitybased study. Here it would be almost impossible to conduct a single definitive study. Instead, researchers focus on the major factors relevant to their research question. The issue then is to be clear about, and to justify, the choices that were made, providing details on what was excluded, so that subsequent researchers can replicate the research or review the conclusions. It is not just a matter of making evident the choices made. When researchers ignore an issue that is central to the research question, the conclusions must lose credibility. For example, there is much concern at present about weight gain in primary schoolchildren and programs have been devised to encourage children to walk or cycle to school. If the research has not taken account of the physical and social environment that may make walking and cycling impractical or unsafe, then its value is undermined. While editors make their own judgments, the peer review process helps identify those contextual factors that must enter into an account if a paper is to count as a substantial contribution to knowledge. Our commitment is to select reviewers from a diversity of relevant backgrounds to ensure that different views on the topic are considered. Researchers know that there are inequities in health and many are aware that we often fail to see the needs of those disadvantaged by class, race, gender, age or disability, as well as other factors operating in specific contexts. There is the expectation in public health that researchers should at least be conscious of not further increasing disadvantage. Some sociologists go further and see it as incumbent on the researcher to ‘represent the underdog’. This is easier said than done. When social or community groups are excluded and disadvantaged, the most common way in which this is effected is by silencing them. That complicates the task of the researcher. In the study conducted by Steel et al, we have a clear example of the difficulties of doing research under constrained circumstances. There is concern about the situation of people in the detention centres that house newly arrived refugees and people judged to be illegal immigrants; a particular concern has been the mental health of the children detained. It has been very difficult to gain research access to these children. Under such circumstances, data collection is necessarily compromised. What then is the test that is applied to assess the credibility of research? The first requirement is that the study must be done as well as it could be, given the circumstances. The second is that the study’s limitations should be fully disclosed. Those who question the results can then replicate the study, perhaps with the advantage of better access to study participants. The article by Steele et al met these requirements. Should we have required that the views of the authorities in the detention centre or government depar tments be represented as well? That, we believe, is a task for other researchers than those already facing substantial research challenges. The book on the rise of evidence-based medicine addresses a completely different situation and one more familiar to researchers using qualitative methods. Here balanced representation is complicated by a long history; there are extensive data sources, a growing literature and a large number of highly articulate potential research participants. There must be exclusions and only a fraction of the relevant material can be addressed in any one study. The troublesome question remains whether some less vocal contributors are unfairly overlooked in the book or, indeed, in evidence-based medicine itself, and whether this is linked to class, race or gender or even age or disability. It is certainly possible to write an account of evidence-based medicine that is focused on those groups whose views are overshadowed in the public discourse. Understanding the situation in which disadvantage manifests is an important research topic in its own right. On the other hand, it also seems justif iable to concentrate on a critical analysis of the rise of evidence-based medicine, concentrating on its public face, identifying disadvantage where it occurs without making that disadvantage the focus of analysis. As in the case of children in detention, this is a substantial task and, in time, there will be other complementary representations. In both these cases, the requirement is that the authors be alert to issues that compromise the scientif ic credibility of their studies and be clear about the practical and conceptual limitations of their research. Our task, as editors, is to ensure that competing, well justified representations also see the light of day.