Why the Qualitative Health Research (QHR) Review Process Does Not Use Checklists

Using checklists in manuscripts are perceived to indicate quality, transparency, and rigor. Generally, these checklists consist of a list of all of the strategies that may be used to ensure rigor and transparency. Beside each item, there is usually a box to check (or tick) to indicate whether a component is present, and a space on which to note the page each item is listed in the manuscript. Some of these forms also include space for the author to make brief comments to the reviewer. The intent is that the checklist guides the review process to ensure that all components are present in the article, and therefore, that the article is solid enough to publish. However, these checklists consist only of technical/mechanical management of the creation and sorting of data. These lists ignore the value of the product of the research: They do not address the originality, the substance, the contribution, and the potential results to the actual topic—which is after all the purpose of the project itself. Paradoxically, these checklist reviews are undermining the quality of qualitative inquiry. In seeking quality, the criteria for systematic reviews, clinical trials, and evidence have spilled over to represent quality criteria for all qualitative research. They are becoming commonplace for evaluating qualitative research by journal editors, directing the review process, and subsequent evaluation of the research. Of greatest concern is that checklists items are being used by authors themselves to represent their actual text (e.g., “data were saturated”), and the items for completing these forms are read by the reviewers and editors in lieu of reading the article itself (e.g., for signs of “saturation”). Furthermore, the use of these criteria by authors/researchers to guide the conduct of their research, yet meeting all these criteria, whether relevant or pertinent or necessary for their project, and may even invalidate the findings. In this way, these criteria are redefining processes of qualitative inquiry.