Statistics, ethics and probiotica

Ethical issues involved in the design of the ‘PROPATRIA’ probiotica trial are discussed. This randomized clinical trial appeared to be well conducted according to accepted good practices. The finding that the treatment was actually rather harmful, and that despite this, and despite a built‐in interim analysis, the trial was not stopped earlier, led to strong criticism in the media. I argue that ‘accepted good practices’ need to be reconsidered in the light of this experience. First, a much stronger distinction needs to be recognized between the immediate interests of the patients being treated in the trial and the interests of future patients of future doctors elsewhere. Secondly, it is in the interests of future patients that well‐conducted clinical trials are accepted by society. As it is unavoidable that an occasional trial will result in an unpredicted severely negative outcome, ethical screening committees must ensure that those performing a trial can never be accused of putting the interest of ‘science’ above the interest of their own patients when such ‘accidents’ happen. There are two consequences of this. First, the design of a trial should also explicitly lead to minimizing the number of patients who are treated by the researchers with a potentially seriously harmful medicine. Secondly, the disadvantages of triple‐blinding far outweigh the advantages. Although it might at best only have saved a few lives if the PROPATRIA trial been re‐designed with these issues in mind, I argue that the scientific value of the trial would not have been significantly reduced; the damage to medical research, and hence to future patients, would have been substantially less. Closer inspection of the data from the PROPATRIA trial brings a new and quite unexpected failing to light. The decision for stopping the trial early was accidentally based on the one‐sided test looking in the wrong direction, partly through the inadequacy of the output of the statistical package, SPSS and partly through lack of statistical expertise on the part of the users. If the envisaged one‐sided stopping rule had been used correctly, the trial would in fact have been terminated at the time of the interim analysis ‘for futility’; it was at this moment highly unlikely that a significant end‐result in favour of probiotica was going to be attained. The decision to continue the trial was a result of looking at the test statistic ‘in the wrong direction’. In effect, the trial was continued because there was still a good chance to show that probiotica is actually very harmful. I recommend that data‐monitoring committees should always be advised by a professional statistician, who is not blinded to the treatment allocation.