English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Given that even elementary issues like simple statistical testing have engendered no consensus in their 300-year history, how could we expect agreement about more subtle issues? Without consensus on the basics, it should be no surprise that such a complex topic as multiple comparisons is so widely misunderstood and in conflict. Sjölander and Vansteelandt are leading contributors to epidemiologic statistics, with many fine works to their credit. Their article [1], hereafter SV (singular), ascribes aspects of multiple comparisons (MC) controversies to frequentist statistics, and advises using no formal MC adjustments, instead relying on “informal, qualitative” judgments to deal with multiplicity problems. That advice may be a pragmatically defensible heuristic in settings in which the analysis targets only one focused research question represented by a few closely related statistical hypotheses or parameters, and all analyses and estimates are reported with equal emphasis and detail. In these simple cases it is possible to hold in mind all relevant associations and their interactions and uncertainties at once, and accusations of “fishing,” “hacking,” or “dredging” can be deflected. Unfortunately, SV appears to propose such “informal adjustment” as a general compromise for all MC analysis, when it is not really a compromise at all; it is too close to repeating the old extreme advice to do no formal MC adjustment [2]. That advice remains common but is unwise when there are several interdependent parameters or hypotheses in the analysis (as in the studies of composite exposures or multiple outcomes that typify occupational, environmental, and nutritional studies), because their dependencies can be used to dramatically improve the overall accuracy of the multiple results [3–12]. Worse, providing no formal adjustment is likely to be very misleading when the setting is highly exploratory, aiming to inform decisions about which of many weak possibilities to pursue with focused efforts, and it is wildly impractical in large-scale searches [13–16]. In those settings multiplicity problems are not only real, but far beyond the capabilities of human intuition to grasp without applying appropriate MC adjustments to gauge their impacts. We will here outline these problems and the extensive developments that neither SV nor their opponent [17] describe, but which are pivotal for MC debates and adjustments. We will emphasize once more that hierarchical (multilevel) regression enables one to exploit dependencies among tests and parameters or hypotheses to improve statistical summaries used for reporting and decision-making [7, 8, 12]. We will also emphasize the central importance of decision costs (utilities or loss functions) implicit in every statistical method that claims to produce a conclusion or decision. Much of modern statistics abuse and controversy stems from these loss functions being ignored in typical research applications, even though every conclusion and decision is infinitely sensitive to error costs. As discussed below and in more detail elsewhere [10], the problems from this neglect are intensified in the MC controversy. The present essay will begin by explaining how these problems have nothing to do with differences between frequentist and Bayesian methodologies and everything to do with contextual justification of statistical procedures (algorithms) and valid interpretation of their outputs. Here, validity is judged by pragmatic criteria that represent a mix of * Sander Greenland lesdomes@ucla.edu

Multiple comparisons controversies are about context and costs, not frequentism versus Bayesianism