Optimal detection of heterogeneous and heteroscedastic mixtures

Summary.  The problem of detecting heterogeneous and heteroscedastic Gaussian mixtures is considered. The focus is on how the parameters of heterogeneity, heteroscedasticity and proportion of non‐null component influence the difficulty of the problem. We establish an explicit detection boundary which separates the detectable region where the likelihood ratio test is shown to detect the presence of non‐null effects reliably from the undetectable region where no method can do so. In particular, the results show that the detection boundary changes dramatically when the proportion of non‐null component shifts from the sparse regime to the dense regime. Furthermore, it is shown that the higher criticism test, which does not require specific information on model parameters, is optimally adaptive to the unknown degrees of heterogeneity and heteroscedasticity in both the sparse and the dense cases.