Controlling the False-Discovery Rate in Astrophysical Data Analysis

The False Discovery Rate (FDR) is a new statistical procedure to control thenumber of mistakes made when performing multiple hypothesis tests, i.e. whencomparing many data against a given model hypothesis. The key advantage of FDRis that it allows one to a priori control the average fraction of falserejections made (when comparing to the null hypothesis) over the total numberof rejections performed. We compare FDR to the standard procedure of rejectingall tests that do not match the null hypothesis above some arbitrarily chosenconfidence limit, e.g. 2 sigma, or at the 95% confidence level. When using FDR,we find a similar rate of correct detections, but with significantly fewerfalse detections. Moreover, the FDR procedure is quick and easy to compute andcan be trivially adapted to work with correlated data. The purpose of thispaper is to introduce the FDR procedure to the astrophysics community. Weillustrate the power of FDR through several astronomical examples, includingthe detection of features against a smooth one-dimensional function, e.g.seeing the ``baryon wiggles'' in a power spectrum of matter fluctuations, andsource pixel detection in imaging data. In this era of large datasets and highprecision measurements, FDR provides the means to adaptively control ascientifically meaningful quantity -- the number of false discoveries made whenconducting multiple hypothesis tests.Comment: 15 pages, 9 figures. Submitted to A