Opinion: Promoting open science

Many scientific fields are facing a reproducibility crisis, revealed where replication fails to reproduce findings from previous work. This irreproducibility leads to the promulgation of inappropriate evidence. Journals have a tendency to publish positive, statistically significant results. This creates a skewed evidence base and encourages questionable research practices such as p-hacking and HARKing. p-Hacking is the manipulation, analysis and reporting of data to obtain a significant pvalue. This can often be done with good intentions under the belief that analytical flexibility provides greatest insight into the data collected. The flexibility of available design and analytical choices makes it possible, however, to produce significant p-values in almost any situation. Researchers may be tempted to hypothesise after the results are known (HARK) – proposing hypotheses as though they predicted the results a priori – to support a result obtained through selective analysis. This confuses the distinction between confirmatory and exploratory research. p-Hacking and HARKing are more likely when studies are underpowered. The low statistical power from studies using small samples reduces the chance of finding a significant effect, increasing the temptation to p-hack or HARK to obtain a publishable result. Low power also inflates the size of any effects that are found. The reproducibility crisis in science has also been labelled a credibility revolution as it offers an opportunity to improve how research is done. Embracing the principles of open science can increase reliability and transparency. Publishing research designs and analytical plans before collecting data by preregistering studies creates greater transparency and credibility in the end results. Preregistrations can be incorporated into the peer-review process through the Registered Report format of journal article. Openly publishing data, analysis code and research materials through free-to-use services like the Open Science Framework enables greater scrutiny of research and allows others to attempt to reproduce the results. Open publication of research materials also supports development of replication studies that aim to confirm the findings of previous studies by using high-powered designs and clearly specified methods that match the original study. Replication studies are essential for developing cumulative knowledge. A priori power analysis is rare in lighting research. Power analysis prior to data collection ensures adequate sample sizes are used, giving studies the best opportunity to address the research questions. Adequately powering studies can also reduce temptations to p-hack and HARK and increase the positive predictive value of studies, meaning if a significant result is found it is more likely to be true. Lighting research is not immune from the reproducibility crisis and the problems it highlights. The open science movement offers an opportunity for researchers to embrace research practices that should improve scientific quality and credibility.