Erroneous efficiency reports harm organic solar cell research

The efficiencies of both organic and hybrid (organic–inorganic) solar cells have increased markedly in recent years, making them an exciting and commercially valuable prospect. Despite this progress, a literature survey that we have performed suggests that a large number of reported solar cell efficiencies are being incorrectly measured and reported in journals, with the device efficiency being overestimated in many cases. Our survey of 375 papers in 13 journals, which compared published values of external quantum efficiency with the measured short-circuit current density, shows that over 37% of publications report data or make claims that overestimate the efficiency of solar cells. Such a frequent occurrence of incorrect data-reporting risks damaging the credibility and reliability of the field. Thus, it is imperative that the community ensures that articles are published with correct efficiency values. This is only possible if responsible data collection and analysis follows widely-recognized methodologies or can be traced to accredited standard laboratories. Recently, there have been urgent calls to 'clean up' solar cell measurements and correct measurement procedures have been clearly documented1, 2, 3, 4. Common sources of measurement errors have been described and simple methods have been proposed to avoid such errors. Nevertheless, the number of publications reporting questionable or clearly incorrect efficiencies is still substantial. This negligence is probably due in large part to the following two factors: (1) proper characterization requires substantial care and some researchers are not always prepared to commit the required time and effort as long their research results will be published regardless; (2) it is convenient to continue with erroneous measurements especially if they yield higher efficiency values and increase the chance of publication of results in high-impact journals, which typically demand state-of-the-art efficiencies. The hunt for record efficiencies can be dangerous and counterproductive in the absence of an incentive to carefully and correctly measure solar cell efficiencies. Still, one of the most common errors is to report measurements without using a shadow mask, which can easily increase the efficiency value by 30% or even more5, 6. Even though this error can be easily fixed experimentally7, it seems that the driving force to do so is currently not strong enough. It is often not mentioned in papers whether solar cell efficiencies have been measured with or without a mask. Some researchers report using no mask, even though measurements in its absence cannot be trusted and do not carry any significant value. Some authors also now mention an efficiency value measured with and without a mask. For example, in the abstract of their Letter in Nature, Chung et al.8 report an efficiency of 10.2% without and 8.51% with a mask. The choice of the spectral response of the reference cell, and the homogeneity of the light output by the solar simulator, also play a critical role that can cause under- or overestimation of the device efficiency3, 9. Devices and calibration cells should always be placed at the same position to avoid such errors. For accurately monitoring progress in the field, correct measurements are essential and wrong values can severely mislead the research community. The power conversion efficiency is often taken as the key parameter for judging solar cell performance. Often the significance of research results — as judged by editors and reviewers — is primarily on the device efficiency achieved rather than scientific value. It is possible to make considerable advances in solar cell research using a system that does not achieve the highest efficiencies. On the other hand, it is possible to find published papers in high-impact journals where a reported record efficiency seems to be the sole progress — sometimes based on slightly improved and very specific processing conditions or improved material sources or, in the worst case, on errors in the power conversion efficiency measurement. We believe that a paradigm shift is required, otherwise data in the field will become increasingly unreliable and progress will be hampered. Only the attention of authors, referees and journal editors can change the situation. It needs to be better recognized that new scientific insights and a deeper understanding of material properties and physical mechanisms in solar cells is the basis for the future development of devices with increased efficiency. This understanding can only be gained in well-designed experiments and often requires the fabrication of reproducible and reliable devices, which in turn do not always exhibit record efficiencies. Therefore, device efficiency should not be the only (or even a major) criterion for the assessment of the quality of a manuscript. Frequently, as a referee of a manuscript, it is impossible to judge whether the efficiency has been measured correctly. However, we suggest that prior to publication it should be confirmed that the rules outlined in ref. 3 have been followed and that all mentioned errors have been avoided (see Box 1 for a list of points to consider when assessing the validity of solar cell data). Authors sometimes tend to select the data that best fits their assumptions. Although the reproducibility of devices is challenging to achieve, selectively picking agreeable data is not a valid practice. As correctly urged by Luber and Buriak10, a careful analysis of the data — including a statistical analysis — is required to verify the significance of experimental results. However, if this is not applicable, the authors should at least report how many devices they measured, how the efficiency values are distributed and how they selected the data presented in the manuscript. The authors should always demonstrate at least a second independent batch that shows the same trend. Finally, care should be taken when judging a manuscript solely on the reported efficiency. Record efficiencies can be important as they chart progress in the field. However, it is essential that the reported efficiency values are correct. The safest procedure would be to have such solar cells certified by an accredited standards lab like the National Renewable Energy Laboratory (NREL) in the USA, the National Institute of Advanced Industrial Science and Technology (AIST) in Japan, the Fraunhofer Institute for Solar Energy Systems (ISE) in Germany, or the company Newport, which also offers such a service. For inorganic solar cells, measurement standards are routinely applied and the process of recording tables of certified record efficiencies has been in place for decades. More recently, organic and hybrid solar cells have also been included in these tables. For example, the NREL is updating its table of record efficiencies to include organic and hybrid solar cells11. The solar cell efficiency tables that are published regularly by Green et al.12 are another source of record efficiencies. To be included in these published tables, values must be independently certified and, for the latter, there is an obligatory minimum cell area that differs for the different device types. This is because the size of the measured cell plays an important role — measurement errors increase as the active cell area gets smaller. The size of cells should always be provided and the procedure for measuring the efficiency should be clearly stated in the experimental section or supplementary information of the publication. The description of the experimental methods needs to be detailed enough to allow other researchers to reproduce the results. Often, a discrepancy can be detected when comparing the external quantum efficiency (EQE) — also called the incident photon-to-current conversion efficiency (IPCE) — with the measured short-circuit current density (JSC) of the cell. In general, the convolution of the EQE with the solar spectrum should give the JSC. However, a discrepancy between measurements of the JSC and the EQE can be explained by the difference in illumination conditions. In the case of the EQE, the photoresponse is measured at a single wavelength, whereas broadband light with a full spectrum is used for measurements of the JSC. Nevertheless, this check still gives a good indication of whether or not the JSC has a realistic value. Most organic and hybrid solar cells have a conversion efficiency relationship with light intensity that is nonlinear and falls off as the intensity increases. Cell performance typically peaks at around 10%–50% of 1 sun illumination (1 kW m−2). As the EQE is usually measured under lower light intensities, it can be expected that the measured JSC is less than or equal to the value calculated from the EQE measurements, but not higher. Although this is the usual case, there are some exceptions (for example, ultraviolet light activation in hybrid solar cells can increase the efficiency). Nevertheless, we believe that comparing the expected JSC from EQE with the measured JSC is a simple, valid and useful method for cross-checking measurements. In most cases, it can be expected that less care is taken with EQE measurements than with measurements of the current–voltage (J–V) curve, as more attention is often paid to the latter. Furthermore, it seems improbable that a researcher will use a mask for one measurement, but not for the other. However, due to the different set-ups, the measurement errors will be different in each case. For instance, the size of the light spot coming through a monochromator in an EQE set-up is much smaller than the light spot coming from a solar simulator. This means that a measurement error (due to, say, a missing mask) is likely to be much smaller in the recorded EQE data than in the J–V data. Therefore, we believe that a comparison of the expected (calculated) JSC value derived from EQE data with the JSC determined by J–V measurements using a solar simulator is a valid way of determining if a paper contains questionable efficiency values. The authors should declare the reasons for any significant discrepancy between these values, for example, by showing that the cell has a strong dependence on illumination intensity. To quantify the frequency of questionable published efficiencies, we have performed an analysis of the data in a total of 375 papers including 1,262 J–V and EQE curves published during the period 2011–2012 (see Box 2 for details of the analysis). In each case we calculated the JSC from the reported EQE data and compared it with the reported measured values of JSC. We set a threshold for mismatch at 20% (where measured JSC is at least 20% higher than the value of JSC calculated from the EQE), on the basis that a larger discrepancy can only be explainable by device-specific behaviour in very rare cases. In around two-thirds (62.5%) of the cases the calculated discrepancy is below this 20% threshold (the reported measured JSC was lower than the calculated JSC in only 16.2% of the cases). However, worryingly, the results show that in 37.5% of the published papers one or more reported value of JSC exceeds the discrepancy threshold (Fig. 1). More than 30% of these JSC values are in a discrepancy range of 20–30%, which can be expected when measuring cells without a mask and might be related to this effect. Several publications — 18% of the total and 47% of the publications with reported values exceeding the threshold (including some highly cited papers) — have systematic discrepancies as large as between 40% and 200%. Furthermore, about 13% of the investigated JSC values have discrepancies of more than 100%, with the largest at 490%. Although it is more probable to overestimate the current of low-performance solar cells, this also occurs in cells with high currents. Authors do not usually mention this discrepancy and do not give any explanation for it. Despite these papers being peer-reviewed, the discrepancies seem to have been missed during the review process. In summary, our analysis suggests that a highly significant fraction of publications in the field of organic solar cell research are reporting efficiency values that are questionably and suspiciously high (or the authors present non-contiguous plots). This is a dangerous trend, which risks misleading and discrediting the entire field. It should be noted here that questionable values seem to be reported not only in low- but also in high-impact scientific journals (Fig. 2). As already pointed out by Potscavage13, an extreme example of this behaviour is included in ref. 14, where the authors report an improvement of a cell leading to a JSC of 38.71 mA cm−2. The value alone should already make the reader very suspicious, as it is extremely high and would lead to an efficiency greater than the Shockley–Queisser limit if the cell's fill factor was increased to 0.8 with an ideal open-circuit voltage (VOC). The EQE curve for this device is not exceptionally high and a quick calculation shows that a JSC value of 7.6 mA cm−2 can be expected. In a reply to this comment, the authors have admitted to using a mask larger than the pixel size and have since revised their JSC claims to 24.2 mA cm−2 (ref. 15), which still seems unrealistically high. In other cases — notably for cells featuring TiO2 — the J–V curve can show some hysteresis, depending on the scanning direction and speed of the voltage during J–V measurements. The reason for this effect is a slow reaction time of the cell — slower than the sweeping delay time due to a number of possible effects (such as strong capacitive effects). To avoid this, in some cases it is necessary to use sweep delay times on the order of seconds rather than milliseconds when recording the J–V curve16. Often in perovskite-based solar cells an even stronger hysteresis effect can be observed. Here, a longer sweeping delay time does not necessarily help to fully overcome the hysteresis17. Therefore, it is important to find a consistent and reliable measurement rule for characterizing such cells. For example, it seems necessary to ensure stabilization of the current close to the maximum power leading to a stabilized power output value, allowing comparability of results from different research groups, as suggested by Snaith et al.17. Ideally, in the case of unexpected results that claim very high numbers and are difficult to explain, other research groups should attempt to reproduce the work to build confidence in its validity. However, for such replication to be feasible it is important that papers include a sufficiently detailed methods or supplementary information section, which does not omit any important information on the material synthesis, solar cell fabrication, device processing and measurement procedures. As organic and hybrid solar cells have exceeded 10% efficiencies (with some reports of close to 18% efficiency for perovskite cells), it is important to have strict standards in place for solar cell characterization, as is the case for inorganic photovoltaics. From an industrial point of view, high solar cell efficiencies are only significant if they are transferable to industrial processes and real-world applications. Currently, devices typically have active areas much smaller than 1 cm2 and only such small devices seem to give impressive efficiency results. Even though small cell sizes are acceptable for general device-analysis, it becomes problematic when announcing record efficiencies. Certainly any measurement error will have a much higher effect on smaller cells than on larger cells. Therefore, we suggest that record efficiencies should always be accompanied by a cell size ideally 1–2 cm2 or larger. The progress over the past few years in the field of organic and hybrid solar cells has been tremendous. As this emerging technology starts to become industrially viable, it is important to carefully check if reported device efficiencies are measured correctly and that they can be reproduced with device sizes of 1–2 cm2. Researchers, editors and reviewers all have a role to play in improving the current situation. If nothing is done, the field will lose its credibility and progress will continue to be hindered by misleading reported results. Download references