Understanding Error Grid Analysis

The article entitled "Reservations on the use of error grid analysis for the validation of blood glucose assays" (1) presents a technically accurate description of error grid analysis (EGA), but fails to address the basic idea behind it. EGA is a clinically oriented nonparametric approach to blood glucose (BG) data, based on three assumptions: 1) BG readings <3.9 mmol should be raised, 2) BG readings >10 mmol should be lowered, and 3) acceptably accurate estimates are within 20% of the reference BG or when both the estimates and reference BG are <3.9 mmol. While the latter was the "standard of the day" (2), the upper and lower limits of acceptable BG were confirmed by the Diabetes Control and Complications Trial (3). The proposed "more effective" alternative is a generic parametric statistical technique described by Bland and Altman (4), which involves plotting the difference of each couple of values against their mean. It was reported that Moberg et al. (5) had compared the Bland-Altman approach with EGA. Actually, Moberg et al. compared EGA only with linear regression. We analyzed the data in Fig. 1 of Gough and Botvinick (1), using EGA and the Bland-Altman approach (4). We found that 46% of the estimates were accurate, as defined by the EGA, which is markedly less than the 95% we previously recommended for devices (6). Approximately 4% of hypoglycemic readings were not detected (potentially clinically dangerous overestimates in upper zone D), and ~8% of hyperglycemic readings were not detected (potentially clinically dangerous underestimates in lower zone D). In addition, EGA identified a possible systematic error of the measuring device: it tended to underestimate BG, having 62% of its readings in the lower zones. The Bland-Altman method correctly assessed the poor agreement between the estimate-reference values because of a 95% confidence interval for the deviations of the measuring device involving overestimates up to 8.6 mmol and underestimates of 6.6 mmol. However, the Bland-Altman approach has two major disadvantages: J) it does not provide objective criteria for agreement and 2) it relies on linear clinical equivalence across the entire BG range and suggests a data transformation when this equivalence is not accurate (4). We propose and validate a data transformation specific to the clinical nature of BG readings (34). The use of EGA was reported in different studies (1). However, this list (5,7-11) is incomplete, with there being many additional studies incorporating EGA (12-31). What is important to point out is that in all of these published studies the authors reported both EGA results and standard statistical results. Neither these, nor our own studies, assume that either EGA or standard statistical approaches are totally adequate. One approach is not more appropriate than the other, but rather they are complementary. Four specific issues are raised. We would like to address these separately.