Premium
Revisiting a Statistical Shortcoming when Fitting the Langmuir Model to Sorption Data
Author(s) -
Bolster Carl H.
Publication year - 2008
Publication title -
journal of environmental quality
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.888
H-Index - 171
eISSN - 1537-2537
pISSN - 0047-2425
DOI - 10.2134/jeq2007.0461
Subject(s) - sorption , nonlinear regression , statistics , regression analysis , errors in variables models , linear regression , regression , mathematics , variables , non linear least squares , variable (mathematics) , chemistry , explained sum of squares , adsorption , mathematical analysis , organic chemistry
The Langmuir model is commonly used for describing the sorption behavior of reactive solutes to surfaces and is often fit to sorption data using nonlinear least squares regression. An important assumption of least squares regression is that the predictor variable is error free. In the case of sorption data, this assumption is not valid, and therefore the potential for parameter bias exists. Although alternative regression methods exist that either explicitly account for error in the predictor variable (Model II regression) or minimize the error in the predictor variable, these methods are not commonly used. Therefore, this paper more fully explores the differences in fitted parameters and model fits between these different data fitting methods by fitting P sorption data collected on 26 different soil samples using three different regression methods. For a majority of soils tested in this study, the differences in model fits between the three regression methods were not statistically significant. Statistical differences were observed in over a third of the soils, however, suggesting that errors in the predictor variable may be large enough to produce biased parameter estimates. These results suggest that multiple regression methods should be used when fitting the Langmuir model to sorption data to better assess the potential impact of error on model fits.