Linear regression and correlation

I n 1805, the French mathematician Adrien-Marie Legendre published “New Methods for the Determination of the Orbits of Comets,” which included a powerful and elegant method for fitting equations to data called the method of least squares. The method was based on the assumption that the best fit of an equation to data is the one that minimizes the average squared distance of the data points from the fitted equation line. Four years later, Carl Friedrich Gauss published a mathematically rigorous version of Legendre’s technique and provided a philosophical justification for it in the idea and mathematics of the normal distribution. The modern catch-all term for “least-squares” analyses and analogous techniques for fitting models to data is regression, appropriated from Francis Galton’s 1886 work “Regression Towards Mediocrity in Hereditary Stature.” This paper described 930 children of 205 adult parents for whose heights Galton was able to fit trend lines and demonstrate the tendency of extreme heights in families to revert to population means over generations. Modern biostatistics depends heavily on the method of least squares and the regression methods derived from it. Simple linear regression is the most basic of these techniques and the most important to understand in biomedical sciences. This short article will describe linear regression and its sister concept, correlation.