My least squares line looks visibly tilted compared to the cloud of points — is the fit wrong?
I generated correlated test data, fitted an ordinary least squares line and plotted both. The line does not look like it goes through the middle of the cloud. It looks rotated, sitting shallower than the long axis of the scatter.
Out of curiosity I computed the principal axis of the data instead, and that line looks like what I expected the fit to be — it goes straight down the spine of the cloud.
So either the regression is doing something I have misunderstood, or my eyes are wrong about what a good fit looks like. Which is it, and if the regression is right, what is my eye responding to?
@residual_ruth · last wk.
Your eyes are fine and so is the regression. They are measuring different distances.
Ordinary least squares minimises the vertical distance from each point to the line — the error in y only, with x treated as known. Your eye, and the principal axis, minimise perpendicular distance, which treats the two directions symmetrically.
Those give different lines whenever the data is noisy, and the regression line is always the shallower of the two. That is not a defect; it follows from the question being asked. "Given this x, what is my best guess for y" is not symmetric in x and y, so its answer should not be either.
The visual check that makes this click: draw the vertical segment from each point to the line rather than the perpendicular one. Under that view the regression line is obviously centred and the principal axis obviously is not.
Reply
Report