Yes but when the result is always skewed to one side then - even if the result is within the margin - the predicted mean is wrong.

Otherwise the real result would be distributed around the mean within the margin of error.

There is some bias and the polls did not correctly factor that into their statistical model.