Relationship between simple linear regression and sample mean, correlation, standard deviation.

Regression equation:

yᵢ = α + βxᵢ + εᵢ, i = 1, 2, ..., n

(where εᵢ represents random noise)

Then the least squares estimator of β is

b = Cor(x, y) * Sd(y) / Sd(x),

and the estimator of α is

a = Mean(y) - b * Mean(x)

(where Mean, Sd, Cor and the sample mean, sample standard deviation, and the sample Pearson correlation coefficient respectively)

0-fold cross-validation@0foldcv@mathstodon.xyz...An equivalent relationship holds in terms of random variables.

Relationship between simple linear regression and moments of random variables.

Regression equation:

yᵢ = α + βxᵢ + εᵢ, i = 1, 2, ..., n

(where εᵢ represents random noise)

If xᵢ is a random variable that is independently and identically distributed (i.i.d.) for all i=1,2,...,n, and εᵢ (also i.i.d.) has mean 0 and is independent of xᵢ. Then:

β = Cov(xᵢ, yᵢ) / Var(xᵢ)

= Cor(xᵢ, yᵢ) * Sd(yᵢ) / Sd(xᵢ),

α = E(y) - βE(x),

#stats