Linear Regression (OLS)

Standard regression for continuous, normally distributed Y — model $Y = β_{0} + \sum β_{j} X_{j} + ε$ , estimated by least squares.

Also known as: OLS, least squares, multiple linear regression

Linear regression (Ordinary Least Squares, OLS) is the classical regression model. It minimises the sum of squared residuals — the $\hat{β}$ are chosen so that predictions land as close as possible to observations.

\hat{β} = (X^{⊤} X)^{- 1} X^{⊤} y

Simple — one X, one Y: $Y = β_{0} + β_{1} X + ε$
Multiple — several X: $Y = β_{0} + β_{1} X_{1} + β_{2} X_{2} + \dots + ε$
Assumptions — linearity, independent observations, constant error variance (no heteroscedasticity), normal residuals

Used in

Regression Analysis
DoE Planner
Correlation Analysis
MSA Type 4 (linearity)

In the Algorithm Lab

regression/regression-coefficients
regression/regression-anova

Sources

Montgomery, D. C., Peck, E. A. & Vining, G. G. — Introduction to Linear Regression Analysis, 6th Edition
Draper, N. R. & Smith, H. — Applied Regression Analysis, 3rd Edition, Wiley

Linear Regression (OLS)

See also

Used in

In the Algorithm Lab

Sources