Logistic Regression (Binary)

GLM for binary responses (0/1, pass/fail) — models the probability $P (Y = 1∣ X)$ via the logit link.

Also known as: logistic regression, logit regression, binary logistic regression

Binary logistic regression is the GLM for 0/1 responses. Instead of modelling Y directly, the logit of the success probability is tied linearly to X.

logit (p) = ln (\frac{p}{1 - p}) = β_{0} + β_{1} X_{1} + \dots + β_{p} X_{p}

Back-transformed this gives the sigmoid response curve:

P (Y = 1 ∣ X) = \frac{1}{1 + e ^{- (β_{0} + β_{1} X_{1} + \dots)}}

Coefficients are read as odds ratios: $exp (β_{j})$ = OR per unit of $X_{j}$
Model fit via pseudo- $R^{2}$ (McFadden, Nagelkerke), AUC/ROC, Hosmer-Lemeshow test
Watch for separation — a perfectly separating predictor drives coefficients to infinity

Used in

Regression (Attributive)

In the Algorithm Lab

Sources

Hosmer, D. W., Lemeshow, S. & Sturdivant, R. X. — Applied Logistic Regression, 3rd Edition, Wiley
Agresti, A. — Categorical Data Analysis, 3rd Edition, Wiley