Ridge Regression

I Studied Regressions *Only* for 30 Days So You Don’t Have To But You Have To Subscribe

I Studied Regressions *Only* for 30 Days So You Don’t Have To But You Have To Subscribe Part II

Chief Ridge Officer strikes back

I fell out of love with LASSO. it’s not you it’s me baby :(

why statsmodels >>> sklearn

The linear regression model cannot be fitted to high-dimensional data, as the high-dimensionality brings about empirical non-identifiability. Penalized regression overcomes this non-identifiability by augmentation of the loss function by a penalty (i.e. a function of regression coefficients). The ridge penalty is the sum of squared regression coefficients, giving rise to ridge regression. Here many aspect of ridge regression are reviewed e.g. moments, mean squared error, its equivalence to constrained estimation, and its relation to Bayesian regression. Finally, its behaviour and use are illustrated in simulation and on omics data. Subsequently, ridge regression is generalized to allow for a more general penalty. The ridge penalization framework is then translated to logistic regression and its properties are shown to carry over. To contrast ridge penalized estimation, the final chapters introduce its lasso counterpart and generalizations thereof.

I developed the lecture notes based on my ``Linear Model″ course at the University of California Berkeley over the past seven years. This book provides an intermediate-level introduction to the linear model. It balances rigorous proofs and heuristic arguments. This book provides R code to replicate all simulation studies and case studies.