Blog

optimisationseriesJul 2026

The Lasso and MM Algorithms

The lasso is one of the most familiar examples of coordinate descent in statistical learning. This five-part series develops an alternative optimization framework based on majorize-minimize algorithms. Starting from the lasso KKT conditions, we derive an equivalent auxiliary problem, a separable MM update, and an adaptive choice of weights that can substantially improve convergence.