Majorize-minimize is a general recipe for designing algorithms rather than a single update formula. Its surrogate viewpoint includes EM and connects to proximal-gradient and Newton-like quadratic-majorization schemes; see Hunter and Lange (2004) and Lange (2016). The recipe is simple: replace a difficult objective by an upper bound that touches it at the current iterate, improve that surrogate, and repeat.
To minimize an objective $g(\v\gamma)$, an MM algorithm replaces it at the current iterate $\widetilde{\v\gamma}$ by a simpler surrogate $h(\v\gamma,\widetilde{\v\gamma})$ satisfying
$$ \begin{aligned} g(\widetilde{\v\gamma})&=h(\widetilde{\v\gamma},\widetilde{\v\gamma}), \\ g(\v\gamma)&\le h(\v\gamma,\widetilde{\v\gamma}) \quad\text{for every }\v\gamma. \end{aligned} $$Minimize the surrogate and repeat. If $\v\gamma^{(t+1)}\in\argmin_{\v\gamma}h(\v\gamma,\v\gamma^{(t)})$, then
$$ \begin{aligned} g(\v\gamma^{(t+1)}) &\le h(\v\gamma^{(t+1)},\v\gamma^{(t)})\\ &\le h(\v\gamma^{(t)},\v\gamma^{(t)}) =g(\v\gamma^{(t)}). \end{aligned} $$Consider the smooth one-dimensional objective
$$ g(x)=\cos x,\qquad 0\le x\le2\pi. $$On this interval the unique minimizer is $\widehat x=\pi$, where $g(\widehat x)=-1$. Because $g''(x)=-\cos x\le1$, the quadratic upper-bound principle gives the smooth majorizer
$$ h(x,x^{(t)}) =\cos x^{(t)}-\sin x^{(t)}(x-x^{(t)}) +\frac12(x-x^{(t)})^2 \;\ge\;\cos x. $$The bound touches $g$ and has the same slope at $x^{(t)}$. Its minimum gives the closed-form MM update
$$ x^{(t+1)}=x^{(t)}+\sin x^{(t)}. $$Starting at $x^{(0)}=0.2$ gives a sequence long enough to watch: $0.2000\to0.3987\to0.7869\to1.4950\to2.4921\to3.0969\to\cdots\to\pi$. With $L=1$ the iterates reach $\pi$ to display precision after seven MM updates.
The curvature constant $1$ is the smallest value that works, but any $L\ge1$ also gives a valid majorizer,
$$ h_L(x,x^{(t)}) =\cos x^{(t)}-\sin x^{(t)}(x-x^{(t)}) +\frac{L}{2}(x-x^{(t)})^2, \qquad x^{(t+1)}=x^{(t)}+\frac{\sin x^{(t)}}{L}. $$The curvature selector under the animation switches between $L=1$, $2$ and $4$. Every choice descends monotonically, but a looser bound hugs the objective less closely, so each step is shorter and many more iterations are needed to reach $\pi$. Validity is cheap; tightness is what buys speed. That is exactly the tension Parts 4 and 5 resolve for the lasso.
Orange is the objective; the dashed peach curve is the surrogate at the current iterate $x^{(t)}$ (white dot), and the peach guide marks its minimum, the next iterate. Earlier iterates stay on the objective so the monotone descent is visible. The guide at $\widehat x=\pi$ marks the true minimum.
Majorization guarantees descent, but it does not guarantee speed. The choice of majorizer is therefore both a mathematical and computational decision. Common constructions use tangent inequalities for concave terms, quadratic curvature bounds for smooth terms, Jensen's inequality for latent-variable objectives, or matrix inequalities for coupled inverse problems.
Part 2 left us with the convex auxiliary objective
$$ \begin{aligned} g(\v\gamma) &=\v y^\top \left(\m I+\sum_{j=1}^p\gamma_j\v x_j\v x_j^\top\right)^{-1}\v y\\ &\quad+\alpha^2\v 1^\top\v\gamma. \end{aligned} $$The inverse couples all $p$ scales, so $g$ is not separable. The design problem is now concrete: find an upper bound that touches this inverse term at the current iterate, separates across $\gamma_j$, and remains tight enough to make a useful concurrent step.