The central design choice in an MM algorithm is the surrogate. A valid majorizer guarantees descent, but the speed of convergence depends strongly on how tightly it follows the objective near the current iterate. For $g(\v\gamma)$, the coordinates are coupled inside a matrix inverse. A matrix Cauchy-Schwarz inequality gives a tangent upper bound that separates into $p$ scalar problems with closed-form updates. Interestingly, the construction introduces a simplex weight vector $\v w$: each choice of $\v w$ gives a different valid surrogate.
Titu's lemma, also known as Sedrakyan's inequality, is the following form of Cauchy-Schwarz. It holds for real \(\widetilde\sigma_j\) and strictly positive \(\sigma_j\):
$$ \frac{(\sum_j\widetilde\sigma_j)^2}{\sum_j\sigma_j} \le \sum_j\frac{\widetilde\sigma_j^2}{\sigma_j}. $$For the matrix analogue, let $\m V_1,\ldots,\m V_m$ be symmetric positive-semidefinite matrices, take $\v\sigma\in\bb R_{++}^m$, and take $\widetilde{\v\sigma}\in\bb R_+^m\setminus\{\v0\}$. In addition, require both weighted sums $\sum_j\sigma_j\m V_j$ and $\sum_j\widetilde\sigma_j\m V_j$ to be positive definite, so that the ordinary inverses below exist. Then
$$ \left(\sum_j\sigma_j\m V_j\right)^{-1} \preceq \left(\sum_j\widetilde\sigma_j\m V_j\right)^{-1} \left(\sum_j\frac{\widetilde\sigma_j^2}{\sigma_j}\m V_j\right) \left(\sum_j\widetilde\sigma_j\m V_j\right)^{-1}. $$Equality holds when $\sigma_j=\widetilde\sigma_j$, which is exactly the tangency an MM surrogate needs.
Let $\v w\ge\v0$ satisfy $\v1^\top\v w=1$. Because each standardized predictor has $\|\v x_j\|_2=1$, the matrix $\m I-\v x_j\v x_j^\top$ is a projection. Therefore
$$ \m G^{-1}(\v\gamma) =\sum_{j=1}^p(\gamma_j+w_j)\v x_j\v x_j^\top +\sum_{j=1}^p w_j(\m I-\v x_j\v x_j^\top). $$Apply the matrix inequality at the current iterate $\widetilde{\v\gamma}$. With $\widetilde\tau_j:=|\v x_j^\top\m G(\widetilde{\v\gamma})\v y|$ and $\widetilde r:=\|\m G(\widetilde{\v\gamma})\v y\|_2$, the resulting surrogate is
$$ h_{\v w}(\v\gamma,\widetilde{\v\gamma}) =\sum_{j=1}^p\left[ \frac{(\widetilde\gamma_j+w_j)^2\widetilde\tau_j^2}{\gamma_j+w_j} +w_j(\widetilde r^2-\widetilde\tau_j^2) +\alpha^2\gamma_j \right]. $$Boundary convention. When $w_j=\widetilde\gamma_j=0$, the corresponding term is understood by continuous extension. This avoids a meaningless $0/0$ and leaves an already-zero coordinate unchanged.
The variables now separate. Minimizing the bound over each $\gamma_j\ge0$ gives
$$ \gamma_j^{(t+1)} =\frac{1}{\alpha} S_{\alpha w_j}\!\left((\gamma_j^{(t)}+w_j)\tau_j^{(t)}\right), \qquad j=1,\ldots,p. $$One solve for $\m G(\v\gamma^{(t)})\v y$ supplies every $\tau_j^{(t)}$, and all $p$ coordinates then update together. If $w_j=0$, the rule becomes multiplicative:
$$ \gamma_j^{(t+1)} =\gamma_j^{(t)}\frac{\tau_j^{(t)}}{\alpha}. $$A multiplicative step cannot move a zero coordinate away from zero. Positive weight is the scarce resource that lets a coefficient enter or leave the active set.
Valid does not mean equally useful. The optimal weights minimize the surrogate's one-step upper bound over the simplex. Part 5 derives that weight problem, interprets its KKT conditions, and shows how to compute its exact solution.