Convergence Proof of the Steepest Descent Method
Students often conflate the convergence of the gradient with the convergence of the iterates . While the former is guaranteed under mild conditions, the latter requires the stronger assumption of strong convexity to ensure the sequence does not wander along a flat plateau.
Institutional Reference: Fundamentals of Optimization
View Full Proof →