The Descent Property of Gradient-Based Optimization Methods
Students often assume descent occurs for any . However, if , the objective value can increase or oscillate wildly. The descent property is strictly dependent on the relationship between the step size and the local smoothness (Lipschitz constant) of the function.
Institutional Reference: Fundamentals of Optimization
View Full Proof →