Shikhar Saxena supervised by Tejas Bodas received his Master of Science – Dual Degree in Computer Science (LCD). Here’s a summary of his CGD: Modifying the Loss Landscape by Gradient Regularization 

Line-search methods form a fundamental class of algorithms for solving optimization problems. The most basic among them is the steepest descent method, where one always moves in the direction of the negative gradient of the objective function. Newton’s method, on the other hand, is a second-order approach that exploits curvature information through the Hessian to adaptively select more informed descent directions. In this work, we introduce a new linesearch method called Constrained Gradient Descent (CGD), which implicitly reshapes the optimization landscape to enable more efficient progress toward the solution. 

CGD is derived by considering a constrained version of the original optimization problem, where the constraint is imposed on a function of the gradient. By optimizing the corresponding Lagrangian, CGD effectively modifies the geometry of the objective, leading to a line-search procedure in which the Lagrangian penalty regulates the descent direction. This control mechanism allows CGD to navigate toward iterates with smaller gradient magnitudes compared to the standard steepest descent method. We establish global linear convergence guarantees for CGD and primarily validate its performance through extensive numerical experiments on the practical variants, CGD-FD (a Hessian-free implementation) and CGD-QN (a quasi-Newton implementation), both of which demonstrate strong empirical performance on synthetic benchmark functions.

Building on these results, we further explore theoretical interpretations and extensions of CGD. In particular, we interpret CGD as a gradient descent method under a matrix-induced norm and also place it within the broader mirror descent framework, thereby highlighting its geometric underpinnings. Finally, we investigate the applicability of CGD in the context of Bayesian optimization. In this setting, CGD is used to modify the optimization of acquisition functions, incorporating gradient-based regularization into the acquisition optimization step.  

Through these developments, our work positions CGD as a versatile optimization framework that bridges classical line-search methods with modern perspectives on geometry, regularization and black-box optimization.

September 2026