December 2022
Naram Jayadev received his Master of Science – Dual Degree in Computer Science and Engineering (CSE). His research work was supervised by Dr. Pawan Kumar. Here’s a summary of his research work on Applications of Riemannian Optimization to Machine Learning Problems:
Geometric approach to solving a problem seems very natural or sometimes elegant compared to an analytical solution. Several machine learning problems have hidden geometric structures that can be exploited in proposing new solutions. In this work, we consider a specific structure called the Riemannian structure of the search space of the problem. Often such spaces are nonlinear spaces where the standard techniques of optimization cannot work for the lack of linearity. But these spaces have a very useful property of being locally linear, which in turn helps us to consider extensions of classical optimization to such spaces.
In particular, we consider two machine learning problems – the extreme classification problem and the tensor completion problem, where we show the presence of Riemannian structure. Once we identify this structure, we show how to use it to come up with efficient algorithms. For the extreme classification problem, a proof of the convergence is given for the proposed algorithm. We experimentally verify the efficiency of the method in terms of train time and test accuracy comparing with several state-of-the-art methods on extremely large datasets. In the case of the tensor completion problem, we propose a general problem that considers several works in the literature as its special case. For this general problem we propose a Riemannian optimization algorithm that solves the problem efficiently. We verify the correctness of the proposed algorithm both theoretically and experimentally through the special cases.