In June 2020, when two student researchers from the Centre for Visual Information Technology (CVIT) turned in their entry to the Multiple Object Tracking (MOT) challenge, the submission gained them the first rank in the famously challenging problem. Here’s a simultaneous look at the benchmark as well as the students, Karthik Shyamgopal and Ameya Prabhu, who make complex research look easy-peasy.
One of the modes of admission into the International Institute of Information Technology Hyderabad (IIITH) is via certified Olympiads. Essentially, students who have been selected for training in Informatics, Science, Mathematics, and Linguistics to represent India in the International Olympiads during their class 11 or 12, are directly invited for an interview (without an entrance examination) for admission into one of the Dual Degree programmes. Thanks to a reasonable intake of students through this route, there exists a little clique of ‘Olympiadees’ in the Dual Degree programme. Karthik Shyamgopal, popularly known as Shyam, a 5th year Dual Degree student and Ameya Prabhu, a recent graduate of the same programme met through this sub-community. Famously labelled as a “research zealot” for promoting research among his peers, Ameya willingly embraces the moniker and is known to inspire many others on campus. He says, “Well, I’m from the Chemistry Olympiad but luckily overlap a lot with other true Informatics and Linguistics Olympiad folks.” He explains his seemingly exclusive liaisoning with this bright sub-set of people at the institute who also happen to be close friends by saying, “All random discussions lead to serious paper ideas!”
Multiple Object Tracking
Shyam’s predominant area of research concerns a principal challenge of Computer Vision – that of visual object tracking where given the location of an object in the first frame of a video, you try to track it in all the other frames too. Ameya’s expertise lies in making machine learning processes more efficient– in terms of requiring lesser data, or smaller models which can for instance, fit in a mobile phone – and enabling them to learn in a never-ending fashion known as lifelong machine learning. In other words, research that is taking a step closer to the distant goal of human-level intelligence. It was when Ameya visited Hyderabad in February this year that he pitched in with an idea for submission to the Multiple Object Tracking Challenge (MOT) that Shyam was working on under the guidance of Prof. Vineet Gandhi. As the name suggests, in multiple object tracking, there are multiple objects to track and the tracking algorithm has to not only determine the number of objects in each frame (of video) but also keep track of their identities from one frame to the next. MOT is a challenging problem as it is and evaluating it is another. Since 2015, the most used benchmark for MOT has been the MOT challenge which focuses on pedestrian tracking. Their website consists of a leaderboard always open to new tracker submissions. Interestingly, the IIITH team submitted their methods to MOT 2016 and MOT 2017 in April as well as MOT 2020 in early June. At that point, they had the best results and were ranked number one on all these 3 benchmarks. What was unique about their submission was that it was the only unsupervised algorithm at the time. “Since then a couple of works seem to have improved upon our performance,” says Shyam. The corresponding research paper on the developed algorithm is currently under review at the British Machine Vision Conference.
Fun With Research
Ameya’s primary research guide has been Dr. Anoop Namboodiri. However thanks to his enthusiasm and willingness to collaborate where ever possible, he has had the opportunity to interact with everyone in the research centre. Prof. Vineet Gandhi, under whom Ameya worked as a TA several times is all praise for him. Speaking of his first encounter with Ameya in the Digital Image Processing class, Prof. Gandhi says, “Ideally, the course is taken later, but he ended up taking all the computer vision-related courses much early, in his second year itself. He had even started auditing the Computer Vision course in the first year itself.” Underscoring the pluses of pursuing research, the professor goes on to add that Ameya’s research journey which he embarked upon rather early and enthusiastically – brought forth great results, the sort that are typically missing in normal course work. His research, “GDumb: A Simple Approach that Questions Our Progress in Continual Learning” was in the top 2% of papers in European Conference on Computer Vision (ECCV2020). He has also received outstanding reviewer awards for both CVPR19, CVPR20 and ECCV20.
In Shyam’s case, Prof. Gandhi had been witness to his rational arguments and positive attitude displayed in the course on Human Values. “His principled approach in discussion at his age was a pleasant surprise,” he remarks. Hence, when Shyam approached him while selecting an advisor, Prof. Gandhi did not think twice. Shyam is very well known in the Hyderabad quizzing circles and often reminds the professor of his own college-going self who was an avid quizzer too. Early this year, Shyam presented a paper at the premier Winter Conference on Applications of Computer Vision (WACV 2020) at Colorado. There are a few more research papers that are under way.
On To Greater Things
Before he sets off to Oxford in pursuit of a PhD next year, Ameya continues to work with students at CVIT and is hopeful of having some more publications come out of the association. For the MOT challenge, Shyam says that they demonstrated that it is indeed possible to create an unsupervised multi-object tracker. They are now working on creating not only the world’s best unsupervised multi-object tracker, but the simplest and most cost-effective one. “The simplest one we develop still outperforms existing supervised counterparts popular today. But we intend to do it for only ethical applications like sports analytics,” he adds. According to Prof. Gandhi,“The two of them (Ameya and Shyam) understand each other very well. Their tuning (dual reading, dual programming, dual writing) has played a major role in the MOT project. And I believe this is just a start, both will go a long way”.