Inside CVIT: 25 Years Of Looking and Staying Ahead

Guided by the same mandate as the rest of the institute – to be a ‘Stanford of the region’, the Center for Visual Information Technology was officially inaugurated in May 2001. Twenty five years later, the proof of the pudding is in the seeing, and it is only fitting to take a trip down memory lane. 

There’s a sense of palpable pride that resonates through the corridors of the Center for Visual Information Technology (CVIT) at IIIT-H when news of an alum’s achievements trickles through. “Just yesterday, we heard of our former gold medallist from the second batch of the BTech program in Computer Vision joining as Research Director of the AI Security Institute in UK,” reports Prof. PJ Narayanan whose razor-sharp memory can list names of alumni batchwise along with the positions they hold globally. “Name any industry and you will find CVIT alumni active in that area. It in fact prompted Prof. Larry Davis, renowned researcher in the field of computer vision, to remark that IIIT-H has unleashed ‘vision monsters’ worldwide,” laughs Prof. CV Jawahar. 

It All Began With OCR
There may be 100s of CVIT alumni today who are researchers of great repute. Yet when the centre began modestly from a little room in Nilgiris 25 years ago, it was a center “with no PhD candidates, Master’s or Dual Degree students”, recalls Prof. Jawahar. The institute’s inaugural batch was already in its 3rd year. With the only center on campus then being the Language Technologies Resource Centre (LTRC) helmed by Prof. Rajeev Sangal, it made sense to give an opportunity to the large group of students there to work at CVIT. That established the first formal student association with the lab. At the same time, Prof. Raj Reddy, former chairman and founding member, IIIT-H and faculty at Carnegie Melon University kicked off a book digitising project named The Million Book project. The project scanned books in many languages, using OCR to enable full text searching, and providing free-to-read access to the books on the web. Prof. Ravi Kiran S, current faculty, CVIT, from the second batch of IIIT-H was one of the six students who had worked on an OCR project and sought to continue research in the area. “We began by working on Hindi and Telugu OCR, the data for which was obtained by printing out the characters before scanning them. A giant-sized scanner had to be manually fed and I recall spending many nights falling asleep next to the comforting hum of the machine continuously scanning pages!”, exclaims Prof. Ravi Kiran. 

Recalling the ‘vision’ which was way ahead of its time, Prof. Vasudeva Varma, LTRC,  says, “With language and vision being two primordial cognitive abilities, our (then) futuristic idea was to combine the two – algorithms, data structures and so on – to create things together. We envisioned a system where a Telugu newspaper could be scanned via OCR, translated into English or any language text and then read out in Hindi for the benefit of a Hindi-speaking person. This kind of convergence was what we aspired to since the beginning.”

A much needed boost for the fledgling lab at the time came in the form of funding by HP Labs which had set up a research centre in Bangalore. “They were involved in some OCR activity too and were interested in what we were doing,” says Prof. Jawahar. 

No Lab Is An Island: Vision and Security
Interdisciplinary research is one of the hallmarks of IIIT-H and it explains the presence of labs in lieu of traditional rigid departments. The partnership between LTRC and CVIT was old hat by now. But the most unlikely of collaborations took place between the Computer Vision researchers and Prof. Kannan Srinathan, Security researcher and expert at the Center for Security, Theory and Algorithms. “Since there were no PhDs, MS students , no BTech projects also to work on, we did what was next best – the faculty got together and collaborated on security in vision,” he chuckles. This partnership culminated in a number of papers from crypto-biometric verification protocol to privacy preserving video surveillance and secure video encryption that were presented in conferences like the International Conference on Computer Vision (ICCV), International Joint Conference on Biometrics and others. 

Vision and Robotics
When Prof. Madhava Krishna, Robotics Research Center joined IIIT-H, computer vision was not yet part of his research vocabulary. The first ever joint work done with CVIT was with Prof. Jayanthi and centred around a robotics problem involving tracking motion using a camera. But the collaboration deepened when Prof. Krishna began working with Prof. Jawahar and his students. “When I initially arrived, I had no idea of computer vision. I attended Jawahar’s classes on computer vision and even took a summer of training with one of his students, Anil Neelakanti in 2008.” Gradually, students from the two centres began working together, eventually leading to joint mentoring, projects and publications, including what Prof. Krishna describes as the first ICCV oral paper from India, in 2011. 

The early research was in classical computer vision: using a camera to understand what in a scene is moving and what is stationary, when the camera also is moving, since both the stationary and moving world appear to move in a moving camera. Over time, this evolved into more sophisticated work involving depth estimation from a single image, recognising and reconstructing moving objects, and predicting their trajectories. Vision eventually became an integral part of RRC’s research rather than something it needed to access through collaboration.

According to Prof. Krishna, “The initial impetus for RRC to get into vision was via CVIT.” He adds that the presence of a strong peer group and well-trained students “certainly accelerated things.” Today, RRC is able to undertake vision research independently, while continuing to collaborate with other professors like Vineet Gandhi and Ravi Kiran on projects. As Prof. Krishna puts it, “This help from CVIT essentially enabled RRC to become independent or self-sufficient on vision on its own.”

Came For a Degree, Stayed For Research

IIIT-H’s distinguishing feature lies in its emphasis on undergraduate research. “Our research reputation is built on the backs of the undergraduate workforce – the Masters and Dual Degree students,” agrees Prof. Jayanthi Sivaswamy, adding that the PhD numbers have been fewer. In terms of academic chronology, an undergraduate honours program was the first to be kicked off. It continues to date where end-of-second year students have an option of choosing to do extra project work under any faculty member for the remainder of their two years. Around the same time, a 5-year Dual Degree program was conceptualised too, affording students a BTech degree along with a thesis-based Masters. Prof. Ravi Kiran was one of the students who opted for an additional year of research-based thesis to obtain the Dual Degree. He perhaps speaks for the majority of his batch when he says that the aspiration at the time was to snag a job at either Infosys or Wipro. He was introduced to Computer Vision through the Digital Image Processing course taught by Prof. PJN. That and the OCR research work he began, convinced him that “working at Wipro or Infosys was not something I wanted to do in life”. 

Pawan Mudigonda, from the same batch as Prof. Ravi Kiran had similar thoughts. Prof. PJN refers to him as being a deeply academically-oriented person, who is known for not just publishing a number of papers whilst in his third year but also for forging the first ties between IIIT-H and the University of Oxford. “Pawan had a paper presentation at ICVGIP – Indian Conference on Computer Vision, Graphics and Image Processing in 2002 when Prof. Andrew Zeisserman was one of the organizers. Clearly, he was impressed by what he saw and offered to co-guide Pawan for his PhD along with Prof. Philip Torr at the University of Oxford,” narrates Prof. PJN. It paved the way for several other CVIT-ians to be offered admission into the PhD program at Oxford. Notable among them and perhaps the first female student who went on to pursue research was Mukta Prasad. As part of an alumni-connect series, she had famously remarked, “I think we studied a master’s worth of courses in computer vision while still in our bachelors program.” Fascinated by what she was exposed to, she continued her research journey to see where it led to. From a stint in academia as a professor, she has since moved to industry.    

The Creation Of a Master
The introduction of a Masters degree was the next logical step. Prof. Avinash Sharma who obtained his Masters degree here holds the distinction of being the first alum to return to IIIT-H as CVIT faculty.  Currently a professor at IIT Bombay, he speaks of how one interaction with Prof. Srinathan convinced him to further his computer vision research with a PhD. “We were showcasing face reconstruction at an R&D Showcase when Prof. Srinathan placed a plastic bottle of water in front of the projector and said, reconstruct this. When I protested and said, it’s not possible because of refraction, reflection, transparent surface of the bottle and so on, he simply said, ‘if you can do this, this is research’ and walked away”. 

One in a Million
During the mid-2000s, the Ethiopian government prioritized strengthening its local ICT sector. To achieve this, senior lecturers and scholars from the Addis Ababa University Department of Information Science were sponsored to pursue doctoral degrees at IIIT-H so that they could lead local university departments in Ethiopia after moving back. One of them was Million Meshesha. He became the very first PhD graduate from CVIT in 2008. His research centered on optical character recognition (OCR) and document image analysis for indigenous African languages like Amharic. 

Shared History and Growth

CVIT got off to a steady start under the stewardship of Prof. PJN and Prof. Jawahar – two friends and colleagues who gave up “safe, government, pensionable jobs” to embark on a seemingly uncertain career in a new institute. They were soon joined by Prof. Jayanthi Sivaswamy who quit her role as faculty in the Department of Electrical and Electronic Engineering, University of Auckland. 

“For me, 25 years of CVIT cannot be separated from 25 years of the institute. CVIT didn’t exist in a vacuum. It existed in an institute that required a lot of time and effort in building from the ground-up,” remarks Prof. Jayanthi, emphasizing that the real achievement has not only been about building a center with an amazing reputation but also IIIT-H with esteemed credentials. Clarifying that her work is a bit different from the rest of the ‘core’ Vision researchers due to the fact that she is at heart an Electronics and Communications person, Prof. Jayanthi says that she chose to work in the medical domain from the start. Her work is centered around medical image computing, from diabetic retinopathy to the creation of a brain atlas. 

Six Degrees Of Separation
The next faculty to join the center was Prof. Anoop Namboodiri. He was a former colleague of Profs. PJN and Jawahar at the Centre for Artificial Intelligence and Robotics, DRDO who left to pursue a PhD in the US. When he came back to India, his most obvious choice was to come back to a lab that was buzzing with activity and old friends. Known for his research in biometrics, he is also associated with a startup, DreamVu that emerged from CVIT.

Prof. Vineet Gandhi’s journey to IIIT-H is, in many ways, a story that began long before he officially joined the institute. While pursuing his MS and PhD at Inria in France, he found himself surrounded by researchers and students from IIIT-H and its extended academic circle. Avinash Sharma, Vishesh Chari, Amit Sangroya and others became close friends and collaborators, making his social and intellectual circle in France feel almost like an informal IIIT-H community. During his PhD, Prof. Gandhi presented his first paper at CVPR where he met Prof. C.V. Jawahar. “After my PhD, upon Vishesh Chari’s insistence, I visited IIIT-H, gave a talk and was so impressed by the quality of the questions from the audience that I joined as a senior scientist.” What began as a request for a temporary position while he figured out his next move has turned into an 11-year association with IIIT-H. The lab itself in general and its research culture in particular is the answer to Prof. Gandhi’s “why”. He describes himself as someone who spends much of his time in the lab, working closely with his students. Unlike other institutes where the bulk of the research is outsourced through external collaborations, he says, with pride, that at CVIT, the intellectual “heavy lifting” happens within the lab itself.

Non-tangible Accomplishments
While much of the celebrations are centered around CVIT’s most visible successes and achievements of its famed alumni, Prof. Jayanthi draws attention to the culture of the lab that may well be the raison d’etre for the said milestones. One of them concerns the exacting standards set by the lab. ”We are known to be a center that is demanding and yet students still come here. For me, that is a testimony of a good center,” she says, continuing,”CVIT has so far accepted all honours or dual-degree students who applied to it.” The other, she refers to, is the  well-known practice that when students apply here for research, the selection is decided as a group. “Two things happen this way; one, every faculty member’s need is met and two, we try to allocate the best students to the youngest faculty because they need a good start. We look out for each other and it’s the reason for team success,” she reasons. 

The Computer Vision Evolution
When CVIT was launched, the buzz words around the key technologies and techniques ranged from geometry, 3D image processing to pattern recognition. “We’ve now moved to machine learning, and deep learning to multi-modal AI. People talk about speech, audio, vision, text – looking at communication in a holistic way rather than just face or expressions,” says Prof. Jawahar. An aspect that is close to the professor’s heart and one that he continues to pursue is assistive technology. One of the newest projects that the lab is working on deals with using AI and video-understanding models to study the diversity of the Indian culinary landscape. “It’s part of a larger healthcare ecosystem. We are looking at the impact on agriculture, the farmers and eventually on how we consume food,” explains Prof. Jawahar. 

Thanks to the ubiquitousness of facial unlocking mechanisms on smart phones, social media apps using filters and others, CCTV cameras in public spaces, facial recognition systems at airports, and so on, computer vision as a field of technology is not novel, even to a lay person. The center’s popularity hinges on this, coming in at a close second behind LTRC. “The work done at both and the results obtained are very tangible, hence people want to come here,” explains Prof. Jayanthi. CVIT has grown with the addition of two more faculty in the recent years – Profs. Makarand Tapaswi and Raghavendra GS. The center has spawned generations of acclaimed researchers and the high academic calibre of the students presenting at top Vision conferences is often a talking point. “Hopefully we have done our bit in training them well and making them capable of navigating this changing world,” signs off Prof. Jawahar.