For those who love soaking in history and enjoy an unhurried pace through ruins and monuments, an audio guide seems to be the preferred choice. Europe-bound travellers, hard-pressed for time yet desperate to tick off the must-sees from their list of museums, castles, amphitheatres and palaces, prefer popular (and free!) audio tour guides such as Rick Steve’s Europe among others. Lately, Indian monuments like The Taj Mahal too have seen a rising trend in the renting out of audio guides not just to foreign tourists but to desi ones as well.

More Than Just A Guide

At the Centre for Visual Information Technology (CVIT), IIIT-H researchers though have gone beyond a mere audio guide and made a visit to a heritage site an interactive experience. Sahil Chelaramani, 2nd year, MS by Research student and one of the researchers behind the Interactive Tour Guide project says that the intent behind digital heritage projects at IIIT-H was to digitize information typically found in archaic ASI records and make it accessible to the common man in a way that is user-friendly and engaging. He explains how work in the field of digitizing heritage information has been going on for a while at the centre. “Our paper, An Interactive Tour Guide For a Heritage Siteis a small part of ongoing work, but we’ve refined and taken it further. We’ve introduced the ability to customize the story telling”, says Sahil modestly.

CVIT Research: Marrying Computer Vision and NLP

Recognizing the increasing processing and storage capacity of mobile devices on which a multitude of computer vision applications are run today, CVIT has been researching vision solutions to handle the complexity and variety of use cases. And this requires appropriate adaptation of the mobile vision applications with minimal changes in the algorithm or implementation.

Instant Recognition of Images

One of the earliest research projects in 2013 dealt with instantly recognizing or identifying a monument based on the picture clicked on a mobile camera. The same project explained its application in augmenting 3D heritage structures with parts that no longer exist. For example, it allowed any tourist to see the missing dome on the Hampi Stone Chariot augmented over the original structure.

Retrieval Sans Network

Successive research found solutions to similar identification even when a network connection is limited or not available. In this scenario, the entire visual search index must reside on the mobile device itself.

‘Tour-Guide’ Feel

In 2016, researchers from CVIT proposed a method to generate on-the-fly dynamic narratives for the heritage monuments giving a ‘tour guide’-like feel on the tour path. To avoid carrying maps and looking like a typical tourist, this was next integrated with GPS. The unique feature highlights nearby points of interest to users visiting heritage sites.

Select Your Stories

The latest enhancement that Sahil and his colleague Vamsidhar Muthireddy have made under the guidance of Prof. C.V. Jawahar, is to create a mechanism that automatically generates questions to the users to gain insights into their interests. For example, if you’re interested in the cultural aspects of history, your response will elicit a different story from the one a user interested in architecture will hear. Learning user preferences early enough, the approach works on ‘improving’ stories. Simultaneously educative and entertaining, this Android-based app can also cater to children and adults differently. The efficacy of the approach was demonstrated on the Golconda Fort, in Hyderabad.

In The Pipeline

“We would like to extend the project to create an immersive Virtual Reality for web applications. Also on the anvil is to apply this technology to different heritage sites, cities, museums and public places,” says Sahil. Well, we can’t wait to try it out on the other local wonders!

Sarita Chebbi

Sarita Chebbi is a compulsive early riser. Devourer of all news. Kettlebell enthusiast. Nit-picker of the written word especially when it’s not her own.