A research team of three joins hands with a multinational TV network and a graphics broadcasting company to produce a unique viewing experience.
An unbeaten 71 at the grand finale of the Asia Cup in the UAE saw Bhanuka Rajapaksa pulling the Sri Lankan team from the brink. From paddle scoops over the keeper’s head to offside flicks, the key highlight of his innings was picking the right field. Viewers across the world were privy to how this strategy worked out thanks to a real time feed of live fielding positions made possible by researchers from the International Institute of Information Technology Hyderabad (IIITH).
Detecting and Tracking
The world of cricket is no stranger to advanced analytics. From sensors mounted on bats that generate data for every stroke played by a batsman, to ball tracking information, machine learning and AI is used not only by TV commentators to analyse the game but also by sports production houses to enhance the viewing experience for fans. However the focus in cricket is typically on the batsman or the bowler. They are the ‘heroes’ and the story is never told from the fielders’ point of view. The Computer Vision team at IIITH tried to address this by tracking the positions of fielders from a top angle view which would then generate a graphic to be aired for viewers.
Prof. Vineet Gandhi with his students Swetanjal Dutta, Jeet Vora, and Kanishk Jain have developed a fully automated machine learning algorithm that can live track and display the positions of 11 fielders on the screen. The patent-pending technology uses one or more static cameras to capture the top view of the entire cricket field. It includes a classifier that obviates the presence of batsmen and the umpires focusing instead on only the fielders. Besides positions, the tracking system also automatically computes statistics such as the velocity with which they are moving, distance between two fielders, the distance they are travelling and so on.
Lab-Industry Engagement
The research centres and labs at IIITH are focused on the creation and development of new technologies with particular emphasis on their application to the real world. As part of an industry collaboration with Star Sports, IIITH has been testing out its computer vision-based sports analytics. “Star Sports worked with us for the first time when we piloted our badminton analytics at the Premier Badminton League in 2019. Perceiving a similar requirement in other areas of broadcasting and entertainment, they have partnered with us in the Centre of Excellence model where we can build multiple technologies for them,” says Prakash Yalla, head of Product Labs, an initiative of the Technology Transfer Office at IIITH. This time around, player tracking technology found its application in the live broadcast of the Asia Cup cricket tournament held in the UAE.
Enhancing The Narrative
“Knowledge of the field position adds value to whoever is watching a match. For instance if I know that there is no fielder on the offside and the batsman tries to hit a shot there taking advantage of this gap, it’s easy to predict that the shot will go for a four or a six without any risks involved,” says Prof. Gandhi. A representative from Star Sports iterates the advantage with another example: “If a batsman has been scoring runs at the long off on consecutive balls, the captain of the fielding team can suddenly change tactic with a fielder there. And this change in fielding positions is something that a normal broadcast camera can’t show. But now with this tool, we can see who moved where.” For a cricket crazy nation, such player positions and movements when presented in attractive and informative graphical format boost viewers’ understanding of the game. At the Asia Cup, the raw sports data was converted into on-screen graphics with the help of AELive, a specialist in broadcast graphics and data solutions.
MS student, Jeet Vora draws a comparison from the realm of movie-making to explain viewer experience during live matches. “Just like the Director of a film decides the plot narrative and stitches together the order in which scenes are eventually screened, the Director and/or Producer of a live game orchestrates what exactly is to be broadcasted during the live broadcast,” he states. A visual of the field placement thus makes sense in the critical aspects of the game. Take the run up to a ball. Or nail-biting death overs where field positions are in a state of flux for every ball bowled.
What’s Unique
Most player tracking technologies that exist in the market require a significant amount of human effort to make it work seamlessly. “Our solution is completely automated and the human largely acts as a verifier. We designed a human-in-the-loop system, that allows intervention if needed in some rare scenarios,” remarks Prof. Gandhi. According to him, humans can’t be dispensed with entirely; they play the role of observers in this largely automated scenario, stepping in only in extraordinary circumstances. The other aspect he draws attention to is the dynamic nature of the graphics displayed. “Since the tracking is live, the dots signifying the fielders are constantly moving on the screen in sync with the player movements which is a great viewer value-add,” he says. Despite the huge advantages afforded by the solution, the technology by itself is a no-fuss, simplistic one with the camera also being very basic and low cost.
Against All Odds
The player tracking solution by itself is not new, states Prof. Gandhi, adding that they have done similar work in the past for soccer and other sports too. “Last year, we tested out our tech at the Tamil Nadu Premier League. The difference is that it didn’t go live. This time, we went ahead and deployed it in real life, in live matches at the Asia Cup”, says Jeet. The project which took off 2 years ago suffered a derailment due to the pandemic. With stadiums shut and no matches being played, it was difficult to train their algorithm. “Later, even with stadiums opening up, access to the grounds was limited thanks to a host of Covid-induced measures enforced such as quarantines and so on,” says Prof. Gandhi. This impacted their main task of procuring data of recorded footage from live games. Finally, there were engineering challenges that one faces in the wild. Enumerating the on-site hiccups faced by their small crew of three, Prof. Gandhi says, “The original camera that we had tested out before going to Dubai itself didn’t work due to the special design of the stadium. Unlike the typical floodlight towers found in Indian stadiums, the Dubai International Stadium has a unique set of floodlights known as the Ring Of Fire’.” Despite such hardware challenges that contributed to a high-pressure situation, the researchers successfully managed to pull it off by going live in a majority of the matches. “Our learning from the event has been on how to work in a multi-party scenario. We made our mistakes, learned to improvise on the fly and know that we can improve further,” says the professor adding that Star Sports demonstrated its commitment to the partnership by providing the necessary support when the researchers faced a rocky start. “The experience of working with both Star and AE.Live who are stalwarts in the broadcasting space was phenomenal. In fact all three of us (parties) were very well coordinated and in perfect sync.”
Next Up
For Kanishk, the live deployment of what was originally a research project checks one of his boxes. “In the Asia Cup, it was in the 4th over of the India vs. Hong Kong match that the team’s field plotting graphic went live for the first time. There was a simultaneous sense of jubilation and relief to see it up there,” he says, adding that he’s always dreamed of converting theory into something practically useful. “When it happened, there was an overwhelming feeling of great satisfaction to actually achieve it”. With display of field positions being the first step, Jeet muses that future technology could leverage this for building its analytics. “Coaching centres could use information on field positions and wickets taken to strategise on their own gaming techniques and to predict wickets taken or runs scored.” With the first live translation of their tech a huge success, other cricketing scenarios where player tracking can be used is under consideration. “The T20 format is extremely fast-paced and with the narrative changing constantly, player data can’t be shown too frequently. However it can be displayed more often in test matches and ODIs,” observes Prof. Gandhi.