Dr. Kesav Kaza, a researcher from the University of Ottawa and an alumnus (DD-ECE-2006 to 2012) of IIIT Hyderabad visited TCS Research Group@IIITH and gave a talk on  Decision Making Under Uncertainty in Cyber-Physical Systems on 8 September at KRB Faculty Meeting Room. Here is the summary of the talk as explained by Dr. Kesav Kaza: 

Complex systems such as modern power grids, computer/data networks, disaster management infrastructure, or even large-scale public health and smart urban infrastructures are increasingly falling into the category of cyber-physical systems, as they aim to monitor and control large-scale physical processes over the internet through a network of interconnected components. These systems are increasingly required to make decisions regarding resource allocation in uncertain and rapidly changing environments. These decisions are difficult because current actions shape the future evolution of the system/environment states, resources are limited, and information is often incomplete.

In this talk, we will consider “constrained restless bandits”, a class of sequential decision problems that model the problem of dynamic resource allocation under uncertainty in rapidly evolving systems. We will discuss how this framework can be used to maximize long-term performance, and how it can be applied to real problems such as scheduling sensors or planning wildfire response.

In the second part of the talk, we will introduce a two-timescale hierarchical control architecture for cyber-physical systems based on Markov Decision Processes. This framework reflects how decision-making in the real world often occurs at different levels: central agencies can plan at a slower timescale with a broader view and long-term goals, while local agents need to make decisions at a faster timescale with more direct knowledge of their environment. We will discuss how this two-timescale structure leads to two types of optimization problems—the central and the federal—representing the trade-offs that arise between the autonomy of local agents and centralized control. Together, these ideas point to different ways of designing hierarchical decision-making systems and planning for uncertainty.

Kesav Kaza is a research fellow at the University of Ottawa, Canada, where he works on automated decision making applications including mission planning for autonomous systems. Earlier he was a post-doctoral researcher at the University of Montreal from 2021 to 2023 where he worked on human-machine systems for decision making. Kesav Kaza received a Ph.D in Electrical Engineering from IIT Bombay in 2020. He received B.Tech and M.S by research degrees from IIIT Hyderabad. His research interests include decision making under uncertainty, human-machine systems, reinforcement learning, stochastic control, and networked systems.

September 2025