Dr. Karthik Vaidhyanathan and his students presented the following papers at he 48th International Conference on Software Engineering (ICSE 2026)  held from 12 – 13 April at Rio, Brazil:

• Sreemaee Akshathala, MS by Research in CS working with  Dr. Karthik Vaidhyanathan presented their work done by the SA4S group, SERC  in collaboration with MontyCloud on  Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems at the AGENT workshop, co-located with the 48th International Conference on Software Engineering (ICSE 2026). The authors of this research paper are Akshathala Sreemaee, Bassam Adnan, Mahisha Ramesh, Karthik Vaidhyanathan, Basil Muhammed, and Kannan Parthasarathy.

Here is the summary of the paper as explained by the authors: Recent advances in agentic AI have shifted the focus from standalone Large Language Models (LLMs) to integrated systems that combine LLMs with tools, memory, and other agents to perform complex tasks. These multi-agent architectures enable coordinated reasoning, planning, and execution across diverse domains, allowing agents to collaboratively automate complex workflows. Despite these advances, evaluation and assessment of LLM agents and the multi-agent systems they constitute remain a fundamental challenge. Although various approaches have been proposed in the software engineering literature for evaluating conventional software components, existing methods for AI-based systems often overlook the non-deterministic nature of models. This non-determinism introduces behavioral uncertainty during execution, yet existing evaluations rely on binary task completion metrics that fail to capture it. Evaluating agentic systems therefore requires examining additional dimensions, including the agent’s ability to invoke tools, ingest and retrieve memory, collaborate with other agents, and interact effectively with its environment. These challenges emerged during our ongoing industry collaboration with MontyCloud Inc., when we deployed an agentic system in production. These limitations surfaced during deployment, highlighting practical gaps in the current evaluation methods and the need for a systematic assessment of agent behavior beyond task outcomes. Informed by these observations and established definitions of agentic systems, we propose an end-to-end Agent Assessment Framework with four evaluation pillars encompassing LLMs, Memory, Tools, and Environment. We validate the framework on a representative Autonomous CloudOps use case, where experiments reveal behavioral deviations overlooked by conventional metrics, demonstrating its effectiveness in capturing runtime uncertainties. 

• Dr. Karthik Vaidhyanathan and his SA4S group along with the collaborators had 9 papers presented across various workshops (AGENT 2026), co-located conferences (CAIN 2026, SEAMS 2026) and main tracks (Research Track, Birds of Feather in Software Architecture) of the 48th International Conference on Software Engineering (ICSE 2026), that was held from 12-13 April 2026, Rio. (Let me know if all paper details are needed).

Dr. Karthik Vaidhyanathan has been selected to be in the Organizing Committee of ICSE 2027 to be held in Dublin from 25 April – 1 May 2027 as Proceedings Co-Chair.

More details at:  https://conf.researchr.org/home/icse-2027

ICSE is the A* Conference in Software Engineering. This year it had 2K+ participants from about 51 countries. 

ICSE 2026: https://conf.researchr.org/home/icse-2026

April 2026