Research work done on identifying methodological biases in a widely used large language model (LLM)-based metric for evaluating narrative flow (Sap et al., 2022, PNAS) by Amal Sunny, a CSD dual-degree student working under the supervision of Dr. Vishnu Sreekumar was awarded the Best Paper Award at the 29th Conference on Computational Natural Language Learning (CoNLL 2025), co-located with ACL 2025 in Vienna, Austria.

The project originated as a summer research initiative by SRISHTI intern Yashashree Chandak, who encountered difficulties reproducing the performance of the LLM-based sequentiality measure (Sap et al., 2022) on a conceptually similar dataset. Her inability to replicate the reported results prompted a more in-depth investigation led by Amal Sunny, with support from Advay Gupta. Their analysis uncovered a fundamental methodological bias in the original study. The CoNLL award committee recognized the importance of such validation and replication efforts, emphasizing the need for increased scrutiny in computational language research.

August 2025