Amal Sunny supervised by Dr. Vishnu Sreekumar received his Master of Science – Dual Degree  in Computer Science and Engineering (CSD). Here’s a summary of his research work on Narrative Coherence and Human Memory: Reassessing Sequentiality with LLMs and Human Judgments:

Understanding narrative flow is central to studying how memory and schema shape human storytelling. With the rise of large language models (LLMs), new measures like sequentiality—which uses LLM probabilities to quantify narrative coherence—have gained attention but remain under-validated against human judgment. This thesis first attempts to replicate the original sequentiality findings using a curated dataset of autobiographical and biographical texts. Failing to replicate the expected differences, we uncover a methodological bias in the original formulation that inflates topic effects. Removing this bias significantly reduces the metric’s discriminative power, raising questions about its validity. To address this, we propose a rectified, context-only version of sequentiality and validate it using human-annotated Automated Essay Scoring (AES) datasets. The revised measure aligns more closely with human ratings of narrative coherence and improves AES model performance when used as an interpretable feature. Overall, this work highlights both the promise and pitfalls of LLM-derived metrics in cognitive research and underscores the need for rigorous validation before deploying such measures in applied NLP systems.

November 2025