Akshit Kumar supervised by Dr. Parameswari Krishnamurthy received his Master of Science – Dual Degree in Computational Linguistics (CLD). Here’s a summary of his research work on Linguistic Interpretation of Large Language Models

Large Language Models have grown to be remarkably capable at tasks which require a sophisticated use of language, yet what they have actually learnt about language is poorly understood. The models have billions of parameters and are trained on vast corpora, making their internal representations difficult to inspect directly. This opacity is a practical concern for general deployment of these models. We already possess rich, formally precise theories of linguistic structure. Whether language models, trained without explicit linguistic supervision, converge on anything resembling those theories is a question that bears on both linguistics and interpretability: it tests whether the theories capture structure latent in language, and it gives interpretability research a principled target to search for. In this thesis, we pursue this question through the Paninian grammatical tradition, a dependency based syntactico-semantic framework centred on participant roles (karakas ¯ ) and morphological case markers (vibhakti). The framework is particularly interesting due to its categories encoding the syntax-semantics interface rather than surface syntax alone, and is better suited than mainstream frameworks like Universal Dependencies for the morphologically rich, free word order languages of the Indian subcontinent. We present two complementary studies, moving from representation to mechanism. In the first, we probe three multilingual transformers across seven Indian languages annotated under the Paninian scheme, and find that karaka-labelled dependency ¯ relations and vibhakti features are linearly decodable from frozen representations. The layer profile follows a consistent hierarchy: lexical features emerge earliest, morphological features plateau broadly, and syntactic structure peaks latest, suggesting that the models implicitly recapitulate the informational order of the Paninian analytical framework. In the second study, we move from correlation to causation. Using causal tracing and activation patching on Gemma-2-2B, we reverse-engineer the circuit implementing the sampradana/ap ¯ ad¯ ana (recipient/source) k ¯ araka distinction, isolating a compact mechanism in which ¯ a prepositional gate token, functioning as a vibhakti marker, steers late-layer attention to retrieve the correct entity. The circuit aligns with the Paninian view that surface morphological cues mechanistically drive semantic role selection. Together, these findings suggest that formally specified linguistic theories can productively guide interpretability research: from establishing that a grammatical distinction is encoded in a model’s representations to identifying the mechanism by which it is computed

July 2026