Subba Reddy Oota, alumnus of Cognitive Science Lab and Language Technologies Research Center (LTRC) gave a talk on Do large language models (LLMs) align with human brains? on 23 May. Here is the summary of his talk:
How does the brain represent different modes of information? Can we design a system that can automatically understand what the user is thinking? We can make progress towards answering such questions by studying brain recordings from devices such as functional magnetic resonance imaging (fMRI). The brain encoding problem aims to automatically generate fMRI brain representations given a stimulus. The brain decoding problem is the inverse problem of reconstructing the stimuli given the fMRI brain representation. Both the brain encoding and decoding problems have been studied in detail in the past two decades and the foremost attraction of studying these solutions is that they serve as additional tools for basic research in cognitive science and cognitive neuroscience.
Recent brain encoding studies highlight the potential for natural language processing models to improve our understanding of language processing in the brain. Simultaneously, naturalistic fMRI datasets are becoming increasingly available and present even further avenues for understanding the alignment between brains and models. However, with the multitude of available models and datasets, it can be difficult to know what aspects of the models and datasets are important to consider. Is the choice of stimulus modality (reading vs. listening) important for the study of brain alignment? Are all naturalistic fMRI datasets equally good for brain encoding? How does the type of model (text vs. speech and encoder vs. decoder) affect the resulting alignment? Apart from these questions, we will also look at the implications of these efforts for models of natural language processing (NLP).
Subba Reddy Oota is a final-year Ph.D student at Inria-Research at Bordeaux, France and a visiting scholar at Max Planck Institute for Software Systems. He received his M.Tech (CSIS) from IIIT Hyderabad in 2016. His research interests are in the areas of language analysis in the brain, brain encoding, and decoding, grounded language acquisition, robot grounding, geometric deep learning, and brain imaging analysis. He has also presented several research papers in refereed conferences like ACL, NAACL, COLING, INTERSPEECH, ICASSP, CogSci, WACV, IJCNN, ICDAR, ICONIP, and workshop papers at NeurIPS conferences. He served as a reviewer for all the leading conferences, including NeurIPS, ICML, ICLR, INTERSPEECH, ACL, ICASSP, AISTATS, ECML PKDD, WACV, IJCNN, Cognitive Science Society Annual Conference, and ACM CHIL. He has delivered tutorials at top conferences like IJCAI, IJCNN and Cogsci.
More details at: https://sites.google.com/view/subbareddyoota300/home?authuser=0
May 2023