[month] [year]

Himanshu M – Emotions in Text

August 2022

Himanshu Maheshwari received his MS Dual Degree in  Computer Science and Engineering (CSE). His research work was supervised by Prof. Vasudeva Varma. Here’s a summary of his research work on Understanding and Controlling Emotions and Intensity in Text:

Emotions are an essential part of daily communication and play a critical role in conveying a person’s mental state. Understanding, categorization, and controllable emotions generation are widely researched in NLP and social sciences. Different work in social sciences aimed to define emotions and identify different emotions. Ekman’s initial work identified six primary emotions: anger, disgust, fear, joy, sadness, and surprise. Plutchik identified eight emotions and classified them into four polar opposite pairs. With the advent of computer science, different computational approaches have been developed to identify and categorize emotions and generate emotional text.

In this thesis, we develop data-driven, psychologically grounded computational approaches to solve two broad problems relating to emotions in text. The first problem is identifying emotions at a paragraph level. Existing work detects emotion at a sentence level having a limited context.

We aim to detect emotions at a paragraph level consisting of multiple sentences. Thus, the model needs to consider the emotional cues present in an individual sentence and paragraph as a whole unit. We also explore the classification problem in scientific documents when the context is large but cannot be broken down into individual sentences. The second problem that we explore is changing the emotion of a sentence while controlling its intensity. In the process,

the meaning of the sentence should be preserved. This task of changing specific attributes of a sentence while preserving its overall meaning is defined as style transfer. For both the tasks, the proposed system achieved state-of-the-art.

Pretrained language models are current de facto in Natural Language Processing (NLP). We develop an ensemble solution consisting of three fine tuned language models for the first task.

The language models are RoBERTa, BART, and RoBERTa model that is first fine tuned on sentence-level emotion detection tasks. RoBERTa model is used due to its remarkable performance in many natural language understanding tasks. BART has shown good performance for tasks involving multi-sentence level context like summarization, thus is used here. RoBERTa model that is first fine tuned on sentence-level emotion detection tasks is used to identify strong sentence-level cues. Our proposed system was ranked one globally for the Empathy Detection and Emotion Classification shared task at the WASSA workshop at ACL 2022. We also explore the power of the pretrained language model for the classification of scientific documents when the context is large but cannot be divided into sentences. Our proposed system of using highly contextualized embeddings of science BERT achieved a new state of the art and was ranked one globally for 3C Citation Context Classification shared task at the Scholarly Document Processing workshop at NAACL 2021.

The task of changing specific attribute(s) of a sentence while preserving its overall meaning/context is defined as style transfer. In the second part of this thesis, we solve the task of changing the emotion of the sentence to a target emotion while controlling the intensity of the target emotion. In the process, the overall meaning or the theme of the sentence should be preserved. We first study the existing dataset for emotional understanding and then augment it with distance learning. We then propose a deep-reinforcement learning-based model to solve the task. The model is trained to maximize five rewards to enhance meaning conservation, naturalness, grammatical correctness, target emotional, and intensity rewriting. A psychologically grounded bootstrapping technique is used to warm-start the model. We conduct extensive ablations studies and human and automatic evaluations to evaluate our model and understand its various components. Our proposed model outperformed all the baselines, showing our approach’s efficacy in solving the task of emotion transfer with specific intensity.