Agrawal Aparna Nitin supervised by Prof. Jawahar C V received her Master of Science – Dual Degree in Computer Science & Engineering (LCD). Here’s a summary of her research work on Towards Natural Sign Language Generation: Scalable and Fluid Sign Synthesis for Inclusive Communication

Sign Language Production (SLP) is a critical technology for bridging the communication gap between the hearing and Deaf communities. However, current systems face a persistent trade-off between visual realism and semantic scalability. This thesis proposes a multi-modal framework to achieve scalable, fluid, and natural sign language synthesis across 2D and 3D representations. 

First, we introduce Sanket Vaani-1K, a curated Indian Sign Language (ISL) dataset spanning over 40 high-utility domains, such as healthcare and banking. We present a 2D production pipeline that utilizes FramePack diffusion-based interpolation to stitch isolated sign units into fluid video, significantly reducing data requirements compared to end-to-end models. 

Transitioning to 3D to overcome 2D limitations like occlusion and depth nuances, we utilize ExAvatar, a Gaussian-based neural renderer driven by optimized SMPL-X body models, which we modified and optimized for the specific requirements of the sign language use case. 

By driving this renderer with person-specific skeletal offsets and high-fidelity facial registration, we ensure the accurate synthesis of manual and non-manual cues. This is integrated into Gloss-SLQA, a proof of concept for a sign language question-answering system that enables natural interaction through a gloss-centric 3D avatar. 

Finally, we present SignWeave-3D, which treats sign production as a masked motion infilling task. By ”weaving” isolated 3D motion units with a diffusion transformer, the model synthesizes natural co-articulation. Quantitative evaluations and user studies with the Deaf community demonstrate that our framework produces significantly more understandable and natural signing than existing stateof-the-art baselines.

July 2026