Devarakonda Sri Charan supervised by Prof. Anil Kumar Vuppala received his Master of Science by Research in Computer Science Engineering (CSE). Here’s a summary of his research work on Synthetic Data Regulation for Low-Resource Medical Speech Recognition in Dravidian Languages

Building Automatic Speech Recognition (ASR) systems for languages like Telugu and Kannada is difficult because these languages do not have much labeled speech data available, especially in medical settings. On top of that, these languages have complex word structures, and when synthetic speech is used to fill the data gap, it often sounds different enough from real speech that the model gets confused. This thesis presents two training approaches that tackle these problems from two different angles.

The first approach assigns each training sample a confidence score based on three factors: audio cleanliness, similarity to real speech, and transcription accuracy. The model can either use fixed weights or learn the best weights on its own during training, with the learnable variant adapting to the character-istics of each language. A gradual training schedule starts with high-quality samples and slowly brings in the model’s own uncertainty as a guiding signal, ensuring more stable and reliable learning across stages. A medical language model applied after decoding further reduces errors by enforcing domain-appropriate word choices. This approach reduces word error rate from 24.3% to 15.8% on Telugu and from 31.7% to 25.4% on Kannada.

The second framework, Real-Governed Representation-Calibrated Training (RG-RCT), works inside the model by preventing synthetic data from distorting how the model internally represents speech. It scores each synthetic sample by how far it sits from the real speech distribution, using different distance measures depending on how well the sample aligns with real speech. Deeper model layers are made to rely more strongly on real speech patterns, since they carry more abstract and task-critical information. Two training penalties keep internal representations from drifting, and a voice conversion tool called FreeVC adds acoustic variety to the synthetic data. RG-RCT achieves word error rates of 17.1% and 16.6% on Telugu and 23.3% and 25.6% on Kannada using Wav2Vec 2.0 and Whisper, outperforming simpler baselines across all conditions.

Together, the two frameworks show that using synthetic speech effectively in low-resource ASR re-quires reasoning about data quality at both the sample level and the model’s internal representation level. Both methods are evaluated on Telugu and Kannada medical datasets and are designed to generalise to other low-resource languages and specialised domains.

July 2026