Experience

1. Research Intern, Microsoft Research India
   • Period: January 2024 - Current
   • Project: Working under Programming with Representations (PwR): Developed an intermediate representation layer to bridge the gap between users and Large Language Models, addressing issues like hallucination and enhancing non-technical user interactions, thereby transforming chatbot building workflows. We are working on automating the chatbot building process using Conversational Programming paradigm.
   • Built Jugalbandi(JB) Manager, open-source chatbot management platform supporting multiple channels, including WhatsApp, Telegram. It handles multilingual text and voice conversations using Bhashini & Azure Speech models.
   • We have open-sourced our work on GitHub:
     • PwR-NL2DSL, a tool to convert Natural Language to Domain Specific Language.
     • [PwR-Studio] (https://github.com/microsoft/PwR-Studio/)
     • Jugalbandi Studio
     • Jugalbandi Manager
   • Mentors: Sriram Rajamani, B. Ashok, Akash Lal, Sameer Segal
2. Research Intern, AI Institue, University of South Carolina

   • Period: December 2022 - April 2024

   • Master thesis on Knowledge Enabled Multimodal Ingredient Substitution, built knowledge graph incorporating 27K ingredients and 40K substitution pairs. Enabled precise ingredient recommendations using multimodal and constraint-based searches. Developed an LLM-based query module for the ingredient substitution knowledge graph.
   • Formulated cross-modal Recipe Retrieval and developed cooking action recognition for recipe analysis, achieving 95% Recall score leading to paper Cook-Gen at IEEE SMC’23.
   • Mentors: Revathy Venkataramanan Dr. Amit Sheth
3. Visiting Researcher at Societal Computing, Saarland University (SIC)
   • Period: May 2023 - Aug 2023
   • Project: Time and Multispectral Domain Satellite Image Super-Resolution
   • Conducted research in Satellite Image Superresolution, enhancing disaster analysis through high-temporal frequency & low-resolution images using GAN & Diffusion models.The work was focused on improving the spatial resolution (Planet Imagery to Maxar Imagery) using conditional generation and GANs.
   • Mentors: Ingmar Weber, Ferda Ofli
4. Research Intern, University of Maryland, Baltimore US
   • Period: September 2022 - Mayy 2023
   • Project: Personalized AI Assistant, funded under HealthCareNLP grant.
   • Worked on developing Personalized Response Generation models using reward scaling over BART & T5, paper K-PERM accepted at AAAI Symposium’24 and improved NUBIA score by 10%. The work has been focused on using Knowledge and Persona for loss scaling resulting better response generation.
   • Mentors: Manas Gaur
5. AI Intern, EdgeNeural.ai, Pune, India
   • Period: May 2022 - August 2022
   • Project: Worked on model optimization and improving inference speed through various quantizations and hardware-specific CPU and GPU customization. Building Training and Optimization pipelines for state of art Computer Vision models.
   • Collaborators: Sarvesh Devi, Chidhambararajan, Dhanraj
6. Research Intern, Video Analytics Lab, IISc, Bangalore
   • Period: May 2022 - August 2022
   • Project: Worked on Generative Adversarial Networks for Image and Video Generation. There has been exhaustive work on Face Images and videos but other domains are left pristine. We were looking to employ StyleGAN based architectures for disentangled video interpretation in multiple domains.
   • Mentor: Rishubh Parihar
7. Research Intern, Visual Learning and Intelligence Lab, Indian Institute of Technology, Hyderabad
   • Period: November 2021 - March 2022
   • Project: Developed a Novel Architecture for better Image Classification of low-quality images and unbalanced datasets. Worked on Building a U-Net based architecture for Semantic Segmentation on Dental X-Ray (Paper under review).
   • Mentor: R Sai Chandra Teja
   • Collaborators: Dhruv Makhwana, Rohit Pawar
8. Computer Vision Engineer, AI Mage (WETHEKOO)
   • Period: March 2021 - April 2021
   • Project: Building a Fashion Tagging Engine using Deep Learning (Siamese Network, Instance Segmentation).
9. Software Engineer, Rhizicube Technologies
   • Period: June 2021 - September 2021
   • Project: Oversaw Server & REST API development and Database for Consumer Data Platform Using Golang(Gin). Build a real-time streaming data pipeline using Apache Kafka.
   • Collaborators: Udit Sarin, Yash Goyal