Projects
Programming with Representations (PwR)
Developed at Microsoft Research, this project created an intermediate representation layer to bridge the gap between users and Large Language Models, addressing issues like hallucination and enhancing non-technical user interactions.
Repositories:
- PwR-NL2DSL - A tool to convert Natural Language to Domain Specific Language
- PwR-Studio - Studio environment for Programming with Representations
- Jugalbandi Studio - Open-source chatbot framework
- Jugalbandi Manager - Chatbot management platform
Technologies: Large Language Models, Natural Language Processing, Conversational AI
Machine Learning Algorithm Selector (Linux Foundation Networking Project)
Period: August 2021 - October 2021
Contributed to Anuket Project at Linux Foundation Networking, developing a command-line tool for ML algorithm selection based on the use case and data type. Utilized Natural Language Processing for enhanced user response extraction.
Technologies: Python, Command Line, NLP, Transformers, Git/GitHub, Linux
Cyclone Intensity Detection (Smart India Hackathon, AICTE, MHRD, India)
Selected for the India Finals in Smart India Hackathon, organized by AICTE & Ministry of Education, India. Built a Cyclone Intensity Detection Application using Deep Learning (Image Processing) under the guidance of ISRO and utilizing INSAT-3DR satellite data. Our goal was to provide error-free real-time updates to avert damage and loss.
Technologies: Remote Sensing, Contrastive Learning, Python, Object Detection, Classification, PyTorch
Heart Rate Estimator (IEEE Mega Project)
Developed a system to remotely detect an individual’s heart rate per minute from a still video of their face. Implemented Eulerian Video Magnification Algorithm initially developed by MIT CSAIL Lab to magnify subtle color changes in the face that correspond to blood flow.
Technologies: Python, OpenCV, Tensorflow/Keras, SciPy, Numpy
Multimodal Ingredient Substitution Knowledge Graph
Created as part of my Master’s thesis work, this project:
- Built a knowledge graph incorporating 27K ingredients and 40K substitution pairs
- Enabled precise ingredient recommendations using multimodal and constraint-based searches
- Developed an LLM-based query module for the ingredient substitution knowledge graph
Resources:
Cook-Gen: Recipe Analysis and Action Recognition
Formulated cross-modal Recipe Retrieval and developed cooking action recognition for recipe analysis, achieving 95% Recall score leading to published paper at IEEE SMC’23.
Technologies: Computer Vision, Natural Language Processing, Deep Learning