Projects¶
Azure function orchestration based ETL¶
Feb 2023
Worked on Azure function (function, durable function, activity function) to orchestrate the activities to perform the ETL process.
Triggers worked on 1. HTTP Trigger 2. Blob Event Trigger
[Proof of Concept] Celery - Redis - Airflow¶
Oct 2022
The objective of this PoC is to trigger a Airflow pipeline when new message send to Redis queue
Ref: https://github.com/muthugit/event-driven-celery-redis-airflow
[Open Source] Airtable ORM¶
Oct 2022
Created the python package called airtable-orm to act as the Object Relational Model for the Airtable. It helps the developers to query data from Airtable to python objects.
Ref: https://pypi.org/project/airtable-orm/
Dagster workflow engine¶
Apr 2020
Creating independent Dagster solids for the client's specific operations. Also, creating pipelines with the Dagster solids to complete the workflow.
Custom workflow engine¶
Jul 2019 – Mar 2020
Created custom workflow engine to handle the data and process pipelines in Python. With the experience from CDAP, created OOPs based workflow engine to manage the pipelines to move the data from source to the web based, 3D visualization screen.
CDAP workflow engine¶
Aug 2018 – Jun 2019
Technologies used - CASK's CDAP, Python, JAVA, Paraview & PVPython, Microsoft Azure Blobfuse & File storage
- Understanding the process of the Mining industry
- Worked with the data from the IoT device and managed to store in the server in secure manner
- Creating custom plugins in CDAP to create the automation pipeline process in CDAP
- Workin with the Geo-scientists and automated their process using CDAP
DEVOPS for a Satellite imaging startup¶
Jul 2018 – Aug 2018
Technologies Used – Google Cloud Shell, Bitbucket, Superset, Google Cloud Storage, Open VPN, VPC, Google Single Sign On (SSO)
- Setup secure computing infrastructure on Google Cloud Platforms under the Virtual Private Cloud (VPC) and Virtual Private Network (VPN)
- Setup the Atlassian Bitbucket server on Google Cloud Platform
- Setup Superset server on Google Cloud Platform
- Implemented Google Single Sign On (SSO) for the applications
- Implemented automated backup system on Google Cloud Storage
- Created programmatic restore from Google Cloud Storage
Cloud based Asset tracker for an Autonomous vehicle data management company¶
May 2018 – Jun 2018
Technologies Used – Python, AWS API Gateway, AWS Lambda functions, AWS RDS (MySQL), AWS Cognito User Pool, Ionic, Angular6
- Understood the client’s requirement and designed the Database and the micro services architecture for the application
- Prepared technical design document
- Created micro services based APIs to manage both mobile applications and web applications.
- Managed security over the data transfer using AWS Cognito users management.
- Created Selenium based automation testing tool to verify the APIs.
- Supported the front-end applications (Ionic & Angular6).
- Created automation deployment script to deploy all the services to AWS from the client using AWS CLI SDK.
Object Sorter – Robotic Arm¶
Apr 2018
Technologies Used – Python, OpenCV (Blob Detection), CGI
- Designed the architecture for the system
- Implemented the OpenCV - blob detection algorithm to identify the shape of the object.
- Differentiated objects using different approaches and passed the coordination to robotic arm in Python.
- Implemented IP camera solution to the system to stream the images.
- Created front-end web application to visualize the object’s coordination.
- Packaged the solution in Raspberry Pi.
Cloud based advertisement placement tracking system for a social media advertisement consolidator¶
Jan 2018 – Feb 2018
Technologies Used – AWS Lambda functions, AWS Kinsis streaming, AWS Queuing system (SQS), DynamoDB, Cloud watch, D3.js
- Created the cost effective system architecture for the system based on the client’s requirement
- Created the front-end to the different levels of users to show the graphs and other dashboards using a JavaScript library D3.js
- Collected the data from the various sources (TV Apps) as the stream and stored them into a NoSQL database
Sales forecasting for the manufacturer of Air conditioner¶
Jan 2018 – Feb 2018
Technologies Used – Python, J2EE, Qlik Sense cloud
- Created the centralized database by pulling the external factor data from various sources.
- Created the data model for the data visualization in Qlik Sense.
- Identified the sales pattern by overlaying the various external factors on the sales data (especially the weather data).
- Helped to create the statistical model to predict the sales for the next quarter.
Log analysis and reporting framework for a Bay area startup¶
Dec 2017 – Jan 2018
Project descriptionTechnologies Used – Grafana, InfluxDB, django, AngularJS, MySQL
- Created the SaaS based front-end application to monitor the various systems
- Connected the log data to the InfluxDB
- Created the Grafana dashboard for the individual users dynamically and connected to the InfluxDB
- Integrated the Grafana dashboard into the custom web application
Product identification - iPhone application¶
Nov 2017
Project descriptionTechnologies Used – Python, Ionic, Bootstrap CSS
- Managed the machine learning team to build the real-time product identification model
- Implemented the model from the front-end mobile application by using the custom camera options to capture the real-time frame
- Used the Amazon Advertising API to get the product information includes ratings & reviews count and showcase them in the mobile application after identifying the product
Testing automation for Voice assistant platforms¶
Nov 2017
Project descriptionTechnologies Used – Selenium, Google Assistants, Amazon Alexa, Python.
- Created automation system to test the voice assistants (Google Assistant, Alexa).
- Ask the question to a Chatbot agent from an another chatbot and collect the response from it.
- Validated the responses from voice assistants and prepare the testing reports based on the results.
Chatbot engine on AWS¶
Built a custom chatbot engine on the client's AWS account.
- Architect the chatbot engine with AWS Lex, Lambda functions & DynamoDB
- Managed a team of 4
- Simplified the User Experience to have the Chatbot engine in easy 3 steps
Survey Chatbot for a Clinical trial organization¶
Oct 2017
Project descriptionTechnologies Used – Python, AWS Lambda functions, AWS API Gateway, Amazon Alexa Skills, AngularJS, Bootstrap CSS, Ionic
- Created the Medical survey chatbot system to collect the user responses for the predefined questions
- Showcased the collected response in a web application
- Implemented the same system for the mobile application using custom chatbot agents
Custom Google Actions for a leading retail organization¶
Aug 2017 – Sep 2017
Project descriptionTechnologies Used – NodeJS, Google Dialogflow, Google Firebase Functions
- Created the proof of concept system with custom Google Actions to collect the orders through the chatbot
- Implemented personalization on the chat-bot back-end system to suggest the products based on the customers’ history of orders
- Connected the Magento (E-Commerce platform) system to get the orders from the Google Actions
Custom Amazon Alexa skill – Retail chat bot for a clothing retailer¶
Jun 2017 – Jul 2017
Project descriptionTechnologies Used – J2EE, AWS Lambda functions, AWS API Gateway, AWS Lex, AWS RDS (MySQL), AWS DynamoDB
- Built the Alexa custom skill for AEO for different kinds of intents
- Integrated the content management system to deliver the response through Alexa for the user’s questions
- Programmatically trained the Question & Answer to the AWS Lex NLP system
- Implemented OAUTH security system in Alexa custom skill
Data analytics and visualization solution for a non-profit organization working towards preserving global ice melts¶
May 2017 – Jul 2017
Project descriptionTechnologies Used – JAVA, AWS Lambda functions, AWS Cloud watch events, Grafana, InfluxDB
- Created the programmatic e-mail reader to collect the temperature of the various sensors details.
- Loaded the collected data into the time-series database (InfluxDB)
- Created a visualization system using Grafana with the InfluxDB connection
Sodabot - Alexa Soda vending machine¶
Jun 2017
Project descriptionSodabot is a Conversational based chatbot engine to pour the Soda using Custom Alexa Skill. Technology Stack - AWS Lambda function (Serverless) - Raspberry PI (IoT) - PHP - Python
AptEngine – The product recommendation engine extension for Magento¶
Sep 2016 – Dec 2016
Project descriptionTechnologies Used – PHP, Magento, MySQL, Jquery
- Designed and implemented the Market Basket Analysis on Magento system
- Created the DB design for the application
- Collected the user’s data from the front-end and stored into the database
- Created the cron job to flatten the data for the recommendation
- Implemented the recommendation model in Magento as the extension
Totes¶
Jan 2015 – Dec 2015
Project descriptionManaged Data Loading process for 4 different sites of Totes Business. Created Generic Loading Process model to load file in any format (Excel, CSV or Flat text file)
Catalog loader - Electrolux - B2B & B2C Ecommerce¶
Aug 2015
Loaded Product, Price (B2B & B2C), Inventory information from PIM & JDE ERP systems via BCC Platform: J2EE Framework: ATG Version: 11.2
ArtPrint¶
Dec 2014
Artprint is the Product Customization Magento extension.
Responsibilities - Done the market research about the product customization websites - Understand the client requirements - Create the product customization website with necessary tools like edit, import image. - Checkout process