Ashwin Joseph

MSc in Robotics Systems Engineering

September 2022 - Present

Relevant Coursework

• Linear Control Systems
• Advanced Robotics Kinematics and Dynamics
• Automated and Connected Driving Challenges

B.Tech. in Mechanical Engineering

July 2013 - May 2017

Relevant Coursework

• Machine Design and CAD
• Fluid and Hyraulic Machines
• Gas Dynamics and Jet Propulsion
• Thermodynamics and IC Engines

MS in Computer Science (Machine Intelligence)

June 2017 - May 2019

Relevant Coursework

• Machine Learning, Data Analytics and AI
• Computer Networking
• Data Structures and Algorithms
• Operating Systems

RWTH Aachen University

November 2022 - Present

Research Assitant- AI

• I am contributing towards the preparation of Curriculum and practical exercises related to Computer Vision and Machine Learning for Master’s degree students in the Institute of Highway Engineering at RWTH University. Work involved utilizing Pytorch with Python in the fileds of NNs, CNNs and Transfer Learning.
• Predicting flow of traffic on a Multilane Highway.
• Processing 3D point clouds of Highway captured on a Drone using Python and utilizing tools like QGis and CloudCompare.

Sony India Software Centre, Bangalore (Deputation)

May 2021 - July 2022

Consultant- AI

• Responsible for supporting the Sony Tokyo (Japan) team and deployment of AI-based applications onto Sony's Electronic Gadgets.
• Worked in the Platform Implemetation Team
• Got to work closely with Engineers from Qualcomm and Sony.
• Using C++ and Python for software development.
• Received the Spot Award for contributions in AI/ML activities in the GPULIB Development Project.

Ignitarium Technology Solutions

June 2019 - Present

Senior Engineer- AI and Cloud

• Developing GUI for applications using PyQT5 and Tkinter
• Utilizing State of the art and Custom networks for obstacle classification along railway tracks.
• Attire detection platform, which does real-time video analytics, that helps to ensure PPE of construction/Lab/Factory workers.
• Customization of neural network architectures that are lightweight for deployment on micro-controllers.
• Utilizing Deep Learning and Image Processing techniques for detection and classification of Jointbars along Railway Tracks.
• Have represented the company for project demonstration at Maker Village, Cochin that was attended by Mr. Sanjay Dhotre, Minister of State for Human Resource Development, Communications and Electronics and Information Technology, Government of India.
• Received Team Spark award at the Annual Event held virtually on November 2020 for the teamwork and contribution towards completing the projects exceeding quality targets and keeping schedule.
• Wrote technical blogs for publishing on the company website for marketing activities
  The blogs can be viewed at the following links:
  * AI-driven Face Mask and Attire Detection for the new normal
  * Footwear defect detection using Vision Intelligence Systems
  * FOOTWEAR DEFECT DETECTION USING VISION INTELLIGENCE SYSTEMS
  * Light-weight People Detection on Microcontrollers using Custom Neural Networks
  

OpenFive Technologies (Deputation)

Dec 2020 - Mar 2021

ML Contractor

• Deputed from Ignitarium Technologies on contract (remote) at OpenFive (SiFive Group, US). The various tasks involved:
• Help in the porting of custom neural networks onto FPGA using simulators and emulators.
• Support the newly formed AI team at OpenFive India with the KT Sessions from their offshore team based out of Taiwan.
• Testing custom neural networks developed by OpenFive R&D for accuracy with other standard neural networks. Carrying out tests to ensure 32 bit, 16 bit, and 8-bit quantized models from the simulator yield the same results.
• Successfully deployed 3 custom real-world applications and showcased the demo to convince them of their products capabilities.
• Carry out benchmarking activities for Inference latency and End-to-end latency.
• Provide insight into the theoretical aspects of evaluation metrics to be used for evaluating neural network performance.
• Explore and use custom Tool from CEVA named LiteML for pruning and optimizing neural networks that helped to deploy NN models more effectively.
• Worked on Python scripting for automating tasks.

Implement trajectory for Welding Tip mounted on a UR10e Cobot (Undergoing at IGMR, RWTH, Aachen)

July 2023

The task was to integrate ROS packages for webcam, Moveit (collision avoidance), AR tags (tracking), and an Action Server to process path requests for a welding tip on a UR10e cobot from Universal Robotics at the IGMR Institute, RWTH Aachen University. Utilized ROS and Python for the project.

3D Point Cloud Processing of a Multilane Highway / Autobahn (Undergoing at ISAC - Institute of Highway Engineering, RWTH, Aachen)

June 2023 - Present

Processing 3D point clouds of Highway captured on a Drone using Python (open3D and laspy) and utilizing tools like QGis and CloudCompare.

Case Study for setting up a Battery manufacturers factory (Undergoing at RWTH, Aachen)

May 2023 - Present

Doing a case study to analyze the objectives, strategic goals, area and location selection, planning of layout and capacity keeping in mind about making the plant highly flexible to cater to future growth plans.

Traffic Prediction on Multilane Highway (Done at ISAC - Institute of Highway Engineering, RWTH, Aachen)

May 2023 - May 2023

Used python and Machine Learning for analyzing traffic flow on a Multilane Highway taking into account position, velocities and accelerations of multiple automobiles. A custom dataset was used for this project.

Porting AI applications to OnnxRuntime (Done at Sony India, Bangalore)

Dec 2021 - Feb 2022

Worked on benchmarking and experimenting/ comparison of performance and utility of Vendor's Deployment Tool(Qualcomm SNPE) with open-source alternatives like OnnxRuntime fordeployment onto Qualcomm Snapdragon chipsets. The task involved loading onnx modelsand inference results on inputs using C++ code and cross-compilation of OpenCV.

Deployment of Camera Modules on Qualcomm chipsets (Done at Sony India, Bangalore)

May 2021 - July 2022

Worked on development, optimization (Quantization), memory profiling and deployment ofAI-based modules for Smartphone Camera onto Qualcomm Snapdragon chipsets. Workedon applications related to Image Super-resolution (GANs), object detection, image whitebalance correction and image classifiers. Task involved communication with R&D Engineersfrom Sony and Qualcomm. Worked with Pytorch, Nnabla, Qualcomm SNPE and utilized Python for the Task.

Custom Pose Detector and Classifier for Health industry (Done at Ignitarium, Kochi)

Apr 2021 - May 2021

Developing Custom Pose Detector and Classifier for Health industry utilizing Tensorflow Posenet.

Voice Command Engine- Septra (Done at Ignitarium, Kochi)

Oct 2020 - Jan 2021

Utilized Audio Processing and Deep Learning techniques for developing Voice Activity Detectors,Voice classifiers, and denoisers for developing a Voice Command Engine to be deployed onMicrocontrollers from Renesas. Worked on Speech synthesis APIs for speech data augmentation. Worked on the Tasks with Python and Tensorflow 2. The marketing video for the same can be viewed at Easy Days with Septra

Jointbar Detection on Railway Tracks (Done at Ignitarium, Kochi)

Jul 2020 - Sep 2020

Worked on a Predictive maintenance project that analyzed Footage from IR Depth Camerasfor object detection and classification of faulty Joints Bars along Railway Tracks to improvepassenger safety. Worked on Docker containers for packaging the solution and Python and Tensorflow 2 for development.

Mask and Attire Detection (Done at Ignitarium, Kochi)

May 2020 - Jun 2020

Worked on an Attire detection application, which does real-time video analytics, that helps toensure the attire of construction/Lab/Factory workers for granting entry access using poseestimation (Posenet), object detection, and classification neural networks. The project was deployedremotely onto a Nvidia Jetson. Worked on Python and Tensorflow 1 for the task. The demo video for the same can be viewed at AI-based Protective Gear Detection Platform
The blog on the same can be found at AI-driven Face Mask and Attire Detection for the new normal
This blog was also published on the website of Indian Electronics and Semiconductor Association and can be read at AI-driven Face Mask and Attire Detection for the new normal

Person Detection (Done at Ignitarium, Kochi)

Apr 2020 - May 2020

The project's aim was to identify presence of humans using Computer Vision technique in the nearby vicinity (within 3 meter radius) at real time and sending signal for activating a particular device. The project was to be deployed on an Embedded System board and various constraints were to be dealt with requiring customization of deep neural networks. The demo video for the same is available in the link as follows: DEMO light person

Obstacle Classification on Railway Tracks (Done at Ignitarium, Kochi)

Dec 2019 - Mar 2020

The project was implemented for classification of obstacles at Railway Tracks for a U.S based private equipment company. It involved utilizing Image Enhancement algorithms and different Classifier networks on images provided for improving evaluation metrics of the obstacle classification process. My role was to try out various network architectures and evaluate performance as well as to build a GUI for the same. The project was implemented in Python 3 utilizing the Tensorflow framework.

Image based Automatic Defect Detection of Footwear Raw Materials at production Line (Done at Ignitarium, Kochi)

Dec 2018 - Nov 2019

Worked on Defect Detection and Texture identification of footwear raw materials utilizing image. Processing algorithms and semantic segmentation to identify features of interest. Contributed indeploying a SQL database for storage and data analysis. Worked on Python, C++, Tensorflow 1, Image Processing (OpenCV), PyQT5 (UI Design tool) andSQL for the task. The demo video for the same is available at: Defect Detection for Footwear
The blog on the same was published on NASSCOM's (NASSCOM is the industry association for the IT and IT-enabled products and services sector in India) website and is available at: FOOTWEAR DEFECT DETECTION USING VISION INTELLIGENCE SYSTEMS

Determining Use Case of Z Force Neonode Airbar Sensor (Done at Ignitarium, Kochi)

Dec 2018 - Dec 2018

The Project involved setting up the sensor and using its SDK to establish a use case. I was able to create a use case involving drawing live onscreen using coordinates from fingertips. The demo video for the same is available at: Neonode ZForce Airbar Application

Android Application for Shopping Cart (Done at IIITMK, Trivandrum)

Aug 2018 - Nov 2018

The project involved creating a shopping cart, on Android Platform which had features like searching for products in the supermarkets within a specific radius of the location specified, adding the items to cart, showing offers as well as ordering. The team comprised of me and two other colleagues of mine.

Predictive Maintenance utilizing Image Processing Techniques (Done at IIITMK, Trivandrum)

Jan 2018 - Apr 2018

The work was an attempt to develop a computer aided detection system using sophisticated image processing techniques to help monitor different regions of a factory site/ workshop for regions with high temperatures beyond normal which often is an indication of overrun machinery, faulty electrical connections or at worst a fire breakout and would have assisted in real time predictive maintenance system that could save factories a good amount of money per year.

Biomimetic Bullet Proof Armour (Done at AJCE, Kanjirapally)

Nov 2016 - Apr 2017

Conventional composite armours rely on the superior properties of composite laminates to effectively stop projectiles. Recently, there has been an increasing interest in new designs for armours, that do not rely on material properties alone. In this study, a novel geometry for high velocity impact protection has been proposed, that consists of multiple layers of different materials with specific properties, and has been inspired from the structure of a woodpecker’s head and armadillo armour. The behaviour of the same under the impact of a cylindrical projectile of 9 mm diameter and a velocity of 750 m/s has been studied using numerical methods in order to determine the optimum combination of different materials to be used. A comparison of the same with conventional multiple plate geometry shows that the bio mimetic armour performs considerably better than the latter, in terms of specific performance criteria.

Domains worked on

• Deep Learning
• Machine Learning
• Computer Vision
• Natural Language Processing
• Time Series Forecasting
• Image Processing
• Audio Processing
• Point Cloud Processing
• GUI development
• Platform Deployment (Qualcomm, Nvidia, Renesas, Docker)
• Neural Network Optimization (Quantization)

Industries for which projects have been deployed

• Manufacturing
• Predictive Maintenance
• Healthcare
• Construction
• Multiple OEMs of Electronic Gadgets,
• Embedded Systems boards

Programming Languages

• Python
• C++

Frameworks and Libraries

• Keras,
• TensorFlow 2(Certified by Tensorflow- Google),
• Pytorch,
• Caffe,
• Scikit-Learn,
• OpenCV,
• CEVA Liteml,
• Dialogflow
• Sony Neural Network Console (NNC)
• Laserpy
• Open3D

Version Control Tools

• Github,
• GitLab,
• Gitee,
• Gerrit.

Deployment Tools

• ROS Neotic
• Docker
• Qualcomm SNPE Toolbox
• Qualcomm QNN SDK,
• CEVA Network Generator
• Android Debugger Bridge(ADB)
• Renesas E-AI Translator
• Website hosting (GoDaddy)
• Google Cloud Platform
• OnnxRuntime with NNAPI EP
• Quantum GIS
• Cloud Compare

Project Management and Team Collaboration Tools

• Jira
• Confluence
• Qualcomm QNN SDK,
• Openproject