6 min read
WDIS AI-ML Series: Module 2 Lesson 2: End-to-end process map of Machine learning - Science or Magic
Written by
Vinay Roy
Published on
30th Mar 2024

Many attendees, mostly senior executives and product leaders, in my AI classes, ask how should they think about the end-to-end process of data science. It makes sense that it could be intimidating and confusing for people who are not data scientists to realize what it takes to take a concept from idea to model development to deployment.What I have come to realize while working on machine learning problems for various companies that I partner with is that even data scientists and technology teams struggle with the right process. They frequently jump from one point to the other, eventually confusing non-tech partners, and delivering a model that does not solve business/user problems. What happens then is endless finger-pointing, distrust among teams, and budget cuts of data science initiatives. My aim through this module is to equip you with a brain map of how to increase the ROI of your data science initiatives by ~5x.

So before we go and start understanding various machine learning algorithms, let us discuss the end-to-end framework of data science deployment. We will talk about many interesting concepts and walk you through who should own which part and what should be the expected output of each part.

Just to set the stage below is a map of what we are going to discuss in this module:

End-to-end machine learning framework from PRD to Data Preparation to Model Selection and Iteration

It looks intimidating but trust me, it will become intuitive by the end of this module. Knowing this framework will help you ensure that your Non-Tech and Tech teams are walking in lockstep throughout the process, the business can solve the end user or business problems, and the company can deliver on its AI strategy.

The framework can be broken down into 6 stages and overall 11 steps:

Stage 1: Business objective and framing of the objective function

We started the discussion in the last module on how companies need a strong alignment between Business Objectives and Machine learning objective function. But how do we achieve that? In this section of the module we will discuss in more detail on how to start thinking about machine learning models the right way, how to kick off an initiative, and how to ensure all teams are in lockstep through the process. We will introduce the concept of Product Requirement Documents and discuss its various elements.

             Step 1: Business Understanding

Stage 2: Data Preparation and Processing

Data is the fuel that enables the machines to learn the rules. However, while companies seem to have a strong AI strategy, data is often a neglected component. In this section, we will highlight the importance of starting with Data, putting a concerted effort behind a robust data strategy, and building a culture where data drives decisions rather than opinions or beliefs. We will also talk about the concept of data splitting so that we can use part of the data for training and the remaining for testing.            

            Step 2: Data Collection
            Step 3: Data Cleaning

Stage 3: Feature Engineering Techniques

We discussed in Module 1 Lesson 3 the concept of features. Feature engineering is the process of transforming raw data into a format that is suitable for use in machine learning algorithms. It involves creating new features or modifying existing ones to improve the performance of the models being built. Feature engineering is a crucial step in the machine learning pipeline because the quality and relevance of the features directly impact the effectiveness of the model.

            Step 4: Feature Extraction

Stage 4: Model selection and Evaluation Metrics

One problem statement can be solved in many ways. We can apply Rule Based Models or Machine Learning Methods or Deep Learning Techniques. Not only this, even in Machine Learning methods, we can have many algorithms that data scientists can use to solve a particular problem, so we will look at how in the real world, data scientists evaluate multiple methods based on various parameters such as Run time, complexity of models, and Machine learning Objective functions to decide which models is the most suitable for the problem.                  

             Step 5: Model Selection and Machine Learning Objective Function

Stage 5: Model Training and Testing

We briefly discussed in Stage 2 the concept of Data splitting. In this stage, we will dive deeper into the concept of Model training and testing to choose the most appropriate model for our problem statement. We will also answer why training and testing are done and use real-world scenarios to explain the concept of overfitting.

             Step 6: Model Training
             Step 7: Model Testing                  
             Step 8: Model Finalization

Stage 6: Deployment (MLOps) and Beyond

We have selected our model but how do we ensure it is living up to the expectation in the production environment when it comes in contact with many other algorithms running simultaneously on our product? We will discuss the concept of A/B testing and also introduce the concept of Machine Learning Operations which has gained immense popularity recently. As we send our model to production, we also want to avoid surprises from the rest of the company and our end users, so what does a launch strategy look like? Finally, we will end this stage with a discussion on one of the least discussed topics but in my opinion the most important step - Iteration. After all, rarely does a technology deployment end up achieving its objective in the first go, we will have to build a culture of experimentation and continuous improvement.

             Step 9: Deploy and A/B Test
             Step 10: Completing the test and Announcing                  
             Step 11: Iterate

Let us discuss each stage in more detail in the next few lessons. If there are some concept that are not clear to you, do reach out as me on https://www.growthclap.com/contact or drop me a note on Linkedin. I am deeply passionate about bringing more awareness of advanced technology and have been teaching Machine learning to senior executives in an easy-to-understand way for many years now. I also help companies bring their vision to reality through machine learning and X tech, feel free to connect to discuss more on how my team can help. Look forward to connecting and growing together.

As a photographer, it’s important to get the visuals right while establishing your online presence. Having a unique and professional portfolio will make you stand out to potential clients. The only problem? Most website builders out there offer cookie-cutter options — making lots of portfolios look the same.

That’s where a platform like Webflow comes to play. With Webflow you can either design and build a website from the ground up (without writing code) or start with a template that you can customize every aspect of. From unique animations and interactions to web app-like features, you have the opportunity to make your photography portfolio site stand out from the rest.

So, we put together a few photography portfolio websites that you can use yourself — whether you want to keep them the way they are or completely customize them to your liking.

12 photography portfolio websites to showcase your work

Here are 12 photography portfolio templates you can use with Webflow to create your own personal platform for showing off your work.

1. Jasmine

Stay Updated with Growthclap's Newsletter

Subscribe to our newsletter to receive our latest blogs, recommended digital courses, and more to unlock growth Mindset

Thank you for subscribing to our newsletter!
Oops! Something went wrong while submitting the form.
By clicking Subscribe, you agree to our Terms and Conditions