How to create great Machine Learning solutions
Machine Learning allows people to create great products. It allows us to take large complex data, and extract insights from it. However, designing great ML tools can be very complex. There are a lot of moving parts to the journey. In this article, I will break down the process, from finding the right ideas to the final deployment.
Discover your niche
This first part is absolutely important. This is also what most people overlook. Most effective machine learning is very similar across the different domains it is applied in. The basics are good enough. They are very good in fact. But they aren’t great.
If you want to design solutions on the next level, you will need a lot of domain-specific knowledge. Data augmentation for computer vision is very different than data augmentation in Natural Language Processing. The specifics of Reinforcement Learning on StarCraft 2 are different than RL on Boxing. At this step, you only have to discover what problem you find most interesting.
Finding your interests depends on you. You can get it through books, traveling, hobbies, personal circumstances etc.
This step is absolutely crucial. You will have to put in a lot of work to create great products. If you are doing it towards a topic you are interested in, you will be able to commit and actually see the project through to the end.
Learn the Basics
While you are exploring the different applications where AI/Machine Learning is making a difference, put some effort into learning the basics. Learning the basics has many benefits.
- Most ML is heavy and creative implementations of the basics. Check out 5 Unsexy Truths About Working in Machine Learning for more details.
- Learning the basics allows you to transition between various fields/implementations. They are 80% of the process. To look into and learn the other 20% becomes relatively easy.
- Learning the basics will allow you to learn from and converse with experts. Naval Ravikant describes this as “Being able to read and understand any book in a library.”
One important pillar of the basics is mathematical knowledge. I made video detailing what topics you need to know, how much you will need, and how each topic will help you. Check out the video below. Make sure to watch it till the end to get an understanding of how the topics tie together.
Research
Once you know the basics, it’s time to build your domain knowledge. Different network architectures work best for deep fake detection when compared to cancer detection. The only way forward here is to dig through the research being done in the specific field you’re into.
So how do you go about this? If you don’t have the time or inclination to dig through the newest updates in your field/niche just follow creators. Follow people like me, Yannich Kilcher, Two Minute Papers, Henry AI Labs etc. The benefit of following multiple creators is that it will expose you to lots of ideas and implementations, beyond your fixed niche. For example, Two Minute Papers has a great series on reinforcement learning in teaching robots to walk. This allowed me to learn about a field I am not involved in, and take some of those learnings into my own work.
If you want to know more about how to interact with research to become a domain expert in a Machine Learning niche, check out, “How to become a Machine Learning Expert”. It goes into more detail about the process to building your competence and becoming an expert.
Trial and Error
Next is the non-sexy part of this process. You will need to do a lot of trial and error. You will discover the weirdest things. Take for example the well-known library pandas-profiling. It is used for EDA (exploratory data analysis). Did you know that the second it finishes creating the reports, it shuts down all the python UI? This is not something that anyone knows the reason behind. How do I know this? I was playing around with Sweetviz for my tool and noticed this. You can actually see the issue I raised here, in case you want to try fixing it.
Trial and Error will also give you a fingertip for the code and processes allowing you to build and expand your solutions to fine-tune the specifics.
Deployment
Once you have a fully functioning solution, it is important to look into deployment. This can be very easy, or extremely annoying depending on your needs. For a lot of people, working code is enough. Other times, you will need to build out a web app with a functioning UI. Or design cloud-based services. Maybe push it as an API/package.
The specifics of the deployment will depend on the stakeholders. Keeping in touch with the different developments/trends in the field will help you weigh the pros and cons of the various approaches. Computerphile is a great resource to learn about the various practices.
Following these steps will allow you to build and refine great Machine Learning solutions.
If you liked this article, check out my other content. I post regularly on Medium, YouTube, Twitter, and Substack. I focus on Artificial Intelligence, Machine Learning, Technology, and Software Development.
Reach out to me
If that article got you interested in reaching out to me, then this section is for you. You can reach out to me on any of the platforms, or check out any of my other content. If you’d like to discuss tutoring, text me on LinkedIn, IG, or Twitter. If you’d like to support my work, use my free Robinhood referral link. We both get a free stock, and there is no risk to you. So not using it is just losing free money.
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
If you’re preparing for coding/technical interviews: https://codinginterviewsmadesimple.substack.com/
Get a free stock on Robinhood: https://join.robinhood.com/fnud75
