Learning from the biggest Machine Learning Research YouTuber
Content Creation, Finding important developments in Deep Learning, Stable Diffusion, and much much more.
To help me understand you fill out this survey (anonymous)
How can you get into Machine Learning Research? How can you parse through all the noise and find the most important developments in Machine Learning? How can we combat bias in AI, especially when it comes to large models like GPT-3 and Stable Diffusion? What are the differences between ML Academia and Industry? And how can you get into content creation about Deep Learning developments? Recently, I spoke to Yannic Kilcher, one of the Internet’s premier Machine Learning researchers. Our conversation covered these topics and much more. In this post, I will share some of the highlights from the talk and add some of my thoughts to them.
For those not familiar, Yannic is best known for his amazing YouTube channel. Anyone that follows my work knows that I can’t recommend his work enough. Whether it’s his in-depth paper breakdowns, his punchy ML news segments, or his amazing interviews, Yannic is amongst the best resources to stay in touch with cutting-edge ML research. If you’re someone into Machine Learning, his channel is a godsend. You should definitely check him out. Now let’s get into the conversation.
Key Highlights
- How to build an audience- How can you build an audience in niche topics like Machine Learning Research? And what does it take to create good content? Yannic and I discussed this topic. The conclusion- Create content for yourself. Don’t overthink it looking for what the ‘best’ is. Just create something that solves your problems.
- Finding what is important- Hows does someone like Yannic wade through the flood of ML developments to find what is most important? What factors does he look at? How does he differentiate b/w the important Machine Learning developments and standard ones? Interestingly, he doesn’t. Yannic just covers what he finds most interesting. We’ll cover why this is an amazing strategy for the long term.
- Fighting bias in Large ML Models- Models like Stable Diffusion and GPT-3 are amazing because they have been trained on the data available on the internet. This allows them to pull off some crazy achievements. However, this also means that they encode all the biases on the internet. We cover the debate about this (gate things + add checks vs open everything and let things break), and how we can potentially limit the pitfalls of these amazing technologies.
For those of you that are interested in the full, unedited conversation, you can watch the video below. Windows was giving me weird recording issues, so the audio when I speak is really low. You might need to turn on the subtitles there. But overall, it was a great conversation and a lot of people really enjoyed it. Yannic is very insightful.
Let’s get right into it.
Stop Overthinking Things
A lot of people want to get into content creation (or start their side hustle), and often reach out to me for advice on the same. Often they ask me what the best field is, whether they should create YouTube videos or write blogs, what topics they should cover, etc, etc. Here’s a newsflash- None of that matters. Whether you start looking into Reinforcement Learning or Evolution, get into Computer Vision or NLP (why is this even a debate), write your tutorials in TensorFlow, PyTorch, or Keras- almost all these details are basically irrelevant. Here’s what you should do- create the content that would have helped you the most.
Yannic and I both created content that best fit our needs. It matched the needs that we were intimate with. In doing so, we created solutions that were useful to other people. If you’re trying to figure out what to do, don’t worry about what the ‘hot field’ is. Stop worrying about what kind of content sells, what your best target audience will be, what voice to use etc. Just focus on you.
Let’s get back to Yannic and me. Both of us break down ML papers pretty regularly. However, there is a difference in how we do it. Yannic comes from an Academic background. As he mentioned in our conversation- he saw that there was a large gap between the skills of Students first getting into research and the cutting-edge AI research that they had to interact with. His work bridges this gap. His analysis is very thorough, and he covers a lot of little details about the paper. His paper breakdowns reflect the academic rigor of his journey and touch upon details that people might overlook. Consequently, he attracts a lot of students and hardcore AI researchers. That is how I found Yannic’s channel, many years ago.
I largely taught myself about ML. I learned Machine Learning by freelancing for small businesses and organizations (I would quite literally invent my own jobs and reach out to them) and reading into the solutions for the specific challenges they faced. When I read the papers/engineering blogs, I didn’t have any supervisors that could really explain the papers to me. As a result, I got first-hand experience in how unreadable and complicated ML research papers could be. Thus, I decided to keep my content easy to understand and focus on what I found most essential in any given paper/idea. Such breakdowns would have made my life a lot easier when deciding what aspects of different solutions I wanted to integrate into my systems.
My work tends to attract a lot of Deep Learning Engineers and/or upper-level management involved in Machine Learning. People who don’t necessarily have the time/ability to sit down and read through 6–7 different papers to piece things together, but still want to stay on the cutting edge.
Yannic does mostly YouTube videos because he finds them the most natural (and it’s also much easier to pack all that information through them). I like writing a lot more because it allows me to play music in the background and requires less setup (Medium does a very good job making writing easy). Both have us have seen great results. Thanks to the internet you’re connected to the whole world. Find the path that vibes most with you and stick to it. You will see amazing results regardless.
Consistency and a good process will beat perfection
-A general maxim for life I like to live by
Now the next thing that might be on your mind is how Yannic is able to go through all the content being published online and find the most impactful developments. After all, there is a lot of information online. You’re probably overwhelmed by all the developments in tech. I have over 100 drafts, papers to cover, and ideas because my readers share new and interesting ideas/papers/resources with me all the time. I’m not exaggerating. Yannic is a much bigger creator than I am, and he faces this problem on a much bigger scale than I do. So what fancy magic tricks does he use to filter for the most important developments?
Do what interests you
Yannic talked about how he doesn’t really cover anything for views/because it will blow up. Instead, he covers the topics and ideas he finds most interesting. If he likes something, he will talk about it. If he doesn’t, he won’t.
This might seem like a trivial strategy but this is effective for many reasons. Firstly, this way you prevent burnout better. If you’re looking only into ideas and topics you’re interested in, you will not spend a lot of time forcing yourself to grind through things you don’t care about. Mentally, this will free a lot of energy. Content creation/your side hustle isn’t something you will be able to do full-time as you’re starting out. And you will have to invest a lot of time and effort into it before you start seeing major results. If you burn yourself out before that point, you will just have put in a lot of effort for no returns.
This has another benefit- you will be able to dig much deeper into topics. If you’re genuinely interested in an idea, you will be able to put a lot more effort into researching the intricacies. I am pretty interested in Active Learning, Sparsity, and other ways to reduce costs in ML model training. I’ve covered these ideas many times throughout my content. Recently, I found a very interesting paper by Meta AI that covered how we could reduce dataset sizes in Image Classification benchmarks. I decided to break it down, instead of the trendier topics like Text-Image generators or the huge Large Language Models being released. It ended up being my most popular article yet, even though it hasn’t even been out for 20 days. And yes, me calling this finding shocking is meant to be sarcastic. A lot of people thought I genuinely believed that throwing obscenely large data corpora at huge networks was enough to make useful breakthroughs.
Don’t worry about what will trend or what your audience will like. I’ve wasted a lot of time, energy, and mental peace trying to find the winning formula; to find the perfect article structure/topic to make me blow up. I have literally copy pasted articles of mine that did well and just changed the details to match a new breakdown. None of that has worked. Focus on what you find the most interesting, and stick to it for 2 years. You will see insane results from it. Don’t worry about what’s hot. Forget anybody that tells you that this industry/topic will dominate the future. If something doesn’t interest you, ignore it. If you like it, get into it. Don’t beat your head against the wall, doing things that are a struggle. Do what feels natural. As long as you’re solving a problem, people will see the benefits and reach out to you.
For the final highlight of the conversation, I would like to move on from entrepreneurship/content creation to an important discussion that we touched upon in our conversation. How can we deal with the Bias embedded in Machine Learning Models? Especially the ones trained on huge datasets like GPT-3, Google’s PaLM, and Stable Diffusion?
Tackling AI Bias
AI bias is something that needs to be dealt with. Detecting and tackling bias in datasets is crucial, especially as we see Machine Learning integrated more directly into society. This is especially true of large language models, which will be embedded into other software solutions. It is important that they don’t propagate harmful biases, especially against weaker sections of society. I went over a question that I have personally been thinking about-
Should we implement very strict checks and gate everything that doesn’t match those standards (like his GPT 4-Chan, which I covered here ), or should we open up the systems to public scrutiny and let the community work on fixing things, even if it could cause damage short-term
Yannic brought up an amazing comparison with the cyber security domain that blew my mind. When it comes to patching security vulnerabilities, the approach of opening up everything has paid off in spades. Outside Devs are able to collaborate and fix problems at lightning speeds. Opening things allows for new features and patches to be added very quickly. And everyone learns a lot from it.
Because Stable Diffusion is fully opened up, we have seen the full power of human creativity unleashed. We have plugins, demos, and modifications made to it. Far from making artists obsolete, it has given them a lot more power (more on this in an upcoming article). Because it has been opened up, people can fully explore it, both the good and the bad.
We have seen Meta AI recently adopt this mentality. They have released some path-breaking models/research fully. Researchers can fully poke around their open source LLM, Sphere, and even change the data the model is trained on/searches through. This will have amazing downstream impacts, as covered here
Once again, you can catch our conversation here. Make sure you check out Yannic’s work here, he is a goldmine of knowledge in Deep Learning Research.
If you’re looking to get into ML, this article gives you a step-by-step plan to develop proficiency in Machine Learning. It uses FREE resources. Check it out.
For Machine Learning a base in Software Engineering, Math, and Computer Science is crucial. It will help you conceptualize, build, and optimize your ML. My daily newsletter, Technology Interviews Made Simple covers topics in Algorithm Design, Math, Recent Events in Tech, Software Engineering, and much more to make you a better developer. I am currently running a 20% discount for a WHOLE YEAR, so make sure to check it out.
I created Technology Interviews Made Simple using new techniques discovered through tutoring multiple people into top tech firms. The newsletter is designed to help you succeed, saving you from hours wasted on the Leetcode grind. I have a 100% satisfaction policy, so you can try it out at no risk to you. You can read the FAQs and find out more here
Feel free to reach out if you have any interesting jobs/projects/ideas for me as well. Always happy to hear you out.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, or just say hi. Check out the free Robinhood referral link. We both get a free stock (you don’t have to put any money), and there is no risk to you. So not using it is just losing free money.
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
If you’re preparing for coding/technical interviews: https://codinginterviewsmadesimple.substack.com/
Get a free stock on Robinhood: https://join.robinhood.com/fnud75
