ChatGPT’s Game-Changer Multi-Modality: What This Means
What is multi-modal AI and does it deserve the hype it’s generating
If you went to LinkedIn over last week/2 weeks, you were probably inundated by people losing their minds over GPT integrating multi-modality into its capabilities. Normally, I would take some time to tell you that this is another example of the hype machine working overtime to sell you another fundamentally useless idea.
Well, this time is different. Multi-modality is a genuinely powerful development, one that does warrant the attention that it is receiving. In this article, I will give you a quick introduction to multi-modality, why it’s a big deal for AI Models, and some problems it can come with (remember, nothing is a silver bullet).
Join 150K+ tech leaders and get insights on the most important ideas in AI straight to your inbox through my free newsletter- AI Made Simple
Multimodality 101
- What is multi modal AI- Simply put, multi-modal AI refers to AI that integrates multiple types of data (multiple modalities of information). Traditionally, we develop language models for language, acoustic models for sound, statistical models for tabular data etc. Multi-modal models are trained with a mixture of these inputs in the same training process. This is typically done by running the input through embedding models that will create vector representation of your data in a common n-dimensional space
- Why multimodality is a big deal- Instead of getting all mathy, I want you to go outside right now. Take a walk. Now look at the sky and imagine that you had a jet pack. Think of how many more paths you could take- even if you stayed in the same geographic area. Multi-modality adds another dimension to your data- allowing your model to sample from a search space that is an order of magnitude greater. In our walking example, we went from x² possible points to hit to x³ points. When introducing their multi-modal AI infrastructure Pathways, Google wrote the following-
- The danger of multi-modality- While multi-modality is great for model performance, it doesn’t really address the more fundamental issues with GPT and LLMs that hinder their larger scale adoption. The increased search space rives up costs, multi-modality doesn’t really impact hallucination, and many of the problems with unreliability and fragility still persist. I still maintain that many of the use-cases that people hype around these models can be better addressed by simpler technologies. Don’t let the shiny new thing distract you from the fundamentals.
Overall, multi-modality is really cool. It enables all kinds of applications in compression, data annotation, labeling etc. This might be a bit of a heretical take, but I’m personally more excited by multi-modal embeddings than I am by the multi-modal AI models themselves. I might be the only one here, but I just see more utility in developing better embeddings than I do with building better models. That being said, in the right circumstances integrating multi-modal capabilities into your AI Models can definitely be a big dub.
If you liked this article and wish to share it, please refer to the following guidelines.
If you find AI Made Simple useful and would like to support my writing- please consider becoming a premium member of my cult by subscribing below. Subscribing gives you access to a lot more content and enables me to continue writing. This will cost you 400 INR (5 USD) monthly or 4000 INR (50 USD) per year and comes with a 60-day, complete refund policy. Understand the newest developments and develop your understanding of the most important ideas, all for the price of a cup of coffee.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
In Plain English
Thank you for being a part of our community! Before you go:
- Be sure to clap and follow the writer! 👏
- You can find even more content at PlainEnglish.io 🚀
- Sign up for our free weekly newsletter. 🗞️
- Follow us on Twitter(X), LinkedIn, YouTube, and Discord.
