The Most Important Content in AI, Software, Business, and Tech- 09/11/2024
Content to help you keep up with Machine Learning, Deep Learning, Data Science, Software Engineering, Finance, Business, and more
A lot of people reach out to me for reading recommendations. I figured I’d start sharing whatever AI Papers/Publications, interesting books, videos, etc I came across each week. Some will be technical, others not really. I will add whatever content I found really informative (and I remembered throughout the week). These won’t always be the most recent publications- just the ones I’m paying attention to this week. Without further ado, here are interesting readings/viewings for 09/11/2024. If you missed last week’s readings, you can find it here.
Reminder- We started an AI Made Simple Subreddit. Come join us over here- https://www.reddit.com/r/AIMadeSimple/. If you’d like to stay on top of community events and updates, join the discord for our cult here: https://discord.com/invite/EgrVtXSjYf. Lastly, if you’d like to get involved in our many fun discussions, you should join the Substack Group Chat Over here.
Community Spotlight: This AI and Entertainment Survey
Fellow cultist Fred Graver is looking into understanding the demand for content around AI. If you can, go help him by filling it out this survey to help him understand how you like to get your AI information, what you’d like to see in the future, your hopes and fears from AI etc. His post on the Subreddit-
“I am currently enrolled in the Professional Certificate program for PMs at MIT. As part of this year-long course of study, I need to do a final project — designing a product / platform from scratch.
I am in the early stages of the “Jobs to Be Done” inquiry and need to survey 100 or more people. If you’re interested in AI, entertainment and media, and wouldn’t mind helping a struggling student out, I’d greatly appreciate you taking 5 minutes to answer the survey.”
If you’re doing interesting work and would like to be featured in the spotlight section, just drop your introduction in the comments/by reaching out to me. There are no rules- you could talk about a paper you’ve written, an interesting project you’ve worked on, some personal challenge you’re working on, ask me to promote your company/product, or anything else you consider important. The goal is to get to know you better, and possibly connect you with interesting people in our chocolate milk cult. No costs/obligations are attached.
Previews
Curious about what articles I’m working on? Here are the previews for the next planned articles-
Boeing, DEI, and 9 USD Engineers.
The economics of Open Source.
Highly Recommended
These are pieces that I feel are particularly well done. If you don’t have much time, make sure you at least catch these works.
This week- I’m going to highlight only two since they bring up extremely important discussions, and I want to get your opinions on them. The other work I share in this edition are also excellent, but in terms of relevance and impact to society- I think the two we discuss stand out.
Freedom of Speech & Platform Liability
Tobias Jensen discusses content moderation on social media platforms and recent cases which trend towards preventing the harms that can (and has) been caused by social media messages not being regulated properly.
I have a few thoughts about it (not very well thought out, but that’s the point of starting conversations)-
- If we start holding platforms accountable for the actions of users, it can create a dangerous pressure for censorship where platforms go out of their way to promote sanitized content. Especially when platforms start using AI moderation for the sake of efficiency (cue the story about Facebook banning a guy for teaching Python and Pandas courses b/c it thought he was talking about live animals, not coding). In general, I’m a lot more concerned about the risks of over-censorship than I am about the risks of under-censorship.
- None of us are really prepared for the drug that is social media, and you definitely want some regulation to protect people (especially children). Maybe follow China’s Lead and have a government social media that actively promotes certain values and let that be the only platform for kids below a certain age (also we need better account checks)? Do we embrace Plato and start mass brainwashing society into believing the noble lie?
- Parents/schools being unable to protect their kids from social media challenges (especially obviously dangerous ones) is an interesting outcome. I wonder what it is specifically that makes Social Media such an overpowering influence over family and education- two institutions that never fail to stress how they are shaping us to prepare for the future.
We definitely need to rework our approach to ensure that people deal with the internet and its platforms. I’m unsure what specifically needs to happen, so it’s an open invitation for all of you to share your thoughts.
Whatever your thoughts, I strongly recommend checking out Tobia’s article because it’s great coverage of the cases that will undoubtedly be huge for tech.
The four events we will take a closer look at in today’s post to understand how the regulatory temperature towards online platforms is shifting:
- The draft legislation to sunset Section 230 (link)
- The deadly “black-out challenge” on TikTok (link)
- The charges against Telegram CEO Pavel Durov (link)
- Brazil blocks X (link)
Doctors Go to Jail. Engineers Don’t.
Extremely insightful analysis by Sergei Polevikov. He nails one of the biggest challenges to AI adoption in clinical diagnosis- doctors bear a lot of risk for using AI, while model developers don’t. This creates an obvious barrier against adoption, and it’s important to see how we can align incentives better. There are no real answers on how on my end, but the floor is open to anyone who wants to come in and discuss their thoughts.
From where I sit, it’s the “low-hanging fruit” of AI — the no-risk, zero-liability zones like AI scribing and the so-called “Rev Cycle” management. These non-clinical areas are getting all the hype, as I detail in my digital health industry review of 2024.
But when it comes to clinical progress — where doctors could actually use AI as a “second opinion” tool — we’re seeing a whole lot of nothing.
Why is that?
This is the number one problem and the tragedy of AI adoption in medicine:
Until the liabilities and responsibilities of AI models for medicine are clearly spelled out via regulation or a ruling, the default assumption of any doctor is that if AI makes an error, the doctor is liable for that error, not the AI.
It’s important to point out that if AI makes an error, but the recommendation aligns with the “standard of care,” the doctor may not be held liable for that mistake. So, the real question is:
Until AI itself becomes the standard of care, why would doctors and hospitals pour money and resources into AI systems, when the minimal liability risk is to stick with the existing standard of care?
…
Here is another issue:
Administrators, not medical professionals, are the ones evaluating AI algorithms.
I put a lot of work into writing this newsletter. To do so, I rely on you for support. If a few more people choose to become paid subscribers, the Chocolate Milk Cult can continue to provide high-quality and accessible education and opportunities to anyone who needs it. If you think this mission is worth contributing to, please consider a premium subscription. You can do so for less than the cost of a Netflix Subscription (pay what you want here).
Many companies have a learning budget, and you can expense your subscription through that budget. You can use the following for an email template.
I provide various consulting and advisory services. If you‘d like to explore how we can work together, reach out to me through any of my socials over here or reply to this email.
Other Good Content
Very interesting article on using an older imaging technique in modern cell imaging. I think the the use of black white/crude images reduces the number of features that a model would have to learn, allowing for greater learning capacity on what’s left.
On another note- what do y’all think on this imaging + super resolution? That could help the model explore artifacts that are currently not being explored b/c they don’t get captured (although it might also cause overfitting, but in that case learning about the tradeoff would be interesting). Credit Brita Belli
A rudimentary method of detecting cells in samples using light called brightfield microscopy — first developed in the 17th century — is proving to be a powerful tool for analyzing changing cell states for tech-enabled drug discovery thanks to advances in machine learning. And unlike fluorescence-based approaches that are typically used for phenomics observations, including Cell Painting, brightfield microscopy does not perturb the cells, allowing scientists to examine multiple layers of biology on the same sample — a game-changer when it comes to improving cost, speed, and experimental consistency.
With a brightfield microscope, white light is transmitted through a cell sample onto a detector. To the human eye, the resulting image looks like a vague collection of gray blobs and black dots — nowhere near as detailed as the multi-colored Cell Painting images that Recursion scientists had been using prior.
But during the company’s 2021 “Hack Week” — when employees are given free reign to test out-of-the-box theories — a team at Recursion decided it was a method worth exploring. They reasoned that brightfield would require fewer steps, adding to the speed and efficiency of the high throughput process.
Excellent Collection of resources (as usual) by Logan Thorneloe . I like the diversity of his topic selections (it makes my job my job easier) and his decision to put time into explaining why he picked sources is going to be very insightful.
How to Optimize LLM Pipelines with TextGrad
Damien Benveniste, PhD always drops heat when it comes to in-depth ML Engineering content.
It is possible to build very robust pipelines by implementing routing, check, validation, and fallback nodes at the expense of higher latency. Each node requires its own system prompt, and tuning them correctly can be a challenge as the change in one prompt will influence the inputs into the other ones. With large pipelines, we run the risk of accumulating errors from one node to the next, and it can become tough to scale the pipeline beyond a few nodes.
A new strategy that has silently been revolutionizing the field of orchestrated LLM applications is LLM pipeline optimization with optimizers. DSPy has established the practical foundation of the field, and new techniques like TextGrad and OPRO have emerged since then. Let’s study the ideas behind TextGrad in a simple example.
AI worse than humans in every way at summarising information, government trial finds
Interesting study that shows LLMs (albeit slightly outdated ones) struggle to capture semantic nuances of knowledge-intensive documents. I touched upon this idea when I did the writeup on Knowledge-Intensive NLP for Legal AI.
Credit to Gergely Orosz for sharing the article (over here). I responded to it with discussions on the math on why non-specialized LLMs would struggle with semantic nuance (unless you have MoE and elite routing) over here.
The test involved testing generative AI models before selecting one to ingest five submissions from a parliamentary inquiry into audit and consultancy firms. The most promising model, Meta’s open source model Llama2–70B, was prompted to summarise the submissions with a focus on ASIC mentions, recommendations, references to more regulation, and to include the page references and context.
Ten ASIC staff, of varying levels of seniority, were also given the same task with similar prompts. Then, a group of reviewers blindly assessed the summaries produced by both humans and AI for coherency, length, ASIC references, regulation references and for identifying recommendations. They were unaware that this exercise involved AI at all.
These reviewers overwhelmingly found that the human summaries beat out their AI competitors on every criteria and on every submission, scoring an 81% on an internal rubric compared with the machine’s 47%.
Human summaries ran up the score by significantly outperforming on identifying references to ASIC documents in the long document, a type of task that the report notes is a “notoriously hard task” for this type of AI. But humans still beat the technology across the board.
Reviewers told the report’s authors that AI summaries often missed emphasis, nuance and context; included incorrect information or missed relevant information; and sometimes focused on auxiliary points or introduced irrelevant information. Three of the five reviewers said they guessed that they were reviewing AI content.
The reviewers’ overall feedback was that they felt AI summaries may be counterproductive and create further work because of the need to fact-check and refer to original submissions which communicated the message better and more concisely.
Roaring Bitmaps: A Faster optimization for sets
Your boy learning (and sharing) new software engineering ideas on the sister newsletter.
I was reading Google’s report- “Procella: Unifying serving and analytical data at YouTube”, which solves some very important Data Engineering problems for YouTube…
While reading it, an idea that I’d never heard of stood out to me: Roaring Bitmaps. They were able to use it to reduce their latencies by 500 orders of magnitude (that’s so large that I’d buy it if it was a typo)-
By storing these indices as Roaring bitmaps [10] we are able to easily evaluate typical boolean filters (i.e. ‘WHERE 123 in ArrayOfExperiments OR 456 in ArrayOfExperiments’) efficiently, without having to go through the normal evaluation pathway. In current production use cases we have found that experiment analysis queries have end to end latencies reduced by ∼ 500x orders of magnitude when we apply this technique.
So, I thought this would be a good time to learn about Roaring Bitmaps. Turns out Roaring Bitmaps are one of the most impactful data structures used by organizations today, so knowing them is key for modern software engineering. The following article will be my overview of the basics of roaring bitmaps and how they’re used by organizations.
Memory in use reported by Redis (matches RSS of the redis-server process): 129.48G.
With the same dataset migrated to standalone bitmapist server under the same load: RSS reported at about 300M.
-These numbers are unreal. Source
The Disturbing Truth about Green Powders (AG1)
Good video on the misleading marketing behind Green Powders. I didn’t realize how big they’d gotten. It’s quite surprising how many people will pay lots of money to not eat vegetables (it’s really not that hard guys).
Dissecting Multiplication in Transformers: Insights into LLMs
Transformer-based large language models have achieved remarkable performance across various natural language processing tasks. However, they often struggle with seemingly easy tasks like arithmetic despite their vast capabilities. This stark disparity raise human’s concerns about their safe and ethical use, hinder their widespread this http URL this paper, we focus on a typical arithmetic task, integer multiplication, to explore and explain the imperfection of transformers in this domain. We provide comprehensive analysis of a vanilla transformer trained to perform n-digit integer multiplication. Our observations indicate that the model decomposes multiplication task into multiple parallel subtasks, sequentially optimizing each subtask for each digit to complete the final multiplication. Based on observation and analysis, we infer the reasons of transformers deficiencies in multiplication tasks lies in their difficulty in calculating successive carryovers and caching intermediate results, and confirmed this inference through experiments. Guided by these findings, we propose improvements to enhance transformers performance on multiplication tasks. These enhancements are validated through rigorous testing and mathematical modeling, not only enhance transformer’s interpretability, but also improve its performance, e.g., we achieve over 99.9% accuracy on 5-digit integer multiplication with a tiny transformer, outperform LLMs GPT-4. Our method contributes to the broader fields of model understanding and interpretability, paving the way for analyzing more complex tasks and Transformer models. This work underscores the importance of explainable AI, helping to build trust in large language models and promoting their adoption in critical applications.
The Story Of The Player That Is “Too Dumb To Feel Pressure”
A video that addresses the ignorant slander that our lord and savior Cold Palmer is stupid. I’m telling y’all, that boy is the truth.
If you liked this article and wish to share it, please refer to the following guidelines.
I put a lot of effort into creating work that is informative, useful, and independent from undue influence. If you’d like to support my writing, please consider becoming a paid subscriber to this newsletter. Doing so helps me put more effort into writing/research, reach more people, and supports my crippling chocolate milk addiction. Help me democratize the most important ideas in AI Research and Engineering to over 100K readers weekly.
PS- We follow a “pay what you can” model, which allows you to support within your means. Check out this post for more details and to find a plan that works for you.
I regularly share mini-updates on what I read on the Microblogging sites X(https://twitter.com/Machine01776819), Threads(https://www.threads.net/@iseethings404), and TikTok(https://www.tiktok.com/@devansh_ai_made_simple)- so follow me there if you’re interested in keeping up with my learnings.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
