- Everyday AI
- Posts
- Inside MIT's Viral AI Study. The reasons why 95% of AI pilots do not fail
Inside MIT's Viral AI Study. The reasons why 95% of AI pilots do not fail
Microsoft adds Anthropic models to Office AI, Apple has minimal AI updates at WWDC, Meta inks $140M deal with Black Forest Labs and more!
Everyday AI
September 09, 2025
š Subscribe Here | š£ Hire Us To Speak | š¤ Partner with Us | š¤ Grow with GenAI
Outsmart The Future
Today in Everyday AI
6 minute read
š Daily Podcast Episode: The MIT ā95 % of AI pilots failā study has taken over the internet, and itās one of the worst studies Iāve ever read. So, whatās the truth and how does it impact you? We break it down. Give it a listen.
šµļøāāļø Fresh Finds: OpenAI to keep Standard Voice Mode, Google Veo 3 now generates vertical videos and NVIDIA calls rivals AI doomers. Read on for Fresh Finds.
š Byte Sized Daily AI News: Microsoft adds Anthropic models to Office AI, Apple has minimal AI updates at WWDC and Meta inks $140M deal with Black Forest Labs. For that and more, read on for Byte Sized News.
š§ Learn & Leveraging AI: Should you be worried about your AI implementation failing? We break down the fluff and let you know the truth. Keep reading for that!
ā©ļø Donāt miss out: Did you miss our last newsletter? We talked about Google admitting open web decline, OpenAI backing an animated feature film, Perplexity unveiling Perplexity for Government and more. Check it here!
Inside MIT's Viral AI Study. The reasons why 95% of AI pilots do not fail š¤„
MITās ā95% of AI pilots failā headline is a litmus test: will people think critically, or just swallow clickbait?
Unfortunately, the latter won.
The MIT ā95 % of AI pilots failā study has taken over the internet, and itās one of the worst studies Iāve ever read.
(And Iāve read thousands.)
ā³ So, whatās the truth?
ā³ Is AI a bubble thatās about to pop?
ā³ Why is this study rubbish?
ā³ And how does it impact you?
Donāt miss out.
Also on the pod today:
ā¢ MIT Studyās Biased Participant Selection š¤„
ā¢ How Media Sensationalizes AI Study Results š¢
ā¢ Comparison With Reputable AI ROI Studies āļøļø
Itāll be worth your 36 minutes:
Listen on our site:
Click to listen
Subscribe and listen on your favorite podcast platform
Listen on:
Hereās our favorite AI finds from across the web:
New AI Tool Spotlight ā TaskWand turns words into workflows in seconds, Cake AI turns applications into interviews and Tokyo tracks AI usage and cost by customer.
ChatGPT ā OpenAI is deciding to keep Standard Voice Mode after giving it a 30-day sunset.
Last month we announced that everyone now has access to Advanced Voice Mode, with usage limits expanded from minutes per day to hours for free users and near unlimited for Plus.
We also announced that Standard Voice Mode would be retired after a 30-day sunset. Weāve heard
ā Nick Turley (@nickaturley)
2:16 PM ā¢ Sep 9, 2025
Google ā Google Veo 3 can now generate vertical videos.
Google ā Google has released new guidelines for its software engineers.
NVIDIA ā NVIDIA is accusing its rivals of being AI doomers as U.S. debates sales of China AI chips.
Microsoft - Microsoft and Ralph Lauren are partnering to create Ask Ralph, a new conversational AI experience.
AI Startups ā NVIDIA-backed AI startup Reflection nears a deal for $5.5B valuation
Trending in AI ā Amsterdam company Nebiusā shares have soared 40% after Microsoft AI deal.
1. Microsoft Adds Anthropic Models to Office AI Arsenal š
Microsoft will start paying to integrate Anthropic's models alongside OpenAI's into Office 365 apps, signaling a strategic broadening of its AI suppliers as it also develops in-house models and brings DeepSeek into Azure. Developers found Anthropic's Claude Sonnet 4 can outperform OpenAI on tasks like crafting more polished PowerPoint slides and automating Excel financial functions, prompting Microsoft to route Anthropic access through AWS.
The shift is timely: after years of heavy reliance on OpenAI and more than $13 billion of investment into the startup, Microsoft is diversifying to boost feature quality and resilience while keeping Office AI prices unchanged.
2. Apple Sidelines Flashy AI Demos at iPhone Event ā
Appleās September event leaned into hardware and behind-the-scenes AI rather than headline-grabbing agentic features, with only brief nods to Apple Intelligence and updates first teased at WWDC. Tim Cook framed the new iPhones as big leaps in silicon, speed, and battery life, while Apple emphasized local neural engines and on-device models powering smoother gameplay and background intelligence.
That shift signals Apple is prioritizing reliable, integrated AI rather than spotlight consumer-facing assistants ā a contrast to recent launches from Google and Samsung that put AI front and center.
3. Meta Inks $140M Deal with Black Forest Labs for Image AI š¼ļø
Meta has signed a multi-year agreement to license image-generation AI from German startup Black Forest Labs, committing roughly $35 million in year one and $105 million in year two, according to Bloomberg. The deal signals Metaās fast push to bulk up its AI creative stack as rivals race to own foundational generative tooling.
For creators and companies, that could mean faster access to higher-quality image models through Metaās ecosystem, changing how marketing, design and content production scale and budget.
4. NVIDIA Debuts Rubin CPX GPU for MillionāToken Contexts š„ļø
NVIDIA announced the Rubin CPX at its AI Infrastructure Summit, a GPU built to handle context windows larger than 1 million tokens and optimized for longāsequence inference like video generation and complex code tasks.
The chip will fit into a ādisaggregated inferenceā stack and aims to speed workflows that currently choke on long context lengths, with availability slated for late 2026.
5. Claude Adds Real File Creation to Its Toolkit š
Anthropic Claude can now generate and edit Excel, Word, PowerPoint, and PDF files directly (preview available to Max, Team, and Enterprise users), turning conversational outputs into ready-to-use deliverables. This update lets users upload data and receive polished spreadsheets, slide decks, and documents with working formulas, charts, and written insights in minutes.
Anthropic warns the feature uses a private compute environment with internet access, so users should monitor sensitive data when enabled.
6. Googleās AI Overviews Threaten News Clicks, Publishers Warn š¢
Publishers warn Googleās new AI Overviews are siphoning clicks and ad revenue by answering searchersā questions directly on the results page, with DMG Media telling the UKās Competition and Markets Authority that click-throughs fell by up to 89%.
Reach and other outlets say the shift could deepen a structural squeeze already caused by social platforms, forcing newsrooms to chase newsletters, WhatsApp alerts and other direct channels to retain audiences and revenue.
š¦¾How You Can Leverage:
Imagine if you talked to 52 people that you knew probably hated pizza and then circulated a study on how "95% of people hate pizza."
Their back-of-the-napkin 'study' boldly claimed that 95% of GenAI pilots fail.
MIT's viral "State of AI in Business 2025" study has the sample size of a high school survey project and the conflicts of interest of a tobacco company studying lung cancer.
Yet somehow, very few in the mainstream and social media cared to actually read and dissect the study, instead parroting the shaky-at-best claims that MIT constructed that were a foregone conclusion.
Because of that, now everyone thinks AI is dead!
The AI bubble is popping!
95% of AI pilots fail!
Nah.
That's why we had to give the #HotTakeTuesday treatment to this methodological disaster on todayās edition of Everyday AI.
While competitors panic and pause their AI initiatives based on this overhyped MIT marketing ploy, smart executives see through the quicksand this study was built on and accelerate past the fence-sitters.
Hereās how to spot the holes. (And why we REALLY encourage you to read this for yourself.)
1 ā Catch The Embarrassingly Small Sample š„
Here's where it gets really embarrassing for MIT.
Their world-shaking 95% failure rate came from conversations with 52 organizations. That's like asking one person from each state about pizza preferences and declaring a national food crisis.
But it gets worse.
They surveyed 153 senior leaders total but cherry-picked their viral headline from just those 52 interviews using "directional accuracy." Academic speak for "trust us, bro, we've got a feeling about people feeling this way.ā
Thereās a reason they didnāt survey thousands openly. The 95% that fits their narrative wouldnāt exist.
Meanwhile, organizations doing actual research tell completely different stories. IDC surveyed 4,000 decision makers and found $3.70 return for every $1 AI invested.
EY talked to business leaders and discovered 97% report positive ROI from AI implementations.
Microsoft went nuclear. They surveyed 31,000 professionals and found 66% experiencing measurable business benefits from AI tools.
When literally every other credible study contradicts yours by 600%, maybe the problem isn't AI pilots failing. Maybe itās because the study was defined to show AI failure from the start.
Try This:
Seriously donāt listen to us.
Just go read the marketing piece. Errrrā¦. āStudyā. Old school. Like, print it out and think.
If youāre anything like us, you prolly wonāt give too much credence to any future MIT studies around AI.
2 ā Notice The Rigged Recruitment Strategy š„
This is where MIT really tipped their hand.
They explicitly hunted for organizations "willing to discuss AI implementation challenges." Not companies using AI successfully. Not random enterprise organizations.
Just the struggling ones. (And 52 at that! Lolz)
It's like researching restaurant quality by only interviewing people who got food poisoning. Of course your results will be terrible when you exclusively target dissatisfied customers.
Then they measured ROI at exactly six months. Enterprise AI transformations typically need 12-24 months for meaningful financial returns. And thatās with traditional, clunky digital innovation. Not AI.
Giving a generational tech 6 months to show ROI on a P&L statement is like calling marathon results after the first 800m. No one would do that. (Except, this piece of MIT Marketing, apparently.)
Here's what really exposes the setup though. If 95% of AI pilots genuinely failed, why are 90% of Fortune 500 companies actively scaling generative AI in production right now?
Companies don't invest billions in failed experiments.
Try This:
Choosing a 6-month measurement period for any ROI pilot measurements is obviously laughable. At best, you should be looking at 12-24 months, and this article from Data Society is a great guide on what to measure.
(You knowā¦ instead of vibes from a weekend of phone calls.)
3 ā Recognize The Academic Sales Pitch š„
Here's where MIT went full infomercial mode.
After spending about 20 pages explaining how AI pilots fail everywhere on their SUPER vast dataset, they casually mention the solution. Organizations trapped on the "wrong side" need to stop using static tools and start partnering with vendors offering custom solutions.
Which happens to be MIT's own project for working with Agentic AI. They even had the audacity to position their basically unknown protocol alongside Anthropic's MCP and Google's A2A frameworks.
Like they knew their āstudyā would get eyeballs, so they polished their minor league solution alongside the All Star team like people would think they belong there. Lolz.
Even worse?
The study was essentially gatekept behind Google forms while most people couldn't even access what they were debating. Then Fortune picked up the story in August and every major publication copy-pasted the same headline within hours.
All because 52 conversations became "scientific evidence" that conveniently validated MIT's solution that apparently needed some propping.
Try This:
Anyone else get a hard pass vibes on this NANDA thing or just us?
Our take here? Stick with the leaders when it comes to agentic protocols. Googleās A2A is supported by all, even their chief competitor Microsoft. Anthropicās MCP may be even more popular, and has been adapted by their biggest competitor, OpenAI.
Reply