• Everyday AI
  • Posts
  • 7 common LLM mistakes and how to avoid them

7 common LLM mistakes and how to avoid them

Amazon Alexa gets Claude upgrade, Apple and NVIDIA look to invest in OpenAI, Dell shares rise and more!

Author

Everyday AI
August 30, 2024

Sup y’all 👋

Here in the good ol U-S of A, we have a federal holiday on Monday. So, we’ll see ya back in action on Tuesday with AI News that Matters.

Hope you can join us.

Oh, we’re doing a little giveaway. All you gotta do is VOTE YES if you’re interested in winning a free 90-minute GenAI consult. 

(For your own learning or for your biz. You choose!)

Do you want a free, 1-on-1, 90-minute LLM training session?

Vote YES to be entered, and check our announcement next week which we'll be making exclusively in our newsletter.

Login or Subscribe to participate in polls.

We’ll announce the winner exclusively in our newsletter next week, so make sure you read/watch the for announcement.

✌️

Jordan

(Let’s connect. Just tell me who ya are! lol)

Outsmart The Future

Today in Everyday AI
8 minute read

🎙 Daily Podcast Episode: We’ve trained thousands of business leaders on using Large Language Models, and we see the same mistakes. Over and over and over again. So, we’re tackling the 7 most common LLM mistakes and how to avoid them. Give it a read or listen.

🕵️‍♂️ Fresh Finds: FDA to consolidate AI efforts, Big tech’s secret to acquire AI unicorns, Tom Hanks warns of fake AI ads and more. Read on for Fresh Finds.

🗞 Byte Sized Daily AI News: NVIDIA and Apple look to invest in OpenAI, Amazon Alexa gets a Claude AI makeover, Dell shares rise and more. Read on for Byte Sized News.

🚀 AI In 5: Can AI help you go viral? See if this AI video tool can help catapult you to viral vertical video fame. See it here

🧠 Learn & Leverage AI: Pitfalls. Pitfalls everywhere. We’ve taken hundreds of hours of LLM training and have written an easy guide to help you avoid common LLM pitfalls. Keep reading for that!

↩️ Don’t miss out: Did you miss our last newsletter? We talked about Google Gems explained, Big tech and US Gov. partner and California AI bill advances. Check it here!

Stop making these 7 Large Language Model mistakes 👎

You wouldn't ride a unicycle on a highway. 🚳

Sure, that's technically a way you can travel.

↳ But that doesn't mean pedaling a unicycle is an acceptable way to travel from point A to point B.

↳ That's how people are using Large Language Models.

↳ There's millions using LLMs like riding a unicycle on an interstate.

Don't worry.

We'll set the record straight and help you trade in that unicycle for a friggin Bentley.

(Or like a 2009 Toyota Prius hybrid. Whatever's your speed.)

On today’s show, we showed you how to Stop making these 7 Large Language Model mistakes.

Also on the pod today:

• Avoiding common LLM mistakes ⛔️
• Staying up to date with GenAI 📰
• Preparing for the future of work 🔮

It’ll be worth your 43 minutes:

Listen on our site:

Click to listen

Subscribe and listen on your favorite podcast platform

Listen on:

Here’s our favorite AI finds from across the web:

New AI Tool Spotlight – AFFiNE AI uses AI to help you better draw, write and present, Storyville gives you (or maybe your kids) personalized bedtime stories with the help of AI and Flownote uses AI to transcribe your meetings into concise summaries.

Trending in AI – Tom Hanks has issued a statement on Instagram to warn about fake AI video ads of himself circulating social media.

AI in Medical – The FDA’s drug center is consolidating its AI efforts under one council.

Big Tech – Here’s how big tech companies are acquiring AI companies without having to buy them.

LLMs – Cohere has updated its Command R series enterprise AI models.

Read This – The BBC is starting to use AI to generate subtitles.

AI in Politics – Japan’s military is planning to spend on AI and automation to combat its recruitment crisis.

AI Image Models — Leonardo AI has released an API to its foundational model, Phoenix.

1. NVIDIA and Apple Consider Investment in OpenAI 👀

Whoa.

NVIDIA and Apple are reportedly in discussions to contribute to OpenAI’s upcoming fundraising round, which could elevate the ChatGPT creator’s valuation to an astonishing $100 billion. This news follows Bloomberg's initial report and comes as OpenAI seeks new capital to combat a projected $5 billion loss by year-end while expanding its AI training and staffing efforts.

Additionally, Microsoft, which already holds a 49% stake in OpenAI, may also participate in this fundraising, highlighting the growing interest from major tech players.

2. Amazon's Alexa Set for Upgrade with Anthropic's Claude AI 🗣

Amazon is gearing up to enhance its voice assistant with the upcoming “Remarkable Alexa,” which will be powered by Anthropic’s Claude AI after previous versions struggled to meet user expectations. With a substantial $4 billion investment in Anthropic, the new assistant is expected to debut in mid-October, featuring improvements like daily AI-generated news summaries and a child-friendly chatbot.

However, users should prepare for a subscription model, potentially costing between $5 to $10 per month

3. Dell Technologies Gains Momentum with AI Server Demand 📈

Dell Technologies saw a 4% increase in shares following a strong demand for its AI-powered servers. The partnership with NVIDIA has paid off, leading to a 38% rise in revenue from Dell's infrastructure solutions group, totaling $25.03 billion, while their AI pipeline is now estimated between $11 billion and $13 billion.

Analysts are optimistic, with most rating Dell as a "buy" and raising their price targets, despite the stock being down 36% from its all-time high in May.

4. Google’s Approach to Disease Detection Through Sound Signals 🔊

Google is making strides in healthcare by utilizing sound signals to predict early signs of diseases, including tuberculosis. The tech giant has trained its AI model with 300 million audio samples of coughs and labored breathing, collaborating with Indian startup Salcit Technologies to potentially deploy this technology on smartphones in underserved areas.

This initiative could significantly improve early disease detection in high-risk populations, showcasing Google's commitment to revolutionizing healthcare accessibility.

5. Oprah Hosts Star-Studded Special on AI’s Impact 📺

Oprah Winfrey is set to host a compelling ABC special titled “AI Future Us,” premiering on September 12 at 8 PM ET. This event will feature notable figures, including OpenAI CEO Sam Altman and Bill Gates, as they explore the transformative effects of artificial intelligence on everyday life.

The program promises insightful discussions and demonstrations on AI's potential impact on jobs and society, with additional appearances by content creator Marques Brownlee and technology advocate Tristan Harris.

AI-powered shortcut Shortcut to create viral vertical videos?

Ever wondered how to create those viral vertical videos that dominate social media feeds?

Well today’s AI in 5 is for you!

We’re breaking down Spikes Studio, an AI-generated video creator that takes your long form video and creates recommended viral clips in vertical format.

We’ve literally taught thousands of business leaders how to prompt inside large language models.

And for the past year, we’ve seen the same mistakes.

Debunked the same myths.

And prioritized the same truths.

So, we thought it was time for a dedicated episode going over the 7 most common LLM mistakes that people make. 

So, let’s get to it. 👇

Mistake 7 – Not understanding a LLM’s knowledge cutoff 🧠

Forget what the companies are trying to tell you in their marketing. Even if a model is ‘connected to the internet’ it’s not always up to date.

In short:

  1. Models are trained on data.

  2. Data is scraped from the internet. (Whether that’s legal or not will be decided in the coming years)

  3. Humans train the models based on that data.

But the process between steps 2 and 3?

There’s an expiration date, of sorts. The model training process can take many months, in which case the model’s training data (and knowledge cutoff) only get more and more stale.

Do this instead:

The LMSYS Chatbot Arena has a pretty up-to-date list of what each popular model’s knowledge cutoff is.

Side note — it incorrectly lists the GPT-3.5 knowledge cutoff as September 2021 whereas it’s actually January 2022. (The rest all look good!)

Need to know more about what a knowledge cutoff is, how it impacts LLMs and how they work?

Mistake 6 – Not investigating internet connectivity 🛜

If Big Tech pinky promises that their model is connected to the internet, that means knowledge cutoffs don’t matter, right?

And that we can also feel confident in any model’s output?

Wrrroooooong.

The approach (and consistency) of how different models talk to the internet varies.

And sometimes, it’s downright awful. (We’re looking at you, Google!)

Do this instead:

Nothing beats first-hand experience of trying different time-sensitive queries, and observing how different “internet-connected” LLMs act.

Or, you can sit on the couch and watch as we did the heavy lifting for you.

This episode compares with real-world tests how LLMs interact with the internet. This single episode is gonna save you time, improve your accuracy, and cut down on those dang hallucinations.

Mistake 5 – Not managing your memory ⁉️

LLMs aren’t infinitely smart.

Just like us (and goldfish) they can only remember so many things.

And while models like Claude-3 and Gemini have stolen the show in terms of big memories and long context windows, they’re not always accurate.

And the GPT models still lag a bit behind compared to Anthropic and Google’s big brained models.

Do this:

It’s like how the Star Wars text scrolls up and out of the screen — LLMs can only retain so much information at a time.

Wanna dork out and go all-in on understanding tokenization and memory? If you really wanna up your LLM game, this is essential reading/watching.

Mistake 4 – Paying attention to screenshots 🖥️

Guess what a screenshot from a large language model means?

Absolutely nothing.

You can tell a model to parrot anything you want then share that screenshot online.

I’m rich!

Do this:

Screenshots are a dime a dozen.

“AI experts” are trying to share screenshots showing their super duper AI skills.

AI skeptics are trying to share their screenshots showing how dumb LLMs are.

  • All those things mean?

  • Those people don’t understand how models work.

  • If you really wanna show your work, you can always just share the chat URL, like this.

(Yeah, Jordan really didn’t win the lottery after all.)

Mistake 3 – Thinking that LLMs are deterministic 🚫

AI chats aren’t like search engines.

  • You can put 1 prompt in 100 times and get 100 very different answers.

  • Or 50 different answers.

  • Or 2 slightly different answers.

Large Language Models are generative by nature, which means their next-token prediction abilities are meant to be generative.

A little random. A bit unpredictable.

Do this:

Go into the OpenAI playground, and play around with Top-P, temperature and more. We’d love to walk you through this, step-by-step, but going through the process on your own really helps you understand how generative models actually work.

(Reply to this email if you’d be interested in an episode on the OpenAI Playground.)

Mistake 2 – Thinking copy and paste prompts work 💾

If you see someone shilling copy-and-paste prompts promising to solve all of your problems, run for the hills.

Don’t pay attention to Billy Boys like this.

Here’s the truth — prompts don’t really do anything.

Sure, they can get you from an F to a C- pretty quick, but that’s about all copy-and-pasting prompts is good for. Going from hot garbage to lukewarm trash.

Do this:

If you’re feeling reaaaaalllllly spicy, go read this 43-page research paper on chain-of-thought prompting. (We have.)

Or, you can just look at this graph and agree with math, science and logic: copy-and-paste prompts give you poor outputs and proper prompt engineering wins out every single time.

(If you’re not a graphs person, this says that ‘few shot’ prompting always outperforms zero-shot prompting. In other words, having a conversation with an LLM and giving it examples and working with it like an expert will always give you better results than copy-paste prompts.)

Mistake 1 – Not understanding LLMs are the future of work 🔮

This ain’t a hot take.

Google — all in on LLMs.

Amazon — all in on LLMs.

Microsoft — all in on LLMs.

Meta — all in on LLMs.

Apple — (reportedly) all in on LLMs.

That’s 5 of the 6 largest companies in the U.S. (and the other is NVIDIA — the one literally powering the GenAI and LLM revolution.)

If you think AI is a fad or something that’s gonna come and go, we’ll try to be nice when we say this: you’re very wrong.

Do this:

Here’s a fun little trick we started talking about last year. Instead of using the term ‘Generative AI’ or ‘Large Language Model,’ start using the term Internet.

  • Would you use the internet to help you get a job? (Yes)

  • Would you use the Internet to do your work? (Yes)

  • Would you use the Internet to grow your business? (Yes)

The future of our personal and professional lives is based around LLM and Generative AI technology.

So the next time you’re thinking: “Should we use a LLM for this?” then just swap out and use the word internet.

Or just know the answer is almost always ‘Yes.’

⌚

Numbers to watch

$320 Million

Generative AI coding startup Magic lands $320M investment from Eric Schmidt, Atlassian and others.

(Or your fave LLM like Claude, Gemini, Copilot, etc)

Reply

or to participate.