• Everyday AI
  • Posts
  • A Complete Guide to Tokens Inside of ChatGPT

A Complete Guide to Tokens Inside of ChatGPT

Grok 2 is released, ChatGPT-4o takes back #1 spot, an explanation of tokens inside ChatGPT and more!

Author

Everyday AI
August 14, 2024

Outsmart The Future

Today in Everyday AI
6 minute read

🎙 Daily Podcast Episode: Every LLM uses tokens. Wondering what they are? We explain how they work to improve your outputs. Give it a listen.

🕵️‍♂️ Fresh Finds: Claude gets prompt caching, MIT researchers release AI risks list and SpaceX is using an NVIDIA GPU. Read on for Fresh Finds.

🗞 Byte Sized Daily AI News: Grok 2 is released, ChatGPT-4o reclaims #1 spot and Huawei looks to challenge NVIDIA. For that and more, read on for Byte Sized News.

🚀 AI In 5: xAI has just released Grok 2. Is it any better than Grok 1? We dive in to find out. See it here

🧠 Learn & Leveraging AI: We’re breaking down what tokens are, how to understand them and why they’re important for LLMs. Keep reading for that!

↩️ Don’t miss out: Did you miss our last newsletter? We talked about Google's big AI announcements, Grok 2 leaks, Gemini 1.5 Flash getting cheaper. Check it here!

A Complete Guide to Tokens Inside of ChatGPT 🪙

Wait.... tokens?

When using a large language model like ChatGPT, tokens really matter.

But hardly anyone understands them.

And NOT knowing how tokens work is causing your ChatGPT output to stink.

Join the conversation and ask Jordan questions on tokens here.

Also on the pod today:

• Explanation of Tokenization Process 🧮
• Comparison of Different AI Models 🤔
• Memory in AI Models 🧠

It’ll be worth your 47 minutes:

Listen on our site:

Click to listen

Subscribe and listen on your favorite podcast platform

Listen on:

Here’s our favorite AI finds from across the web:

New AI Tool Spotlight – Gigabrain is automated Reddit research, Volamail provides open-source AI-powered emails and ToonTalk is AI friends for children.

Trending in AI – MIT researchers have released a repository of AI risks.

LLMs - Anthropic has released Prompt caching in Claude, which lets you fine-tune model responses with more instructive prompts.

AI in Science - SpaceX is launching the first space-hardened NVIDIA AI GPU on its upcoming mission.

AI in Healthcare – National Institutes of Health (NIH) researchers have found that AI struggles to identify genetic conditions from patient-written descriptions.

1. Elon Musk’s xAI Launches Grok 2 🤖

Elon Musk's xAI has officially launched Grok 2, a beta version of its AI assistant that introduces an image generation tool with fewer restrictions than competitors like DALL-E and Gemini.

Available to X users with ‘Premium’ subscriptions, Grok 2 can create striking images based on user prompts, including controversial depictions of political figures such as Donald Trump and Kamala Harris. Musk has positioned Grok 2 as a more liberated alternative to existing AI models, aiming for a blend of wit and a rebellious edge.

2. OpenAI ChatGPT-4o Retakes #1 Spot with Updates 🏆

In a significant development, OpenAI's latest ChatGPT-4o has achieved the number one position in the chatbot arena with an impressive score of 1314, outpacing Google’s Gemini-1.5-Pro-Exp. Following a week of thorough testing and over 11,000 community votes, this new model demonstrates remarkable advancements in coding, exceeding its predecessor by more than 30 points.

With strong performances across various categories, including Math, Hard Prompts, and Instruction-Following, ChatGPT-4o showcases its capabilities as a leading AI tool.

3. Huawei Takes Aim at NVIDIA with New AI Chip 🥊

Huawei is gearing up to compete with NVIDIA by introducing its Ascend 910C chip, claiming it matches the performance of NVIDIA’s H100. Despite facing U.S. sanctions that have limited its technological advancements, Huawei is in discussions with major companies like ByteDance and Baidu for potential shipments as early as October.

However, production delays and the threat of additional U.S. restrictions could complicate Huawei’s ambitions in the AI market

4. AI Art Controversy Heats Up: Artists Take a Stand 🧑‍⚖️

The legal drama surrounding generative AI just got juicier as Judge William Orrick allowed artists to add new claims against Stability AI, the brains behind the popular Stable Diffusion image generator. They’re crying foul over copyright violations, especially after it was revealed that Midjourney included 4,700 artists in its “Midjourney Style List” without permission, implying a false endorsement.

Kelly McKernan, one of the artists involved, called the ruling “a HUGE win,” paving the way for more transparency from these companies.

5. SAG-AFTRA Teams Up with Narrativ for AI Voice Replicas 🗣

SAG-AFTRA just dropped a bombshell with a groundbreaking deal alongside AI startup Narrativ, creating a platform for performers to license their digital voice replicas for ads.

This arrangement means that the union's 160,000 members can negotiate their fees on a project-by-project basis, ensuring they get paid fairly without dipping below SAG-AFTRA's minimum rates.

Grok-2 Review: Better Than Grok 1 or Another Flop from Twitter?

xAI has just released Grok 2, a new and updated model from Elon Musk’s company.

Is this new model better than Grok 1?

We’re giving you a live review and going over the pros and cons.

🦾How You Can Leverage:

Tokens?

What the heck are they and why do they matter?

TBH, not understanding tokens while using a LLM is kinda like not understanding a gas tank when going for a road trip.

Make a mistake, and things will go south quick. And after teaching thousands of business pros the ins and outs of ChatGPT, the token context window is one of the biggest mistakes people make.

Understand tokens, and your ChatGPT outputs (and your biz or department!) soar.

As always — we got your back.  

What they are. 

And why they matter. 

Ready to be the smartest in your company when it comes to how LLMs work under the hood?

You’re welcome. Here’s what ya need to know. 

Let’s get it. 

1 – Tokens: What they are 🤔️

Tokens are like the secret sauce of large language models (LLMs).

Think of 'em as the building blocks that LLMs use to understand and generate text.

LLMs don't actually "read" or “understand” words like we do.

Nope, they break everything down into smaller chunks called tokens. A token could be a full word, part of a word, or even just a single character.

For example, "strawberry" might get chopped up into "straw" and "ber" and “ry.” That’s why you can’t ask a LLM how many Rs are in the word, BTW.  

And get this – even stuff like capitalization, spaces, and punctuation can change how words get tokenized.

It's a whole new level of language hacking!

Try this: 

Hit up OpenAI's tokenizer tool. Type in some words and watch the token magic happen. It's like seeing the Matrix code, but for language!

2 – Tokens: why they matter 🤷

Understanding tokens is your secret weapon for leveling up your AI game.

First off, tokens are how LLMs measure their memory.

Yeah, you heard that right – their MEMORY.

Every model has a "context window" (fancy talk for how much it can remember at once), and that's measured in tokens.

Tokens also affect how well the AI understands context and nuance. The way words get broken down into tokens can change their meaning.

So if you're not clear in your prompts, you might end up with some seriously wonky results.

Try this:

Next time you're chatting with an AI, pay attention to how long your conversation gets. Try asking it to recall something from way earlier.

If it starts acting dumb, congrats – you've just witnessed token amnesia in action!

Buuuut….. we’ll fix that with number 3. 

3 – Tokens: working inside the context window 🪟

Alright, time to get tactical and crush it with context windows!

Remember how we said each model has a memory limit? 

Well, for ChatGPT, that's about 32,000 tokens (roughly 26,000 words).

That might sound like a lot, but trust me, it can vanish faster than free pizza at a tech meetup.

(We use this ChatGPT tokenizer inside of Chrome to keep track of token usage and memory limits.) 

Here's the hack: when you're working on something big, you gotta play token Tetris.

Keep the important stuff fresh in the AI's mind by summarizing key points or using memory recall techniques.

And heads up – different models have different context windows.

Claude's got a monster memory, while some others... let's just say they're working with goldfish brain capacity.

Try this:

Start a convo with ChatGPT, then dump a bunch of text in there (like, pages of it).

Keep chatting until you hit that 32,000 token mark.

Then ask it about stuff from the beginning. Watch it struggle like it's trying to remember what it had for breakfast last Tuesday!

And most importantly….. you GOTTA take our new and improved ChatGPT course — PPP.

It’s live, 100% free and we update the content every week so you understand how to get the most outta ChatGPT.

To get access, just reply PPP to this email and we’ll send ya the registration link! 

⌚

Numbers to watch

$75 Million

EliseAI has raised $75M for chatbots that help property managers deal with renters.

(Or your fave LLM like Claude, Gemini, Copilot, etc)

Now This …

Let us know your thoughts!

Vote to see live results

Do you attend our livestreams?

Every weekday, we bring you fresh AI insights, exclusive interviews, and breaking news with our Everyday AI livestream.

Login or Subscribe to participate in polls.

Reply

or to participate.