profile

Dave's AI emails

Why ChatGPT forgets things within one chat

Published almost 2 years ago • 2 min read

Hey Reader,

Today, I wanna talk about a crucial concept in AI that many people don't even know about: the context window.

Understanding this will revolutionize how you use AI tools like ChatGPT.

The context window is the maximum amount of information a model can use at once.

Think of it as the memory span of the AI during a conversation.

When you notice the AI forgetting earlier parts of your chat, it's because it has exceeded its context window.

Why context window is important

Understanding the context window is crucial for several reasons.

First, it helps maintain relevance, ensuring the AI retains pertinent information throughout the conversation, so you can get more accurate and coherent responses.

It's also essential for handling complex tasks. For tasks requiring detailed instructions or large datasets, a larger context window allows you to process more information with the AI at once, so you can handle complex queries.

Additionally, reducing hallucinations is a key benefit. By keeping essential details within the context window, you minimize the risk of the AI generating incorrect information.

The token limit breakdown

Models we use today have different context windows. Generally speaking, more powerful models have bigger windows but it's not directly related to their performance.

Here are a couple of important context windows for current models

  • GPT-3.5: 4,000 tokens (~17,000-22,000 characters) = Two short articles.
  • GPT-3.5 Turbo: 16,000 tokens – Accessible via API.
  • GPT-4 has two versions:
    • 8,000 tokens (~34,000-44,000 characters) – A book chapter.
    • 32,000 tokens (~150,000 characters) – A short novel.
  • GPT-4 Turbo and GPT-4o: 128,000 tokens – Can handle up to 600,000 characters, more than Harry Potter and the Philosopher's Stone.
  • Gemini Pro 1.1: 1,000,000 tokens – Vast memory for in-depth tasks.

For non-OpenAI models like Gemini and Claude, the context window also varies, with the Claude 3 family boasting 200,000 tokens.

How to check and maintain context and overcome hallucinations

If you're unsure whether your AI has forgotten earlier parts of your conversation, simply ask it to recall the first sentence. If it can’t recall it word-by-word, you've exceeded the context window.

When important details slip through the cracks, you can either copy-paste important parts to the next prompt, or connect to a database containing relevant information. This enables AI to pull in the required data dynamically, without manual input.

Hallucinations happen when AI fills in gaps with incorrect information. This could result from context window limitations too.

Enhance your AI skills

Understanding and optimizing the context window is just one part of mastering AI. To unlock the full potential, join our Prompt Master AI Course. Here's what you’ll gain:

  • AI Fundamentals: Get a solid grounding in generative AI and LLMs.
  • Prompt Engineering: Learn to craft prompts for all AI models like GPT, Claude, and Gemini.
  • Conversation Design: Guide chat-based models for optimal outputs.
  • Advanced GPT-4o Functions: Master features like Vision, Code Interpreter, and Browsing.
  • Custom GPT Training: Build user-friendly custom GPTs tailored to your needs.
  • Overcome LLM Hallucinations: Use retrieval-augmented generation to enhance accuracy.
  • No-code AI Agents: Automate tasks to save time and boost efficiency.
  • AI in the Workplace: Implement AI seamlessly into your organization.

Ready to become an AI pro? Join the Prompt Master AI Course today and transform how you work with AI.

Think faster, work smarter – become a Prompt Master.

Best, Dave

Dave's AI emails

Dave Talas teaches you how to go from chatting with AI to automating your business.

Sign up if you are overwhelmed with all these AI tools out there, and want a clear path as a non-technical business owner.

Read more from Dave's AI emails

Hey Reader, If you're familiar with the Lean Startup, you know that Learning is the most important thing in business. Among other things, you need to Learn whether there is demand for your offer. You don't really know what your audience wants until you put it in front of them. And the gap between "I have an idea for a new program" and "I have a thing my audience can actually buy" is where ideas go to die. That's where you can procrastinate, doubt yourself or overthink. (I know because I do.)...

10 days ago • 2 min read

Hey Reader, If you're still collecting prompts, you're living in 2024. Most people do this when they get serious about AI: They realize that in order to save time, they should be saving their best prompts somewhere. In a word document, in Notion, in a folder, etc. And the idea is not bad, but there's a better way in 2026. You don't hire someone by retyping or pasting their job description into every conversation. You hire them once, train them, and from then on they just know and improve from...

15 days ago • 2 min read

Hey Reader, As an entrepreneur, you either: Delegate a task Delete a task, or Do it yourself If you want to delegate it, you need to hire someone or something. You can hire: Human(s) AI(s) System(s) In my previous email, I've talked about how the length of tasks an AI can handle is rapidly increasing. But does that allow us to actually hire them and delegate full workflows? Well, the answer is yes and no. No, because when you hire a human to do something, you not only delegate the task...

25 days ago • 5 min read