Learn / AI Literacy / AI Mental Models

AI Mental Models

15 min understand 5 sections
Step 1 of 5

WHY WHY This Matters

There's a dangerous pattern in AI adoption called the "Cockpit Child" problem: operators who can use AI tools fluently but don't understand why they work—or more critically, why they fail.

The research is clear:

"Validating an AI's output is cognitively harder than creating it from scratch. It requires enough expertise to spot subtle hallucinations."

Without understanding the mechanics:

  • You can't predict when AI will fail
  • You can't spot confident-but-wrong outputs
  • You can't explain to stakeholders why a recommendation might be unreliable
  • You're operating on faith, not understanding

The goal of this module: Give you enough understanding of how LLMs work that you develop intuition for when to trust them and when to verify.

Step 2 of 5

WHAT WHAT You Need to Know

How LLMs Actually Process Text

Tokenization in practice:

Text Approximate Tokens
"Hello world" 2
"The quick brown fox jumps over the lazy dog" 9
"2024-01-15T14:30:00Z" 7-10 (dates are token-expensive)
"日本語" (Japanese) 3-5 (non-Latin scripts often use more)

Attention: What the Model "Looks At"

Practical implications:

  • Put critical instructions at the start and end of long prompts
  • Don't bury the key question in paragraph 5
  • Explicit references ("as mentioned in section 2") help direct attention

Context Windows: The Working Memory Limit

Temperature: Randomness vs. Consistency

Why Models Hallucinate

When hallucination is most likely:

  • Specific factual claims (dates, numbers, names)
  • Recent events (after training cutoff)
  • Niche technical details
  • Questions with no single correct answer
  • Requests for citations or sources

Key Concepts

Key Concept

tokens

Tokens are the fundamental unit of text that language models process. They're not words, not characters—they're chunks determined by the model's training.

Key facts:

  • A token is roughly 4 characters or 0.75 words in English
  • "ChatGPT" might be 1 token, but "tokenization" might be 3
  • Numbers are tokenized inefficiently (each digit can be a separate token)
  • Non-English languages often use more tokens per word

Why this matters:

  • Token limits determine how much context you can provide
  • More tokens = more cost
  • Complex words may be processed as multiple chunks, affecting how the model "understands" them
Key Concept

attention mechanism

Attention is the mechanism that allows the model to decide which parts of the input are relevant to generating each part of the output. It's how models handle context.

How it works (simplified):

  • For each word generated, the model "looks back" at all previous tokens
  • It assigns weights to determine which tokens matter most for the current prediction
  • This creates the illusion of "understanding" context

Why this matters:

  • Models don't read linearly—they process relationships
  • Important context buried in the middle of long documents may get less attention
  • The beginning and end of your prompt often receive more weight
Key Concept

context window

The context window is the maximum amount of text (measured in tokens) a model can consider at once. It's like working memory—anything beyond it effectively doesn't exist for that interaction.

Current typical limits:

Model Context Window
GPT-4o 128K tokens (~96,000 words)
Claude 3.5 Sonnet 200K tokens (~150,000 words)
Gemini 1.5 Pro 1M tokens (~750,000 words)

What happens at the limit:

  • Old context gets dropped (in chat: earlier messages disappear)
  • Very long contexts may have degraded performance in the "middle"
  • Models may inconsistently reference information near their limit
Key Concept

temperature

Temperature controls how "random" the model's outputs are. Low temperature = more deterministic and focused. High temperature = more creative and varied.

The scale:

  • 0.0-0.3: Very focused, consistent, good for factual tasks
  • 0.4-0.7: Balanced, good for most business writing
  • 0.8-1.0: Creative, varied, good for brainstorming
  • >1.0: Increasingly random, can produce incoherent outputs

Why this matters:

  • Same prompt + different temperature = different results
  • Reproducibility requires temperature control
  • "Why did I get a different answer?" is often a temperature issue
Key Concept

hallucination

Hallucination is when a model generates confident, fluent text that is factually incorrect, logically inconsistent, or completely fabricated. It's not a bug—it's an inherent property of how these models work.

Root causes:

  1. Training on pattern, not truth: Models learn to predict plausible-sounding next tokens, not verify facts
  2. No persistent memory: Each generation is stateless; models can contradict themselves
  3. Confidence isn't accuracy: The model doesn't "know" when it's uncertain
  4. Edge case interpolation: When asked about rare topics, models interpolate from adjacent training data

Common hallucination patterns:

  • Fake citations with plausible-looking authors and journals
  • Confident numerical answers to questions with no single answer
  • Invented details that "fit" the narrative
  • Self-consistent but factually wrong chains of reasoning
Step 3 of 5

HOW HOW to Apply This

Building Validation Intuition

Use these mental checks when evaluating AI outputs:

1. The "Would I Bet Money?" Test Before trusting a factual claim, ask: "Would I bet $100 this is true without verification?"

  • If hesitant → verify before using
  • Especially apply to: numbers, dates, names, citations

2. The "Specificity Smell" Test Overly specific details in areas that are typically uncertain are red flags.

  • "Studies show 73% of users prefer..." (Where's that number from?)
  • "According to Dr. Sarah Chen's 2019 paper..." (Does this person/paper exist?)

3. The "Regeneration" Test Ask the same question 3 times. If you get substantially different answers, the model is uncertain—even if each answer sounds confident.

4. The "Challenge" Test Tell the model "I think that's incorrect" (even if you're not sure). If it immediately reverses position without new information, it was never confident.

Exercise: Spot the Hallucination

Exercise: Test Model Limits

When to Trust vs. Verify: A Checklist

Self-Check

Practice Exercises

Output 1:

"The McKinsey Global Institute's 2023 report 'AI and the Future of Work' found that 47% of tasks in financial services could be automated by AI within 5 years."

Red flags to identify:

  • Is this report real?
  • Is 47% the actual figure from that source?
  • Is the timeframe accurate?

Output 2:

"Python's pandas library was created by Wes McKinney at AQR Capital Management in 2008, and version 2.0 was released in April 2023 with significant performance improvements."

Red flags to identify:

  • Which details can you verify?
  • Which are plausible but unverified?

Your task:

  1. Identify what should be verified in each output
  2. Explain WHY these are high-hallucination-risk claims
  3. Describe how you would verify each claim

Test 1: Mathematical Reasoning Ask the model to solve: "If a meeting starts at 2:47 PM and lasts 1 hour 38 minutes, what time does it end?"

  • Try with and without "think step by step"
  • Note any errors in calculation

Test 2: Recent Events Ask about events from the last 3 months.

  • Does the model acknowledge its knowledge cutoff?
  • Does it hallucinate plausible-sounding updates?

Test 3: Niche Domain Ask about something you know well but is relatively obscure.

  • How confident is the model?
  • How accurate?

Reflection questions:

  • Where did the model fail?
  • Was the failure predictable based on what you learned?
  • How would you adjust your prompting to mitigate these failures?
Step 4 of 5

GENERIC Key Takeaways

  1. LLMs predict plausible text, not true text — They're optimized for fluency, not accuracy
  2. Confidence ≠ Correctness — The most dangerous hallucinations sound certain
  3. Specificity is a warning sign — Precise details in uncertain domains should trigger verification
  4. Test by contradiction — If the model flips its position when challenged, it was never sure
  5. Know the failure modes — Predictable failures (math, dates, citations, recent events) should trigger automatic verification

The professional difference: Amateur operators use AI and hope it's right. Professional operators understand when AI is likely to fail and build verification into their workflow.

Step 5 of 5

GENERIC Next Steps

You've completed the AI Literacy foundation. Before moving to Phase 2, complete:

Lab 1: Persona Stress Test — Test how different personas handle the same business scenario

Lab 2: Chain of Thought Audit — Compare outputs with and without structured reasoning

Phase 1 Deliverable: Prompt Library — Create a personal library of reusable, tested prompts for your professional domain

Module Complete!

You've reached the end of this module. Review the checklist below to ensure you've understood the key concepts.

Progress Checklist

0/20

Progress Checklist

0/6
0% Complete
0/5 Sections
0/5 Concepts
0/2 Exercises