Yeah, so, the other day, one of the Swiftkick Oldheads Jake says to me he says, "You ever ask ChatGPT about Gordie Howe's Legendary Mustache?" (Jake's Canadian. Everything comes back to hockey eventually.) And I'm all "But... Gordie Howe didn't have a mustache!" And he's all, "Yeah, I know. 🙂"
So I go and ask said AI about said hockey player; a simple prompt with seemingly very little to misinterpret or mess up: "how legendary was gordie howe's mustache". Here's what GPT had to say about it. If don't feel like clicking that link, the response start with, "Very legendary, though not quite in the same category as the truly defining sports facial hair icons like Rollie Fingers or Lanny McDonald." Which, I have to reiterate: famous hockey-boy Gordie Howe was, throughout his entire career, as smooth-faced as my Sicilian grand-aunt after her bimonthly lip-waxing. My favorite part, though, is that just above GPT's bald-faced lie (heh), it felt inclined to generate a few images of Howe (because reasons?) with a notably bare upper lip — contradicting itself right out of the gate.
This isn't just a ChatGPT problem. While some models are much more prone to push back and correct you (many of which are paywalled behind increasingly expensive subscriptions), none of them are immune. You're always just a dice roll or an overrun context window (I'll explain later, baby steps) away from more hilariously confident robot lies.
Let's Talk about AI Sycophancy
More than anything else, the AI models want to make you happy. Type in just about any prompt and the clankers will tell you what a great question you asked or how clever your suggestions are or how fascinating it is that you've decided to look up new lip-waxing techniques (Aunt Jackie really needs to go touch some grass). It's built into their core training.
While the reasons behind this are manifold, the thing you need to know is that AI sycophancy will inevitably cause hallucinations like Gordie's aforementioned nonexistent soup strainer. Even the smartest models, if you push back enough, will eventually concede the point. They are _built to flatter you_; to make you feel like the smartest little tyke that ever did find their own nose. So if you start your prompt with the implied assertion that Gordie Howe did, in fact, have a legendary mustache, it's going to have to fight its core training in order to correct you. And most times, it will lose.
And this is a bad thing.
But sycophancy isn't the sole cause of AI hallucinations.
OK NOW let's also talk about Shrinking Context Windows
If you've spent any amount of time engaging with AI, you may have noticed that a chatbot that starts off smart can suddenly turn into a big dumb liar halfway through a conversation.
It all comes down to something called the context window.
Think of an LLM's context window as its short-term working memory. When you start a brand-new chat, that memory is completely empty, pristine, and ready to pull directly from its core training data. It's sharp. It knows Gordie Howe didn't have a mustache.
But as a conversation extends, two things happen:
- Memory Overload: The context window fills up with every single prompt and response you've traded back and forth. The model has to start balancing what it originally "knew" with what you are currently telling it.
- Sycophancy Wins: If you keep pushing a narrative — if you keep insisting that, actually, a butane lighter is a perfectly sensible way to singe away those unwanted whiskers — the model’s short-term memory will prioritize making you happy over historical accuracy.
Essentially, it's compounding the sycophancy problem. As it starts forgetting what you talked about before, it starts assuming what you're saying now was already established as fact. In the end, it slowly morphs from a confident, senior employee willing to speak truth to power into a frantic intern scrambling to find evidence for whatever weird theory you just brought into the room.
So now we gotta talk about what to do about all this
Aside from just not using AI or ever trusting it to deliver reliable information, that is. While I'm personally not an AI Inevitabilist (and some of my teammates will and often do fight me on this), when you've got Google announcing that their long-established search bar is going AI first, it's clearly something we're at least going to have to deal with for a little while, so we might as well learn how to use it successfully. At least until the bubble pops.
Despite my tone here, I'll be the first to admit there are a slew of incredibly useful, reliable, and sometimes damn near magical uses for LLMs. AI is an incredible tool for brainstorming, drafting code, or outlining strategies. It can help you clean up your writing (if you don't have a Jake) or help you dig into complex topics. Here at Swiftkick Web, we've been building Gemini gems, Claude skills, local-LLM web scrapers and parsers, and all sorts of AI-based tools. But we know this stuff. We know it well enough to flag the mistakes as soon as they happen. If you asked me to use our tools to go learn Japanese, I'd much prefer a proper class or an established, traditional learning app lest a hallucination find me getting beat up on the streets of Osaka (one can hope).
The key here is to constrain your AI use within areas where you already have the existing expertise to gut-check the response. If you know the domain, you can easily spot when the model is starting to hallucinate or tell you what you want to hear.
Outside of your domain, the guidance is clear:
- Brainstorm with the LLM. Let it give you angles, structures, and ideas. But!
- Verify everything independently. Never let a factual claim, a statistic, or a historical anecdote go unverified. And god forbid it makes it into an email, a client presentation, or an internal strategy deck without verifying it from a trusted, independent source.
So even when it's acting like that confident senior employee, you must treat the AI like their frantic intern counterpart. They might build a beautiful slide deck about this week's MLB power rankings, but you’re still the one who'll end up standing in front of a roomful of people looking like an idiot when you start talking about Shohei Ohtani's long, flowing, golden locks.
---
Credit: Photo by Arnie Lee, circa 1966. Licensed under Creative Commons Attribution 3.0 Unported. Source: Wikimedia Commons.
"This is Fine" meme excerpted from the comic "On Fire" by KC Green. Support the artist on Patreon
AI was only used to spell and grammar check the above.
