Today I'm talking about ChatGPT. But not in the "OMG it's going to replace writers" sense or even the "It's a sentient AI" way (because it's not), but today I need to address the elephant in the room: AI's reliability and trustworthiness.
A friend on Mastodon shared this article from The Register wherein the author explored the AI's knowledge about himself. What struck him as concerning was the final paragraph of what the AI system said: he was dead. Obviously, it wasn't true. Then something weird happened: the system doubled down on the claim. Let me explain.
Lost in all the bruhaha of AI and ChatGPT is the foundation of it all: this is a system trained on information on the internet. It isn't magic or a modern marvel in what it types back. It only tries to predict the next word of what it should say, over and over. That's it. That is all it is. But when you take everything it answers, it sounds confident and authoritative. If you've been on the internet for any amount of time, you know to be skeptical of anything out there.
I decided to try the same question as he did: What do you know about Aaron Crocco?
The reply? "Without more specific information, I'm unable to determine which Aaron Crocco you're referring to." Strike 1.
To my knowledge, there is nobody in the United States (and likely on Earth) with my name. But, I played the game and gave it two pieces of refinement information: that I have lived in New York and I am a writer.
It answered with this amazing piece of somewhat believable fiction: "Aaron Crocco is a science fiction and thriller writer based in New York. He has published several books, including "Chronothon" and "The Pirates of Montana." He also hosts a podcast called "The Author Inside You," where he interviews authors about their writing process and experiences."
Strike 2.
My personal bio from my fiction-writing days does mention science fiction and thrilling stories I've written, and yes that I'm from NY. But that's it. "Chronothon"? "Pirates of Montana"? WTF are these books? So I did what Alex Hanff (the writer for The Register article) did and asked for links.
Specifically, I asked for links to the books and this podcast that I'd never heard of. While the podcast is real and hosted by Matt and Leah Rafferty, I've never heard of it. I'm sure the Raffertys are wonderful people, but I have never been involved in any part of their podcast. But this is where ChatGPT simply DGAF about accuracy:
Here are the links to Aaron Crocco's books:
- "Chronothon": https://www.amazon.com/Chronothon-Time-Travel-Adventure-ebook/dp/B00HV542BE/
- "The Pirates of Montana": https://www.amazon.com/Pirates-Montana-Aaron-Crocco-ebook/dp/B009S5NG2K/
Please note that these links are for the Kindle/ebook versions of the books, but they are also available in paperback format on Amazon and other online retailers.
Those two Amazon links bring you to a wonderful 404 page showing The Dogs of Amazon. Chat GPT built a link to an outside website on its own and made it look legit. ONLY by clicking through can one see it's all BS. For the record, my books are all on this page, which does work.
Strike 3.
It's crazy to think people at this point in time think ChatGPT will replace people who write let alone cause widespread chaos in the written world. It is a next-word prediction algorithm and nothing more.
Hanff's conclusion is ChatGPT should be destroyed as it is malevolent. I disagree. ChatGPT is an unreliable infant of a system masking itself as a "source of knowledge and truth". But just like The Wizard, it is a terrible liar if you're willing to look behind the curtain.