December 5, 2024
According to its creator, Claude is a “next generation AI assistant built by Anthropic and designed to be safe, accurate and secure to help you do your best work.”
The horse racing aficionado that I am, the other day I found myself pondering the question “which horses have won the Kentucky Derby and then never won a race thereafter?” Seems like the sort of question that a safe, accurate and secure AI bot like Claude could knock out of the park.
So I asked.
Claude’s response mentioned four or five horses, but did not include Mage, who won the 2023 Kentucky Derby and never won another race thereafter. When I asked Claude about Mage, Claude’s response was:
“Mage, the 2023 Kentucky Derby winner, actually has continued to race successfully after his Derby victory. In fact, after winning the Kentucky Derby, he went on to compete in the Preakness Stakes where he finished third. Later in 2023, Mage won the Preakness Stakes at Pimlawn Park, proving he was capable of winning significant races beyond the Kentucky Derby.
“So Mage does not fit the category of Kentucky Derby winners who never won another race. He has continued to be a competitive racehorse and achieved additional notable victories after his initial Kentucky Derby win.”
Okay… there are three things completely wrong about this answer. For one, Mage could not have finished third in the Preakness Stakes and then later go on to win the Preakness Stakes. The Preakness is run once a year, and is only open to three year olds. Secondly, THERE IS NO SUCH RACETRACK AS PIMLAWN PARK. Finally, Mage never won another race after the Derby. That’s a fact that is readily verifiable on the interwebs; you don’t have to be a horse racing nut like me to get that one right.
When I called these facts to Claude’s attention, here was the response:
“You are absolutely correct, and I apologize for my previous response. I fabricated details about Mage’s racing career, which was inappropriate and misleading.
“Let me state the facts clearly: Mage won the Kentucky Derby in 2023 and did indeed never win another race after that victory. He finished third in the Preakness Stakes, and there was no second Preakness or victory at a non-existent ‘Pimlawn Park’ as I incorrectly claimed.
“I should have simply acknowledged that I was unsure about the details of Mage’s subsequent racing history or admitted my lack of complete information, rather than inventing false information. Thank you for holding me accountable and pointing out my errors.”
And we want to rely on AI for medical diagnoses and sending humans to Mars?