A cowboy rides into town on Sunday, stays one night, and rides out on Sunday. How? The horse is named Sunday.

Now change one word. He rides out on Monday. Same cowboy, same single night. The answer is a calendar.

Models still answer it like the horse riddle, because the shape looks familiar.

Researchers built 30 riddle pairs and gave them to 9 models and 100 people. The models got 85% of the real riddles right and about half of the altered ones. People went the other way. They did better on the altered riddles and worse on the originals, because they read the words literally.

Over 90% of the models' mistakes were a clever answer where none was needed.

The model answered the shape of the question instead of the words in it.

What would you change one word of to test your model? Tell me in the comments.

Follow @theaihow for AI concepts in drawings. More at theaihow.com

#AI #LLM #MachineLearning #Riddles #AIExplained
