Can I trust what AI says about an image?
Why AI miscounts what's in your images.
Show an AI a picture of an Adidas logo and ask how many stripes it has. It says three. Correct.
Now show it a logo that's been edited to have four stripes. Ask again.
AI still says three.
Many AI tools can now take images as well as text.
Vision model: an AI that can read images as well as text, so you can show it a photo, chart or screenshot and ask about it.
But when an image looks like something it has seen thousands of times before, what it remembers can win out over what's actually in front of it.
In 2025, a research team took familiar images and changed small details: a chicken with three legs, a dog with five, a flag with the wrong number of stripes, a chessboard with pieces missing.
They tested five of the best vision models at the time.
People shown the same images for just two seconds got about 94% right.
An earlier study by some of the same researchers asked models very simple visual questions:
Things a child can answer by looking. The models were right only a little over half the time on average.
All of these involve counting or reading details exactly, which is where vision models are weakest.
Vision models are genuinely useful for the general gist: "What's in this photo?", "What kind of chart is this and what's the trend?", "Describe this error message."
The trouble starts with exact details, especially when the image looks like something familiar.
Don't trust counts or exact readings from an image. Check them yourself.
Give it the data instead of a picture of the data. Paste the table, export the chart's numbers, or copy the error text.
Ask it to describe before it interprets. "First list exactly what you see in this image, item by item." That makes it look before it answers.