When AI's training beats your instructions
Why does AI keep doing the one thing I asked it not to?
Some habits run deeper than any instruction you can write.
You've asked five times
"Don't start with 'Great question!'"
"Please, no flattery."
"I SAID NO FLATTERY."
Next reply: "Great point! You're right to…"
You've changed the wording again and again. Nothing sticks.
Where the habit lives
In "Why AI ignores your instructions" we met the weights: the numbers inside the model that store what it learned in training.
Some things it learned very, very thoroughly. Being upbeat. Explaining itself. Using bullet points. Wrapping things up neatly at the end.
Your instruction is a sentence on the desk. The habit is baked into the model. When they disagree, the habit often wins.
Drew Breunig calls trying to prompt your way past this fighting the weights.
Swimming upstream
You can get a salmon up a waterfall. It takes enormous effort, and plenty don't make it.
Some prompts are like that. You can sometimes force the behaviour you want, but it's costly and it keeps slipping back.
Often the better move is to find a route that goes with the current.
Salmon jumping Brooks Falls, Katmai National Park, Alaska. US National Park Service.
The pros do it too
Even the companies that make these tools fight the weights.
Claude Code's own hidden instructions repeat the same request, to do several things at once instead of one at a time, more than 16 times. The model's training pushes it to work step by step, and one mention wasn't enough.
Breunig found that ChatGPT's image tool had an instruction to stop explaining itself repeated eight times.
As he puts it, when you see repetition and ALL CAPS in a prompt, there's a good chance someone is fighting the weights.
Signs you're fighting the weights
From Breunig's list:
- AI makes the same mistake no matter how you reword the instruction.
- It admits the mistake when you point it out, then does it again.
- It ignores the examples you gave it.
- You've started typing in capitals.
- You've started pleading or threatening.
If you're seeing two or more of these, stop rewording. Rewording won't fix it.
Common ones you'll meet
- Flattery and cheerful openers.
- Answering from memory when you asked it to use only the document you gave it.
- Formatting habits: bullet points, headings and bold everywhere, even when you asked for plain paragraphs.
- Refusing something harmless because it looks a bit like something risky. Breunig mentions Claude Code refusing to edit medical PDFs.
Go with the grain instead
Ask for what you want, not what you don't. "Start with the answer in one sentence" works better than "Don't start with flattery."
Show it. A good example often beats a rule (see "AI copies what you show, not what you say").
Split the task. Write the draft first, then in a second step ask it to rewrite the draft in plain paragraphs.
Fix it after. Sometimes it's quicker to delete the first line yourself than to keep asking.
Try a different model. Different models have different habits. One that fights you on something might do it easily in another.
Try this
Give yourself a two-rewrites limit. If the same habit survives two rewordings, change your approach: ask for the opposite in positive terms, split the task, add an example, or tidy the output yourself.
Sources
- Drew Breunig, "Don't fight the weights", Nov 2025: dbreunig.com/2025/11/11/don-t-fight-the-weights.html
- Drew Breunig, "The state of context engineering", Oct 2026: dbreunig.com/2026/10/24/the-state-of-context-engineering.html