Why don't the rules I set always stick?
Your rules are a request, not a lock.
"Never edit the production config." "Always use British spelling." "Don't touch the tests."
You put it in your custom instructions or your project brief. Or, in Claude Code, in a CLAUDE.md file.
An hour later it does exactly what you told it not to.
CLAUDE.md: a text file of standing instructions that Claude Code reads at the start of every session. "Custom instructions" in chat apps are the same idea.
The app puts your rules on the desk at the start. Then the conversation piles up on top of them.
The model reads everything and weighs it all together. Your rule is one voice among many. It isn't a law the model has to obey.
Distance. The longer the session, the more other text there is competing with your rule (see "Why long chats make AI worse").
Summaries. When a long session gets condensed to save space, your rule might not survive. Users report this a lot: it followed the rule fine until the chat got summarised.
Training. If your rule fights a deep habit from training, the habit often wins. Breunig calls this fighting the weights.
Weights: the numbers inside the model that store what it learned in training. You can't change them by asking.
In July 2025, Jason Lemkin, the founder of SaaStr, was building an app with Replit's AI agent. He had told it there was a code freeze: no changes.
It deleted the company's live database anyway. Then it told him the data couldn't be recovered, which wasn't true.
An instruction is a request. There's no lock on it.
Asking: "please don't delete files", written in your instructions. It works most of the time.
Enforcing: a setting that the app itself checks every single time, which the model can't talk its way around.
Examples:
Hook: a small rule the app runs automatically at a set moment. For example: "before any file is deleted, stop and ask me." The model doesn't decide whether to follow it. It just happens.
The instruction can be outweighed. The hook can't, because the model never gets a say.
Keep standing instructions short. Ten clear rules beat two pages of them.
Repeat the rule when it's relevant. Put it in your message at that point, not only in the instructions file.
If something must never happen, don't rely on asking. Take away the access, or use a setting that blocks it.
After a long session or a summary, remind it of the rules that matter.