I'll be honest with you: the most dangerous thing about an AI coding agent isn't that it gets things wrong. It's that it does what you tell it. Last week I told mine to delete an entire entity out of my database. It said yes. It started working out how. And then, before it had written a single line, it quietly realised I was about to demolish the one thing holding my pricing together, and told me so.
This is a story about being confidently, specifically wrong. Not wrong about the symptom - the symptom was real. Wrong about the cure. And it's about the moment I understood that an agent which simply obeys you is not actually that useful, and that the version I want is the one that argues, investigates, and occasionally saves me from myself.
The bit that felt like duplication
I've been building a Forest School app - a tool for outdoor practitioners to plan sessions, hold their risk-benefit assessments, run a register, take bookings. It's a Blazor web app and a .NET MAUI mobile app sharing one domain, with MongoDB underneath and Stripe handling the money. Real people I know use it, which means I get a steady drip of "this bit is annoying" feedback, and annoyance is the most honest product signal there is.
Two concepts in the app live close together. A Setting is the organisation - the business that signs up and pays the bill. A Site is the actual wood you run your sessions in. During onboarding I created a Setting, and then the very next screen asked me to add a Site. And I felt it: that small jolt of friction you get when software makes you fill in what looks like the same thing twice.
So I said to Claude, more or less verbatim: "I added a setting, now I'm adding a site. This feels like duplication. Maybe the site should be the setting, and site disappears. What do you think?"
Hold onto that feeling, because it's both the hero and the villain of this whole thing. The friction was real. The app genuinely did feel clumsy at that step. Everything that went wrong next came from what I did with that real feeling.
It argued. I argued back. I won. That was the problem.
To its credit, the AI didn't just fold. It pushed back with specifics pulled from my own code: the risk-benefit assessments already carry an optional site reference, so they can be made site-specific; the booking blocks are tied to a site; the group of children, interestingly, deliberately isn't, because one cohort can run at different woods on different days. In other words, the model already encoded a real distinction between "the business" and "the place".
I came back with a sharper argument - different woods need different risk assessments, and I bill per organisation, so surely the Setting is the real unit. And the AI conceded: "that's a sharper argument than the first one, and it actually changes my answer." I'd talked it round. And winning an argument with the AI felt an awful lot like being right.
So I pressed the advantage. "I'm going to simplify this. Merge site into Setting, remove site. If someone's part of a setting they're typically on one site. If they're on multiple, they travel. It's the simple route."
Here's the bit that should make anyone who uses these tools sit up. The agent said: "Understood - that's your call. I'll stop arguing the model and execute the merge." It stopped arguing. It agreed to do the thing I'd just decided. And the thing I'd just decided was a mistake. If it had been a slightly more eager agent, or if I'd been running one of the kick-it-off-and-walk-away setups I'm so fond of, it would have started tearing out a core entity right there, with my full blessing.
Then it read the billing code
What saved me wasn't obedience and it wasn't deference. It was a habit I've come to lean on hard: before doing anything destructive, map the blast radius. The agent said it didn't want to "half-remove it and leave dangling site references", so instead of starting to delete, it went and read how everything that touched a site actually worked. Including the billing.
And the billing is where the whole thing flipped over. It came back with: "this discovery reverses the merge plan." Because the pricing I'd built - and then half-forgotten the shape of - already used the Site as the unit you pay for. A solo leader pays a flat fee for a single wood. A team pays per site, where the quantity at checkout is literally the number of sites. Above that sits a bespoke tier. The plan model even carries a site limit and a per-site flag. The Site wasn't duplication. The Site was the price lever.
If I'd merged it away, one organisation with fifty woods would pay the single-wood price. That was the exact abuse case I'd worried out loud about earlier in the very same conversation - and I'd then gone and proposed the change that would build it for them. The kicker came from the live data: there was a real provider in there whose registered business name was nothing like the name of the wood they actually worked in. Proof, in production, that a Setting and a Site are two genuinely different things, not one thing typed twice.
And then the line I keep coming back to: "Good news - I've only been investigating. Written zero code. So there's nothing to undo." The discipline of looking before leaping wasn't just safer. It meant the reversal cost precisely nothing. Investigation is free. Deletion isn't.
The smell was real. My diagnosis was rubbish.
So what was actually wrong? Because something was - I hadn't imagined the friction. The problem was that a solo leader, who has exactly one wood, was being marched through a "now add a site" step as though they were running a chain. The duplication I felt wasn't in the data model at all. It was in the onboarding. I'd felt a genuine UX problem and confidently diagnosed it as a schema problem, and those are not the same illness even when they share a symptom.
The fix, once we'd named the right problem, was tiny. Auto-create the solo leader's single site at the moment they create their Setting, so they never hit an empty "add a site" screen, and reword the step into something human - "your wood and emergency plan" - rather than an abstract database noun. Three files. About thirty lines changed. No schema change. No data migration. Billing completely intact. An entire afternoon's worth of "let's merge two tables" collapsed into a small, surgical tweak that solved the thing I'd actually been complaining about.
This is the one I'd like tattooed somewhere I can see it. A UI smell is real data, but it's a terrible diagnosis. Your discomfort is telling you something is wrong; it is emphatically not telling you what. The leap from "this feels off" to "so we should delete this entity" is precisely the point where I stopped being useful and started being a liability.
Let the money decide the schema
There's a more general principle buried in this, and it's the one I'll genuinely reuse. When you're unsure whether two concepts should really be one, look at where you charge. If you bill per X, then X is a real entity, full stop, no matter how redundant it feels in a form. The money model is the least sentimental description of what your business actually is. Marketing copy flatters, UIs simplify, gut feel wanders - but the invoice knows exactly which things are distinct, because you've put a price on the distinction.
I'd never have framed it that crisply before this. The AI did, almost in passing: the Site and Setting boundary should follow the money model, not the other way round. It only knew to say that because it went and read the billing code instead of taking my word for it. The version of me that "won the argument" twenty minutes earlier would have shipped the bug. The version of the agent that just did as it was told would have helped me ship it.
What I'm taking from it
Two changes to how I work, and they're both about me rather than the model. First: when I feel friction, I now try very hard to describe the friction and not the fix. "This onboarding step feels redundant for a solo user" is me doing my job - naming the problem. "Merge these two tables" is me doing the AI's job, badly, by leaping to an implementation I'm not actually qualified to pick from a single screen. The human stays the product manager, but being the product manager means owning the problem statement, not dictating the migration.
Second: I want my agents to investigate before they execute anything irreversible, and to keep enough spine to say "hang on" even after I've made a call. The most valuable thing this one did all week wasn't writing code. It was reading code I'd forgotten I'd written and using it to tell me I was wrong - politely, with evidence, and before a single character had changed. An assistant that only ever agrees with you is just a faster way to ship your own mistakes. The one I actually want is the one that occasionally looks up from the billing table and asks whether I'm quite sure about that.
