There is a particular failure mode in the current AI moment. A client describes a problem; somebody on the team says “we should use AI for that”; six months later there is a fragile prompt-engineering system that does worse than a hundred-line script would have done.
AI is not free. It is slower than a deterministic system, more expensive per request, harder to test, harder to debug, and unpredictably wrong in ways that traditional code is not. None of that means do not use it. It means use it when it earns its place.
The heuristic, in one sentence.
Use AI when the input space is unbounded and the answer can be checked. Use rules when the input space is bounded or the answer cannot be checked.
Most problems where AI shines have one thing in common: free-form input that no rules engine could realistically enumerate. Free-text descriptions, photographs of receipts, customer support emails, document classification across thousands of variants. The input space is genuinely too large to write rules for.
Most problems where AI is the wrong answer have a small, well-defined input space. Order status flows, role-based permissions, working-hour calculations, leave-balance arithmetic. You can sit down with a notepad and enumerate the cases. If you can enumerate them, write them.
“Use AI when the input space is unbounded and the answer can be checked.”
The check matters as much as the input.
The other half of the heuristic is whether you can check the AI’s answer. AI is a probabilistic component, which means it will be wrong some of the time. If your system has no mechanism for catching wrong answers, you are putting probabilistic noise directly into your business.
For receipt OCR, the check is the human approving the expense. They see the extracted total next to the receipt photo. If it is wrong, they fix it. The AI is doing the hard part — reading messy handwriting at speed — and the human is doing the easy part — confirming.
For a system that automatically books refunds without human review, AI is much more dangerous, because there is no check. You are running an unpredictably-wrong component in a closed loop with real money. That is when you reach for rules — even if it is more work to write them.
Three patterns that earn AI’s overhead.
Structured extraction from unstructured input. Receipts, invoices, CVs, free-text forms — anywhere you have wildly varying input that needs to come out as structured data, AI will outperform regex by a country mile, and the human-in-the-loop check makes errors recoverable.
Classification across many fuzzy categories. Customer support routing, content tagging, sentiment analysis at scale. Rules engines work for ten categories; they fall over at a hundred. AI handles the long tail naturally.
Generation, with a human reviewer. Drafting first-pass copy, summaries, code snippets, replies. The AI produces a starting point; the human edits. Even a 70%-accurate draft is faster than a blank page.
- →Structured extraction with a human approval step
- →Classification across long-tail categories
- →First-draft generation reviewed by a person
Three places I have seen AI shoved in by force.
A “smart” permissions system that asks an LLM whether a user can perform an action. This is a closed input space — roles times resources times actions — that should be a few hundred lines of policy. Using AI here is slower, more expensive, and silently insecure.
A “natural language query” layer over a database that has fifteen tables and well-known queries. The users do not actually want natural language. They want a saved-searches dropdown. Building the saved-searches dropdown is a week; building the NL layer is six months and never quite works.
An “AI-powered” chatbot for a FAQ that has thirty entries. A search box would have served users better, faster, more accurately, and for zero per-call cost.
The hybrid pattern.
The systems I am most proud of use AI as one component in a larger system that is mostly rules. Receipts arrive; AI extracts fields; rules validate the fields against expected ranges; humans approve. AI does the open-ended part, rules do the closed-ended part, humans do the judgement part. Each layer plays to its strength.
When somebody says “let’s use AI for this”, the right question is which part. Almost every real system is a sandwich: deterministic intake, AI in the middle for the unstructured bit, deterministic validation and fallbacks on the way out. Treating AI as the whole system is a category error.
“AI does the open-ended part, rules do the closed-ended part, humans do the judgement part. Treating AI as the whole system is a category error.”