We get asked to add AI to things. Sometimes it is the right call. Often the honest answer is that a simple database lookup would do the same job, faster and for nothing, and it would not be wrong twice a month.
Here is how we decide.
The test
Does the job have a right answer?
If it does, AI is usually the wrong tool. "How many orders shipped late last month" has a right answer, and a query gives it to you correctly every time. Putting AI in front of that question adds cost, adds a delay, and adds a small chance of a confidently wrong number. You get nothing back for it.
If it does not, AI gets interesting. "Summarise what these 200 support tickets are complaining about" has no single right answer. Any reasonable summary is useful, a slightly different one next week is fine, and no query can produce it at all.
That is the line. Questions with one right answer want ordinary code. Questions with no single right answer are where AI earns its keep.
Then: what does being wrong cost?
A wrong summary wastes a minute. A wrong invoice total is a serious problem. The higher that cost, the more the model needs a human between it and the outcome, and the more the feature is really a drafting tool rather than an automation.
Where it consistently pays
Reading unstructured text. Emails, tickets, forms, uploaded documents. Pulling the useful details out of ordinary writing is genuinely hard to code, and it is something AI does well. Classifying incoming enquiries, extracting details from a supplier PDF, routing a message to the right team.
Search that understands intent. Not keyword matching. Someone typing "the bracket that fits the older model" finding the right product without knowing its name. For catalogues with technical products, this reliably moves numbers, because the alternative is the customer giving up.
First drafts. Product descriptions, replies, summaries. The value is the blank page problem, not the finished text. Anyone using the output unedited is misusing it, and a good implementation makes editing obvious rather than optional.
Triage. Sorting a queue by likely urgency, flagging what looks unusual, grouping similar things. It does not need to be right. It only needs to beat the order things happened to arrive in, which is a low bar.
Where it does not
Anything with a number that has to be exact. Totals, stock, prices, availability. Use the database. This is not a limitation you engineer around, it is what the tool is.
Compliance and regulated advice. If being wrong has legal weight, a model that is right 97% of the time is a liability rather than a feature.
Replacing a form. A conversational interface over three fields is slower than three fields. People know how to use a form.
Making a bad product look modern. If the underlying thing is confusing, a chat box is a way of asking customers to describe your own product back to you.
The costs nobody quotes
Per call, forever. Unlike code you have written, every use costs money. Fine at a thousand a month, worth modelling at a million.
It is slower than code. Asking AI something takes one to three seconds. Sometimes irrelevant, sometimes the difference between a screen feeling instant and feeling broken.
It changes underneath you. Providers update models and behaviour shifts. So anything you rely on needs regular checks that the answers still hold up. Most people building these skip that.
Someone has to watch it. Reading a sample of what it produces and noticing when it starts getting worse. Budget it or the feature quietly degrades and nobody notices until a customer does.
How we would start
Pick the smallest thing where the job is fuzzy and being wrong is cheap. Put a human in the loop. Measure whether it actually saved anyone time, honestly, against how long the task took before.
Then keep it or remove it. The willingness to remove it is what separates this from buying a feature because it was available.
Most of what we build has no AI in it at all. Clients who ask us for it usually leave the call with a shorter list than they arrived with. That tends to be the useful outcome.
