Most AI pilots do not fail. They just never leave the pilot. A demo goes well, a slide deck gets nods, a channel gets created for "phase two", and then the calendar fills up with other things and nobody ever schedules phase two. Some time later someone asks what happened to that AI project, and the honest answer is that it is still sitting exactly where the demo ended.
That is the pilot scar. Most operators carrying one do not blame the technology. They blame a vague feeling that they did not ask hard enough questions before they started, and by the time the demo was over it was too late to ask them.
The questions below are the ones we think would have caught it. Run them against any vendor pitching you an AI project, ours included. None of them require you to understand the technology. They only require you to understand your own business, and to insist on plain answers before you sign anything.
Ownership
Who owns this after the pilot ends, a name, not a department? A pilot with no named owner has no owner once the person who championed it moves on to the next thing. Ask for the name before you start, not after the invoice arrives.
What happens to it if that person leaves? If the answer involves the vendor, the vendor now owns a piece of how your business runs. That may be fine. It should at least be a decision you made on purpose, not one you discover during someone's notice period.
The production path
What has to be true for this to run in production, not a demo? A demo works with clean data, a friendly test case and someone standing by to fix anything that breaks on camera. Production means your messiest customer, your worst week, and nobody standing by. Ask what changes between the two, specifically.
Who signs off before it touches real customers or real money? If nobody can name that person, the project is not ready to leave the sandbox, whatever the demo suggested.
What is the plan for when the model or the vendor changes underneath it? Every AI system sits on a model that will eventually be deprecated, repriced or replaced. A vendor with no answer here is asking you to bet your operations on their supplier staying still, which no supplier does.
Verification
What number tells you this is working, and where do you check it? Not a testimonial, not a sense that things feel smoother, an actual figure that exists somewhere you can look at whenever you want. If nobody can point to it, there is nothing to verify, only something to believe.
Can that number be re-derived by someone other than the vendor, or do you have to take it on trust? A number the vendor generates, reports and interprets for you is marketing. A number you or your own team could reproduce from raw data is evidence. Ask which one you are being offered.
What does failure look like, and who sees it first? Every system fails sometimes. The question is whether failure shows up on a scorecard you already look at, or whether you find out from a customer complaint weeks later. Ask for the failure case before you ask for the success case.
The exit
What does it cost, in time and effort, to turn this off? If unwinding the project would mean rebuilding a process from scratch because nobody wrote down what it replaced, that is a cost you are taking on silently. Ask for it explicitly, before you need it.
What do you keep if you walk away, the data, the process, the people who understand it? A good vendor relationship leaves you with something durable even if you part ways: data you can export, a process someone wrote down, a team that understands how the system worked without the vendor in the room. A pilot that leaves with the vendor was never really yours.
Ask us these too
We wrote this list against ourselves as much as anyone, so here is roughly what you would hear if you put it to us. A named person runs your function and stays reachable for as long as you are a client, not a rotating account team. We do not consider a project finished at the demo. We consider it finished when it is running unattended on a schedule and someone other than us could look at the output and understand where it came from. The numbers we report come with the method that produced them, so you or anyone you trust can check our working rather than take our word for it. And if the fit turns out to be wrong, in the first call or later, we would rather say so than keep an unhappy client on a contract.
None of that makes us the right choice for every business. It only means we can survive the same questions we are asking you to put to everyone else.
Run it before you commit
Copy the list into whatever you use to take meeting notes and bring it to the next AI pitch you sit through, ours included. If a vendor cannot answer most of it plainly, that tells you something worth knowing before you sign, not after the pilot quietly stalls.
If you would rather start with a conversation than a checklist, book the free audit. We will walk through your own answers to these questions, on the record, before we ever propose running anything.