Interrogating the Pitch: Seven Questions That Separate Production AI From Vaporware

Every enterprise AI pitch follows the same choreography. A slick demonstration, a roadmap slide, a case study with a familiar Fortune 500 logo. The sales team is well-rehearsed, and the average buyer asks the average questions: How much? How fast can we deploy? Who else uses it?

The trouble is that average questions produce average information. What separates a production-grade platform from a well-marketed prototype is rarely visible in the demo — it surfaces only when a buyer asks uncomfortable, specific questions and listens carefully to how the answer is constructed. After sitting through more of these evaluations than we can count, we have assembled the seven that matter most, paired with the tells that distinguish a real answer from a rehearsed dodge.

1. Where exactly does our data reside, and who can access it?

A strong answer names specific regions, cites SOC 2 and relevant compliance attestations, and clarifies whether customer data trains shared models. A deflection sounds like "your data is secure" — a phrase with no verifiable content. If the vendor cannot state data residency and access controls in one sentence, assume the worst.

2. What is the provenance of the underlying models?

Some vendors build their own models; many wrap someone else's. Neither is disqualifying, but the answer should be stated plainly. Evasive responses — "we leverage best-in-class foundation models" without naming them or explaining what was layered on top — suggest the vendor either does not know or does not want you to. Buyers deserve to understand dependency chains before betting operations on them.

3. Walk us through what happens when the system fails.

Every model fails. The question is whether failure is graceful. A credible vendor describes confidence thresholds, fallback behavior, human handoff, and how failures are logged and surfaced. A deflection reframes the question: "Our accuracy rates are industry-leading." Accuracy averages hide the tail events that actually destroy trust.

4. How do you handle model drift over time?

This question separates platforms from products. Production-grade vendors describe monitoring, scheduled evaluation, retraining pipelines, and how customers are notified when behavior changes. Vendors who have never thought about drift have never operated at scale.

5. Can we export everything, in what format, and at what cost?

Ask this early, not at renewal time. A strong answer includes documented export paths, standard formats, and no punitive fees. Hesitation here is a signal about the entire commercial relationship: vendors who make leaving expensive are counting on lock-in, not performance, to retain you.

6. Who owns the fine-tuned weights and custom configurations we create?

Enterprises invest real money tailoring platforms to their data. Ownership of those adaptations — the tuned models, the prompts, the embeddings — should be unambiguous. Contract language that clouds this is not a legal technicality; it is leverage the vendor is keeping for later.

7. What did you ship last quarter, and what did you cancel?

The final question is cultural. Strong engineering organizations ship consistently and kill what does not work, and they can discuss both candidly. A vendor with only a roadmap and no shipped history is selling a future tense. A vendor that has never cancelled anything is not telling you the whole story.

Reading the Answers, Not Just Hearing Them

Notice the pattern across all seven: the substance matters, but the shape of the answer matters just as much. Production-grade vendors answer with specifics, documentation, and named mechanisms. Prototype vendors answer with adjectives — secure, scalable, best-in-class — and pivot quickly back to the demo's highlights.

American enterprise buyers have learned to be skeptical of on-premises promises and cloud security claims; those muscles took a decade to build. The same discipline now applies to AI. The questions above will not eliminate risk, but they will reliably tell you which vendors have operated in production and which are hoping you cannot tell the difference. Ask them before the contract, not after.