We Asked Our AI the Same Question Ten Times. Here Is Why That Matters for Procurement

Procurement runs on numbers people stake budgets on. If an AI tool tells you a SaaS vendor's list price, a cloud provider's in-country presence, or a negotiation lever worth pushing, "the model said so" is not good enough. That belief shaped how we built the Kato Companion, and a benchmark published last week by an enterprise AI team validated it in an uncomfortable, useful way.

How we do it today: three sources, not one

Every insight in the Kato Companion is triangulated across three legs:

• The model. A self-hosted LLM does the reasoning and the language. It runs on our own infrastructure, so member data never leaves the perimeter.

• Theca, our library. A curated, community-fed knowledge base of procurement documents, contracts, benchmarks and category intelligence. This is vetted material, not scraped content. Retrieval is hybrid (keyword plus semantic), and every generated claim is checked against it for grounding and faithfulness before it reaches a screen.

• The live web. For anything recency-sensitive, prices, region launches, licensing changes, we verify against current public sources rather than the model's memory.

Around that triangle sit deliberate honesty rules. A vendor that publishes no list price shows as "Quote only", never an invented figure. A cloud region counts as present only if it is live in-country, and where we cannot verify, the cell says "announced" or "not verified" instead of forcing a yes or no. Pricing always states its own basis (per user per month, per seat, per node) rather than being silently normalised. Community-contributed figures are credited as supplied by a member of the Procurement Union Community. And nothing promotes to production without a human validating it in a sandbox first.

What the FlowX benchmark showed, and why we paid attention

FlowX.AI, a team running AI agents in regulated banking and insurance, recently benchmarked two very different AI engines on their own production questions, then did something most vendors skip: they sent the same question, byte-identical, ten times to each engine.

Three findings stood out to us:

1. Confidence scores are not measurements. One engine answered the same yes/no question eight times "no" and twice "yes", and reported at least 85 percent confidence on all ten answers, including both wrong ones. A model's self-reported certainty does not distinguish the answer it keeps from the answer it abandons.

2. One test run tells you almost nothing. Neither engine returned identical values across ten identical calls. A benchmark you run once is partly noise. Their own repeat run shifted the headline accuracy number.

3. Batched judgments contaminate each other. When one call scored five separate dimensions, a genuine problem on one dimension dragged down four scores that should have been clean. Answers produced in a single pass are not independent verdicts.

None of this says AI is unusable. It says reliability is something you engineer around the model, with thresholds, fallbacks and verification, not something you buy by picking a better model. That is precisely the bet our three-source architecture makes.

Where triangulation helps, and where it honestly does not

Our setup is strong on the question the industry talks about most: is the fact right? Cross-checking model output against Theca and the live web catches hallucinated figures and stale data, which is the failure mode that hurts a procurement professional fastest.

But the FlowX findings target a different question: is the pipeline stable and honest about its own uncertainty? Three sources do not automatically fix that. The model leg still jitters between runs. The step that reconciles the three sources can itself be a weak point if it relies on model judgment. And the sources are less independent than they look, since web content is increasingly AI-generated and models are trained on the web, so agreement is evidence, not proof.

Being candid about that gap is the point of this post. Thorough assessment means knowing what your architecture covers and what it does not.

What we are hardening next

Based on this, the near-term Procurement Union's Companion roadmap will need to add five reliability upgrades:

• Deterministic reconciliation rules. Explicit, testable logic per data type: numeric agreement within a defined tolerance, Theca wins on procurement methodology, the live web wins on prices and availability, and any conflict is surfaced to the user rather than silently resolved.

• Per-source verdict logging. Every insight row will carry what each of the three legs said, not just the fused answer. That gives us a permanent shadow record to audit disagreement rates at scale before changing anything.

• Repeat-run testing. Our evaluation suite will run each test case multiple times, using the exact production prompt text, versioned like code. An answer that flips between identical runs is a defect we want to find before a member does.

• Thresholds placed away from the noise. Wherever a score triggers an action, we will verify that real scores cluster far from the decision line, so run-to-run wobble can never flip a verdict.

• Fail to "no verdict", never to a broken screen. Any auxiliary check that times out or errors returns no result and the Companion keeps working. A safety check must never be able to break the thing it protects.

The principle underneath

We hold one rule across everything we build: verification lives inside the system, not in a promise. Guards, positive controls and self-tests ship with the feature, because a tool procurement professionals rely on for negotiation-grade numbers has to earn trust the same way a good sourcing process does, with evidence.

Built by procurement, for procurement.