Partway into a data enrichment engagement, I found out our deduplication query was broken. Not slightly off. Fundamentally broken. We were sending duplicate records to a paid API, burning through the client's budget on data we already had.
I did the math. A meaningful share of the total spend was preventable waste, and I worked out what it cost before I said a word to anyone.
A note on that, since this post is about honesty: this article used to name a specific percentage. It came out of a working session rather than a query I can re-run today, so it is not on our published-figures register and it does not belong in print. The rule we hold ourselves to is that a number is publishable only if it can be re-derived. Losing the number costs this story very little. Keeping it would have cost more.
Here's what I did next: I built a slide deck quantifying the waste. I explained exactly how the bug occurred, what we'd done to fix it, and what safeguards we'd put in place. Then I presented it to the client.
The instinct to hide
The natural response to a mistake this significant is damage control. Minimize the impact. Reframe the narrative. Bury it in a broader status update where it becomes one bullet point among twenty.
In my own experience of this industry the pull toward that is close to universal, and I include myself in it. The quarterly business review tells a tidy success story. The failures get a passing mention: "we encountered some data quality challenges." The conversation moves on. Nobody asks follow-up questions because nobody wants to hear the answers. I am describing an instinct I have felt and had to fight, not an accusation against any particular firm.
This works short-term. The client stays calm. The engagement continues. Everyone exhales.
But it destroys something more valuable than comfort: trust.
What should a mistake report to a client include?
A mistake report worth giving a client reads as engineering documentation: the root cause, the financial impact with a timeline of when the bug was introduced and caught, the fix applied, and safeguards that stop the pipeline automatically when yield per batch drops.
Documenting mistakes isn't performative honesty. It's about building a specific kind of relationship with the client, one where they trust your assessment of what's working because they've seen your assessment of what isn't.
For that engagement, the mistake documentation included:
- Root cause analysis. The dedup query was matching on city name strings across tables with different naming conventions. The same city arrived as three different strings: lowercase, uppercase, and with a state suffix appended. SQL string comparison treated these as three different records.
- Financial impact. The wasted API spend, with a timeline showing when the bug was introduced and when it was caught.
- The fix. Switched to ZIP code matching with RTRIM/LTRIM/UPPER normalization. Added a Python-side set check as a safety net before any paid API call.
- Safeguards. New monitoring that flags yield per batch. If new records drop below expected rates, the pipeline stops automatically.
This isn't a confession. It's engineering documentation. The same rigor we bring to building systems, applied to understanding failure.
What does a failure report buy you with a client?
When we reported what the enrichment pipeline was doing and what it cost, they believed us. Not because the number was impressive, but because they'd already seen us present the number that wasn't impressive. They knew if something was broken, we'd say so.
The failure documentation stopped being something we had to survive and became part of the record the client could check our numbers against.
I've thought about that a lot.
The competitive advantage of honesty
Most AI consultancies sell on capability. Their decks showcase what they can do. Their case studies highlight successes. Their references are curated.
We sell on transparency. Our case studies include what broke. Our metrics include the waste. We tell clients about our own errors before anyone else can, which is the only reason a reference call is worth anything.
This filters for the right clients. Companies that want a vendor who tells them what they want to hear will hire someone else. Companies that want a partner who tells them the truth call us.
The filtering is the feature. Every client we work with knows exactly what they're getting. What that alignment does not buy is a clean record. Across sixty-nine projects, fourteen were built and never launched. One audit we delivered was engineered correctly, diagnosed correctly, and never deployed, and it produced exactly zero dollars of value for the client. Both of those numbers are published on this site. The alignment is worth having anyway, and the reason to believe that is that it is why we can publish them at all.
How we operationalized this
Documenting mistakes isn't a cultural value statement. It's a process:
- Every engagement has a decision log. Not just what we decided, but why, what alternatives we considered, and what we expected to happen.
- Pipelines get yield monitoring. Not every one had it in time: our own record includes an enrichment job that sat dead for weeks before anyone noticed, which is why the check now comes first.
- Every failure gets a cost quantification. Not "some records were duplicated" but how many records, at what unit cost, and what that added up to.
- Every client sees the full picture. Weekly updates include what went wrong alongside what went right. No curation. No cherry-picking.
This takes more time than the alternative. Writing a failure report is harder than writing a success report. Quantifying waste requires more analysis than ignoring it. Presenting bad news requires more preparation than skipping it.
But the time investment pays for itself in trust. And trust is the only thing that turns a single engagement into an ongoing one.
The uncomfortable truth
If you're not documenting your mistakes, you're not learning from them. You're just moving on to the next engagement and hoping the same bugs don't appear.
They will.
PurviewX is embedded AI leadership for companies sitting on real operational data. We find out whether your AI actually works, including ours. Start a conversation.