Inside BounceShift

How We Measure Email-Validation Accuracy (Honestly)

Most verification accuracy claims are unfalsifiable marketing. Here is how BounceShift measures accuracy against real-world delivery outcomes — and why we report abstentions instead of guessing.

By Hussam Abd 12 min read

Key takeaways

Every email validation service claims to be accurate. You have seen the headline number on the pricing page: some figure in the high nineties, followed by a percent sign and very little else. It is meant to end the conversation. In practice it should start one, because a bare accuracy percentage is one of the least falsifiable claims in this industry. Accurate at what? Measured against what? On whose data? Without answers to those questions, the number is decoration.

This is an essay about method rather than marketing. We want to explain how we think about email verification accuracy, what we actually measure, and why we would rather tell you we are not sure about an address than guess and be wrong. If you are evaluating any validation tool, including ours, these are the questions worth asking.

Why "99% accurate" means almost nothing

An accuracy claim is only meaningful if you can state three things: the prediction being graded, the ground truth it is graded against, and the population it was measured on. Most headline numbers leave all three unstated, which makes the claim impossible to check and impossible to disprove.

Consider the gaps a single percentage hides:

When a number cannot be tied to a task, a truth source, and a population, it is not a measurement. It is a slogan.

The two numbers that actually matter

We think about validation quality as a trade-off between two distinct measurements. Collapsing them into one figure is exactly how the meaningful detail gets lost.

Decisive accuracy

Of the addresses we give you a confident verdict on — a clear valid or invalid — how many of those verdicts turn out to be right? This is decisive accuracy. It only counts the calls we were willing to stand behind. It deliberately excludes the addresses we flagged as unknown, catch_all, or risky, because on those we did not make a confident claim and it would be dishonest to grade ourselves as if we had.

Coverage

Of every address you submit, what share do we resolve to a confident verdict at all? This is coverage. An engine that answered unknown for everything would have perfect decisive accuracy and useless coverage. An engine that forced a confident answer for everything would have high coverage and untrustworthy accuracy. Neither extreme is a product.

The trade-off

These two numbers pull against each other, and that tension is the whole game. You can always raise coverage by guessing on the hard cases — but every guess you are not sure about drags decisive accuracy down. You can always raise decisive accuracy by abstaining whenever you are unsure — but that lowers coverage. Anyone quoting one number without the other is hiding which lever they pulled.

If you optimize only for…You can score…But you hide…
CoverageA high "we have an answer for everything" rateThat many of those answers are guesses
Decisive accuracyA near-perfect "our confident calls are right" rateThat you abstained on most of the hard work
A single blended numberAn impressive headlineWhich of the above two you actually traded away

Our position is that both numbers should be visible and improved together over time, not that one should be sacrificed quietly to flatter the other.

Not all errors cost the same

There is a second principle underneath the math: the two ways of being wrong are not equally bad. This asymmetry shapes how the engine behaves.

A false invalid — telling you a good address is dead — costs you one lost contact. That is unfortunate, but it is recoverable and contained. A false valid — rubber-stamping a dead address as safe to send — is far worse. You send to it, it produces a hard bounce, and that bounce is reported to the mailbox providers who decide whether your future mail reaches the inbox. Enough of them and your sender reputation drops, which quietly suppresses deliverability for the good addresses too. One bad "valid" can poison the well for an entire send.

An honest abstention sits between the two and costs the least. When we tell you an address is unknown, you keep your options: hold it back, send to it cautiously, or warm it slowly. You are informed, not misled. Because the downside of a false valid is so much steeper than the downside of saying "we are not sure," the engine is tuned to abstain rather than guess whenever the evidence is thin.

This is why, when an SMTP probe comes back ambiguous — the mailbox sits behind a catch-all, the server is greylisting us, we are being rate-limited, or outbound port 25 is blocked — the engine returns unknown or catch_all with a low confidence score instead of upgrading the address to valid. A confidence number that does not drop when the evidence is weak is not a confidence number at all.

Grading against real delivery, not against ourselves

If decisive accuracy is the number that matters, then the ground truth we grade it against has to be real. The only ground truth that counts for deliverability is what happened when mail was actually sent: did it deliver, did it bounce, did it draw a complaint?

This is where the crowdsourced reputation network earns its place. When customers connect their email service provider webhooks — from providers like Mailgun, SendGrid, Postmark, and Amazon SES — the real outcomes of their sends flow back to us. A delivery, a bounce, a complaint: each is a fact about the world, not a prediction. Those facts are how we check our predictions. The shared reputation pool stores a one-way hash of each address, never the address itself, so the pool grows as a body of outcomes, not a list of people. (The addresses you check yourself, and their results, stay in your workspace, private to your team, until you delete them.)

Suppressing an address's own reputation when grading

There is a trap here, and avoiding it matters more than any single metric. Reputation data is one of the inputs the engine uses to make a verdict. If we then grade that verdict against the very same reputation data, we are grading the engine against its own input. It would score beautifully and prove nothing — a closed loop congratulating itself.

So when we measure ourselves, we suppress an address's own reputation history before asking the engine to judge it, and only afterward compare its verdict to the real delivery outcome that was held out. We are asking the harder, fairer question: given everything except the answer, did the engine get it right? That is the only version of the question worth scoring, and it is the version that keeps us honest about what the pipeline can do on an address it has never seen.

What happened when our verdicts were mailed

Here is how that works out with real numbers. Some of the addresses we check are mailed afterwards by senders whose email provider reports every delivery and bounce back to us. That gives us an outcome we did not produce. We took every address our current engine (version 16, live since July 30, 2026) checked up to November 10, 2026, and kept the ones that were mailed after the check.

The cleanest comparison is the verdicts that came from a live check, where the mailbox provider itself answered, because no delivery history went into them:

Our verdict (from a live check)Mailed afterwardsHard-bouncedDelivered at least once
valid13,8495 (0.04%)13,849
invalid160149 (93.1%)12 (7.5%)

Five bounces out of 13,849 addresses we called valid is the false-valid rate we care most about, and it is low. The invalid side is less flattering. Twelve addresses we called invalid were delivered to later, and nine of them were on Microsoft domains: of 46 Microsoft addresses we rejected, nine received mail afterwards. We publish that because it is the kind of number a single accuracy percentage would hide, and it tells you where to be careful with our invalid verdicts today.

The uncertain buckets behaved the way their names suggest. Of 3,422 addresses we marked unknown that were mailed anyway, 143 (4.2%) hard-bounced, more than a hundred times the rate for our live-check valid verdicts. Addresses marked risky (2,541 mailed) and catch_all (940 mailed) bounced about 0.5% of the time. (That catch_all figure covers only addresses whose final verdict was catch_all. Our catch-all study counts every address on a catch-all domain, including ones delivery history moved to another verdict, and found 3.4%.)

Two caveats. 97% of these mailed addresses were checked by one customer, and about 94% of the live-check valid verdicts are Gmail addresses, where mailbox checks are most reliable. And the two rows in the table exclude verdicts based on an address's own delivery history, because grading those against later deliveries would be the circular scoring described above. The unknown, risky and catch_all figures can include delivery history, so read them as rough.

How often we give a confident answer

Coverage is the other half. This is the share of checks that ended in a clear valid or invalid, by month, excluding cached repeats:

Month (2026)ChecksConfident verdictUnknown
July (older engines, v16 from July 30)192,28922.9%67.6%
August39,97175.5%14.5%
September60,18471.9%16.1%
October44,10270.9%21.6%
November 1–105,99974.5%11.5%

The averages depend on the mix of addresses, and the mix is lopsided. Between October 1 and November 10 we gave a confident verdict on 95.5% of gmail.com addresses and 66.7% of business and other domains, but only 10.8% of Microsoft consumer addresses (hotmail.com, outlook.com, live.com) and 4.0% of iCloud. Those two providers accept or block mailbox checks wholesale, so for them we usually say unknown. If your list is mostly iCloud, expect a lot of unknowns from us, and be wary of any tool that claims otherwise without saying how.

Why we return "unknown" on purpose

Put these principles together and a clear behavior falls out. When the cheaper, deterministic layers settle an address — broken syntax, a disposable domain, a missing MX record, a known bad reputation — we answer with confidence, because those signals are decisive. When the evidence runs out, we say so rather than inventing certainty.

Returning unknown or catch_all is not the engine failing. It is the engine refusing to convert your reputation into a coin flip. An honest unknown protects your decisive accuracy and protects your sending, which is the entire point of paying for validation in the first place.

The honest way to raise coverage is not to start guessing — it is to gather more real ground truth so that yesterday's unknown can become today's confident answer. This is the quiet compounding value of the reputation network. As more outcomes flow in from connected mail streams, more catch-all and ambiguous addresses gain enough real-world signal to be resolved confidently. Coverage rises because we learned something true, not because we lowered the bar for what counts as a verdict.

What this means for you

You do not have to take our framing on faith. Test it. Submit a sample where you already know the outcomes, send to the addresses we marked valid, and watch your bounce rate — a healthy hard-bounce rate is generally under 2 percent, and complaint rates above roughly 0.1 percent are dangerous. A validation tool earns trust by keeping those numbers low on the addresses it vouched for, not by quoting a percentage on a slide.

The same test works on any vendor, which is the useful part: run a sample you know the answers to through two services and compare what each one claims against what actually happened when you sent. If you are at the stage of picking between them, our tool-by-tool comparison covers pricing and behaviour; this page covers the question that matters more, which is whether to believe the accuracy number either of us prints.

If you want the mechanics of how each layer reaches its verdict, see how email verification works. If you are cleaning a list before a send, how to clean an email list walks through the practical steps. Our promise here is narrower and, we hope, more durable than a headline number: we will tell you what we measure, grade ourselves against reality rather than against our own guesses, and abstain out loud when we are not sure — because an honest "I don't know" is worth more to your reputation than a confident wrong answer.

HA
Hussam Abd
Founder & Engineer

Hussam Abd is the founder and engineer behind BounceShift. He has been building software since 2013 — full-stack engineering, then founding and leading engineering teams, including CTO and founder roles at MSAAQ and product and front-end work on customer-support tooling at Hsoub. He built BounceShift to make email verification that is accurate and honest about what it cannot know.

Related

Keep reading

Check your list before you send it

Upload a CSV and get back the addresses that are safe to send. 100 free credits to start, no card, and no charge for the ones we can't verify.