The Evaluation Process That Produces the Wrong Answer
Most tool evaluations in B2B growth follow a sequence that feels rigorous and produces decisions that are not.
A problem surfaces. Someone identifies a tool that seems responsible. Alternatives get researched — usually through a combination of G2 reviews, peer recommendations, and vendor demos. A shortlist forms. The shortlist gets evaluated on a trial, typically over two to three weeks, typically on a clean sample the vendor either provides or that your team selects because it is easy to work with. A decision gets made.
The problem with this sequence is not the process. It is what gets evaluated within it.
Domain resolution tools get compared on the features that are easiest to compare — interface quality, API response time, integration options, price per lookup. These are real considerations. They are also downstream of the only consideration that determines whether the tool is worth using at all: whether it returns the correct domain for the specific types of companies on your list.
That question — not any feature, not any benchmark, not any analyst ranking — is the evaluation. Everything else is context.
The reason most evaluations do not answer it is that answering it requires work that happens before the vendor demo, not during it. It requires understanding your own list composition well enough to know which companies are hard to resolve and why. It requires building a ground truth set against your own data. It requires measuring accuracy on your hardest cases, not your easiest ones.
This piece is about doing that evaluation correctly — specifically enough to be useful, honestly enough to hold up when the tool is in production.
What Vendor Accuracy Figures Actually Measure
Every domain resolution tool publishes an accuracy figure. Some publish several — overall accuracy, accuracy on enterprise lists, accuracy on SMB lists, accuracy by geography.
Before you use any of these figures to make a decision, you need to understand what they are measuring and what they are not.
Vendor accuracy figures are measured against reference datasets. Those datasets are selected by the vendor. The selection criteria — which industries, which company sizes, which geographies, which naming conventions — are almost never disclosed in the accuracy figure itself. They appear, if at all, in fine print that most buyers do not read before the decision is made.
This matters because accuracy is not a property of a tool in isolation. It is a property of a tool applied to a specific type of input. A tool trained predominantly on North American technology companies resolves those companies accurately. The same tool applied to a list concentrated in European professional services, or Southeast Asian manufacturing, or recently founded companies with minimal web presence, produces different results — results that its published accuracy figure does not describe.
The diagnostic question is not:
"What is your accuracy?" — It is: "What was your accuracy measured against, and how similar is that reference dataset to my list?"
A vendor who answers the second question specifically — disclosing the industry distribution, geographic distribution, and company size range of their reference dataset — is a vendor whose accuracy figure you can evaluate meaningfully. A vendor who cannot or will not answer it is a vendor whose accuracy figure tells you nothing actionable about your use case.
The absence of disclosure is itself information. Treat it as such.
Understanding Your Own List Before You Evaluate Anything
The prerequisite to any meaningful tool evaluation is an honest characterisation of what makes your list hard to resolve — not in general, but specifically across the dimensions that drive resolution difficulty.
Document these four dimensions before you run any evaluation. They tell you which vendor benchmarks are directly comparable to your situation and which ones are describing a different problem entirely.
Building a Ground Truth Set — The Right Way
The evaluation instrument that produces a reliable tool comparison is a ground truth set: a collection of company names where you have independently verified the correct domain, built from your own prospect data.
Most teams either skip this step or do it too quickly with too small a sample to be diagnostic. Here is what makes it actually useful.
Size and Composition
Four hundred to six hundred companies is a workable size — large enough to produce statistically meaningful accuracy comparisons, small enough to be manually verifiable in a reasonable timeframe. The composition matters more than the size. Your ground truth set should deliberately oversample your hardest cases: subsidiaries, recently rebranded companies, companies in your highest-ambiguity industries, companies in your lowest-coverage geographies.
A ground truth set composed entirely of straightforward cases tells you how tools perform on easy problems. Every tool performs well on easy problems. You need to know how they perform on your hard ones.
What Manual Verification Actually Requires
For each company in your ground truth set, correct domain verification means confirming the domain that employees at that specific entity actually use for email — not the domain of their parent company, not a product domain, not a domain that redirects correctly but has no mail infrastructure behind it.
Confirming this requires more than finding the company's website. A domain can have a live website and no email infrastructure. Checking whether a domain has MX records — the DNS records that direct incoming email — confirms the domain is configured to receive mail. Complete verification combines MX record presence with at least one corroborating signal: a contact email visible on the company's website, a LinkedIn employee profile, or direct confirmation through another reliable source.
Using the Ground Truth Set Across Time
The ground truth set you build for a tool evaluation does not expire after you make the selection decision. It becomes the instrument for monitoring whether the tool you chose continues to perform as your list composition evolves. Running your ground truth set against your chosen tool quarterly — and tracking accuracy over time rather than at a single point — tells you whether the tool's performance is holding or degrading.
Running the Evaluation — What to Measure and Why
Submit your ground truth set to each tool you are evaluating. Record four things for every result, not just whether the top answer was correct.
Top-1 Accuracy
The proportion of companies where the first domain the tool returned matched your ground truth. This is the standard metric. It is necessary and insufficient on its own.
Unresolved Rate
The proportion of companies where the tool returned no result. An explicit unresolved result is not a failure — it is the tool correctly communicating the limits of its confidence. A tool with a higher unresolved rate and a lower false confidence rate is often more trustworthy in production than a tool with a lower unresolved rate that fills the gap with wrong answers.
False Confidence Rate
Of the results the tool returned with high confidence, what proportion were wrong? This is the most consequential metric because high-confidence wrong results are the ones that enter your workflow without triggering review. A tool that is confidently wrong on 3% of high-confidence outputs will route wrong domains into your sequences without any flag.
Confidence Signal Quality
Does the tool return a confidence score alongside every result — or does it return only a domain? A tool that returns only domains is presenting all outputs as equally reliable. If the confidence signal does not correlate with actual accuracy in your ground truth evaluation, it is decorative rather than functional.
The Dual-Resolution Framework — And Its Honest Limitations
Running two resolution tools against the same company names and comparing outputs is a technique worth understanding — including where it works and where it does not.
Where this holds: When two tools independently return the same domain for a company, that agreement adds signal. Independent agreement across different approaches is stronger evidence than confident output from a single approach.
Where this breaks down: Many domain resolution tools in this space license data from overlapping or identical underlying providers. When two tools share significant data source overlap, their agreement on a wrong answer is not independent validation. It is correlated error. They are both wrong for the same reason — they both learned from the same incorrect source.
Before using dual-resolution as a confidence signal, establish whether the two tools you are comparing have meaningful data source independence. Used correctly, with genuinely independent tools, dual-resolution is a practical way to identify the companies in your list that require human review before entering your workflow.
What the Integration Layer Does to Real-World Accuracy
A tool's accuracy in isolation and its effective accuracy inside your actual workflow are two different numbers. The gap comes from three places.
-
Input Quality Variance
A tool's accuracy figure is measured on input that has been formatted consistently. Your actual company name data arrives from multiple sources — CRM exports, LinkedIn scrapes, conference lists, intent platforms — each with different formatting conventions. -
Confidence-Based Routing
Where does the resolved domain go after the tool returns it? If low-confidence results flow directly into your email finder without any routing logic, they enter your sequence without review. The accuracy loss here is not the tool's failure — it is an integration design failure. -
Re-Verification Cadence
A domain resolved correctly today may be incorrect in eight months if the company rebrands, is acquired, or changes its email infrastructure. A tool that produces accurate initial resolution but has no mechanism for flagging records that may need re-verification degrades in effective accuracy over time.
The Questions That Separate Serious Tools From Adequate Ones
Not a feature checklist — the specific questions whose answers reveal whether a tool was built by people who understand how domain resolution actually works inside a real pipeline.
- What was your accuracy figure measured against? The answer should include the industry distribution, geographic distribution, and company size range of the reference dataset. A vendor who responds with a single percentage and no dataset description is a vendor whose figure describes an unknown problem.
- What does your output look like for companies you cannot resolve confidently? The answer should describe an explicit unresolved or low-confidence output state. A vendor who says their system always returns a result is a vendor whose system fills uncertainty with guesses.
- How does your confidence scoring correlate with measured accuracy? Ask the vendor to show you the accuracy rate of results at each confidence tier. High-confidence results should be meaningfully more accurate than low-confidence results.
- How do you handle subsidiaries operating under names that do not match their parent entity's primary domain? The answer reveals whether the tool has been built to handle entity hierarchy or whether it resolves names without understanding entity relationships.
- What is your re-verification recommendation for records in a database for six months or more? A vendor who has not thought about this question has not thought about how their tool performs across the full lifecycle of a prospect database.
The Compounding Cost of Getting This Decision Wrong
There is a financial reality to a poor domain resolution tool decision that is worth making explicit, because it is almost never included in the cost analysis that precedes the decision.
Wrong domains do not produce one wrong output. They produce a cascade of wrong outputs — every contact built on that domain, every email generated from those contacts, every sequence those contacts enter.
In a workflow processing 30,000 company lookups per month, a tool accuracy difference of 8 percentage points — the difference between a 91% accurate tool and an 83% accurate tool — produces approximately 2,400 additional wrong domains per month. Each wrong domain generating an average of three contacts produces 7,200 wrong contacts entering your pipeline monthly. Over a quarter, that is more than 21,000 contacts built on wrong foundations.
The tool evaluation described in this piece — ground truth set, composition analysis, false confidence rate measurement, integration assessment — takes two to three weeks to run properly. That investment, made once before the tool goes live, is worth more than any optimisation you will make downstream of a domain resolution layer you never properly evaluated.
The Decision That Determines Everything Downstream
Every tool in your outbound stack operates within accuracy bounds set by the domain resolution layer beneath it. Email finders, verification platforms, sending infrastructure — their performance ceiling is determined by the quality of the domain input they receive.
This is a causal relationship, not a correlation. And it means the return on improving any tool downstream of domain resolution is bounded by how accurate your domains are.
The evaluation framework in this piece is not about finding the best-reviewed tool or the highest-accuracy benchmark. It is about finding the tool that performs most accurately on your specific list — and knowing, with measured confidence rather than assumed confidence, that the foundation your entire outbound stack is built on is one you have actually tested.
The growth lead who closed his laptop that afternoon had good tools. What he did not have was a domain resolution evaluation that would have told him, before eleven weeks of compounding damage, that the foundation those tools were built on was not what he thought it was.
That evaluation takes less time than recovering from not running it.