Statisticians trace bias and variance back to the sampling frame by treating the analysis as a chain:

Target population → sampling frame → selection procedure → observed sample → estimator

1. Define the target population

They first specify whom or what they want to describe—for example:

“All active U.S. businesses with 10–500 employees as of September 2026.”

This matters because a database or list may cover only part of that population. A sampling frame can omit businesses, duplicate them, include inactive records, or contain outdated attributes.

2. Compare the frame with the target population

They audit:

• Undercoverage: eligible units missing from the frame
• Overcoverage: ineligible units included
• Duplicates: the same unit appearing multiple times
• Stale records: outdated addresses, status, size, or ownership
• Unequal inclusion probabilities: some units being easier to select than others
• Nonresponse patterns: selected units that fail to provide usable data

Undercoverage creates coverage bias when omitted units differ systematically from included units. For instance, a business database built from public web records may underrepresent small, private, recently formed, or less digitally visible firms. That is a problem with the data-generating process—not something that more sophisticated modeling automatically removes.

3. Reconstruct the selection probabilities

For each sampled unit, statisticians ask:

[ \pi_i=P(\text{unit }i\text{ is selected}) ]

If selection probabilities differ, they commonly use inverse-probability weights:

[ w_i=\frac{1}{\pi_i} ]

They may then calibrate or post-stratify the weights so the sample matches trusted population totals—such as industry, geography, company size, or age. This can reduce bias from unequal selection, provided the adjustment variables explain the differences between respondents and nonrespondents.

4. Separate bias from variance

For an estimator (\hat{\theta}):

[ \text{Bias}(\hat{\theta})=E[\hat{\theta}]-\theta ]

Bias is systematic displacement from the true population value. Variance is the amount the estimate would change across repeated samples. The two are distinct: an estimator can be tightly clustered but consistently wrong, or centered correctly but highly unstable 411.1. Bias and Variance — Data 88S TextbookThe figure on the top left corresponds to an estimator that is unbiased and has low variance. Its values are tightly clustered around the red dot. This kind of estimator is desirable: it is rarely far from the parameter, and it doesn't systematically overestimate or underestimate. The figure on the bottom left corresponds to an estimator that has low variance but is biased. We will give a ...data88s.

A useful decomposition is:

[ \operatorname{MSE}(\hat{\theta})

\operatorname{Bias}(\hat{\theta})^2+ \operatorname{Var}(\hat{\theta}) ]

5. Use design-based variance estimation

Variance is traced to the sampling design by asking how many possible samples could have been drawn under that design. Methods include:

• Taylor linearization
• Replicate weights
• Jackknife or balanced repeated replication
• Design-based bootstrap
• Cluster and stratified variance estimators

A jackknife, for example, repeatedly omits observations to assess how much the estimate changes 1Jackknife Resampling Explained: Estimating Bias and VarianceJackknife resampling helps estimate the bias and variance of sample statistics by systematically removing one observation at a time from your dataset. This method gives you insights into how stable your statistical estimates are and whether they might be systematically over- or underestimating the true population parameter.statology. Ordinary bootstrap resampling estimates the distribution of an estimator by repeatedly sampling with replacement from the observed data 2Bootstrap Method - GeeksforGeeksBootstrap Method is a powerful statistical technique widely used in mathematics for estimating the distribution of a statistic by resampling with replacement from the original data. The bootstrap method is a resampling technique that allows you to estimate the properties of an estimator (such as its variance or bias) by repeatedly drawing samples from the original data. It was introduced by ...geeksforgeeks.

However, resampling cannot repair a defective frame. If an entire subgroup is absent from the original frame, repeated resampling merely reproduces that omission. It can estimate sampling variability conditional on the observed frame, but not the full uncertainty caused by frame undercoverage.

6. Run sensitivity and external-validity checks

To determine whether frame problems materially affect conclusions, statisticians compare:

• weighted versus unweighted estimates;
• results across frame versions or vendors;
• estimates for early versus late respondents;
• sample distributions against external benchmarks;
• results after removing duplicates or stale records;
• alternative assumptions about missing or unreachable units;
• estimates across demographic, geographic, industry, or size strata.

They may also conduct a selection-bias sensitivity analysis: assume that the missing portion of the population differs from the observed portion by plausible amounts, then calculate how much the final estimate would move.

Practical audit for a commercial database

For a source such as a business-contact database, the audit trail should record:

1. Frame definition: exactly what entities and fields are covered.
2. Reference date: when records were collected or last verified.
3. Eligibility rules: how active, duplicate, subsidiary, or closed entities were treated.
4. Sampling mechanism: random, quota-based, ranked, convenience, or algorithmic selection.
5. Deduplication and refresh procedures.
6. Missingness and bounce/nonresponse rates.
7. Inclusion probabilities or sampling weights.
8. Known coverage gaps and validation benchmarks.
9. Variance method: whether clustering, stratification, and weighting were incorporated.
10. Sensitivity results: how estimates change under plausible frame-error assumptions.

The key principle is:

Variance is usually estimable from repeated selection within the frame; bias requires evidence about what the frame excludes and how excluded units differ.

Thus, statisticians do not infer frame quality from a small standard error. A large database can have low sampling variance and substantial coverage bias at the same time.