Sample Size

What Is Sample Size?

Sample size is the number of people, items, events, or data points included in a study, survey, experiment, or analysis. Usually written as n, it helps researchers draw conclusions about a larger group called the population.

A population is the full group a researcher wants to understand, such as all customers of a service. A sample is the smaller group measured. Each person, purchase, product, record, or other measured unit is an observation. If a survey receives 350 completed customer responses, its sample size is n = 350, even if the business has millions of customers. Sample size describes how much data was collected, not whether it is accurate or representative.

Readers can explore related definitions in the statistical glossary. A useful sample needs enough observations for the decision and a sound selection method.

Why Sample Size Matters in Research and Surveys

Sample size affects the random uncertainty around a result. More independent observations generally produce more precise estimates, making survey percentages or averages less likely to change sharply because a different random group responded.

It also affects statistical power, or the chance that a study will detect a real effect of a chosen size. A product experiment with too few users may miss a meaningful improvement. However, more observations cost money and time and can increase participant burden. The goal is not the largest possible sample, but one large enough for a useful decision.

Sample size reduces random error, not systematic bias. Thousands of responses from an unrepresentative list, a leading question, or an excluded audience can still mislead. Better sampling and measurement can matter as much as a larger n.

How Sample Size Determination Works

Sample size determination should begin before data collection. Its calculation depends on the question and assumptions.

1. Define the decision, such as estimating customer satisfaction or comparing checkout-page versions.
2. Identify the target population and outcome, including whether it is a proportion, average, rate, or group difference.
3. Choose a design, such as random or stratified sampling, a controlled experiment, or a cluster-based field study.
4. Set the needed precision or smallest effect worth detecting.
5. Estimate variation, baseline behavior, and expected response or completion rates using prior data when available.
6. Calculate the required completed, eligible observations, including targets for important groups.
7. Document assumptions, exclusions, stopping rules, and adjustments for clustering, unequal groups, or missing data.

Key Inputs in a Sample Size Calculation

Inputs vary by study type. A survey estimating a percentage uses different assumptions from an experiment comparing outcomes.

The value 1.96 is commonly used for a two-sided 95% confidence interval because it is the standard normal cutoff leaving about 2.5% in each tail. It applies only when the calculation's assumptions are appropriate.

Sample Size for Surveys, Experiments, and Observational Studies

There is no universal rule that 20, 40, or 100 observations are enough. The right size depends on purpose, design, variability, and the consequence of being wrong.

A Practical Sample Size Example

Suppose a retailer needs a customer survey precise enough to guide a service-improvement decision. After setting confidence and margin-of-error goals, it determines that it needs 400 completed responses. That is the target sample size, not the number of invitations sent.

If prior surveys show that one quarter of invited customers complete the questionnaire, the retailer may need to invite about 1,600 people for 400 usable responses. A form-building tool can collect responses, but it cannot ensure that the invitation list represents all customer types. If results are needed separately for new and returning customers, adequate completed samples should be planned for both groups.

Benefits of an Appropriate Sample Size

A well-planned sample provides useful evidence without unnecessary data collection.

• It produces more stable estimates and narrower confidence intervals.
• It improves the ability to detect meaningful effects.
• It supports credible comparisons across planned customer, product, or location groups.
• It helps plan budgets, recruitment, and data collection.
• It reduces the risk of strong conclusions based on random fluctuations.
• It makes methods easier for others to review and trust.

Practical Limits and Challenges

More data is not free or automatically better. Real-world constraints must shape the plan.

• Recruiting participants, running tests, and reviewing records can be expensive and slow.
• Privacy, consent, and ethical limits may restrict contact or data collection.
• Low response rates and missing answers can reduce usable data and introduce bias.
• Rare populations may be difficult to reach in sufficient numbers.
• Clustered data, such as students within schools, provides less independent information than unrelated observations.
• A statistically significant result may still be too small to matter operationally.

Common Sample Size Mistakes

Many errors come from treating sample size as a shortcut rather than a design decision.

• Assuming a fixed percentage of the population is automatically sufficient.
• Confusing a large sample with a representative sample.
• Ignoring nonresponse, incomplete records, and ineligible participants.
• Choosing the calculation after reviewing results.
• Planning for an overall result but not needed subgroup comparisons.
• Counting duplicates or repeated measurements as fully independent observations.
• Stopping collection when a desired result appears, which can inflate false positives.

Best Practices for Choosing and Reporting Sample Size

A transparent report helps readers judge whether a sample supports the claimed conclusion. This is especially important for policy, safety, health, finance, or major product decisions.

• State the target population and sampling frame used to reach it.
• Describe the sampling design, recruitment method, and eligibility rules.
• Report the planned sample size and final completed and excluded observations.
• Explain assumptions about precision, effect size, variation, power, and significance thresholds where relevant.
• Report response rate and likely nonresponse or coverage limitations.
• Specify subgroup analyses and adjustments for clusters, repeated measures, or unequal group sizes.
• Use a clear stopping rule and retain the original plan alongside justified changes.
• Consult a statistician when decisions are consequential, the design is complex, or data may affect people unfairly.