Sample size calculations
Objectives
At the end of the lecture and having completed the exercises students should be able to:
- Outline the necessity of a priori sample size calculations
- Explain the concepts of Type 1 errors, Type 2 errors and statistical power
- Describe the relationships between sample size, power, significance level and standard deviation
Bibliography
Altman, D.G., Statistics and ethics in medical research: III How large a sample? BMJ 281: 1336-8
Written for medics, good brief introduction to sample size calculations.
(Also appears in: D.G. Altman & S.M. Gore, 1982. Statistics in Practice, pp. 6-8. BMA, London)
Altman, D.G., 1991. Practical statistics for medical research, pp. 455-460. Chapman & Hall, London.
Written for medics, contains a good introduction to sample size calculations
Bland M., 1995. An Introduction to Medical Statistics, 2nd ed., pp. 331-341. Oxford Medical Publications, Oxford.
Written for medics, gives simple formulae and basic tables. Similar to, but slightly more mathematical than Altman's book.
Campbell M.J. & D. Machin,1993. Medical Statistics, A Commonsense Approach, 2nd ed., pp. 156-157. John Wiley & Sons, Chichester.
Written for medics, contains a brief summary, equations and examples.
Machin, D., M.J. Campbell, P.M. Fayers & A.P.Y. Pinol 1997. Sample Size Tables for Clinical Studies, 2nd ed. Blackwell Science, Oxford.
Written for statisticians. Tables and equations for most situations, lots of examples. Very mathematical.
Why bother with sample size calculations?
- if the sample size is too small a real difference between treatments could be missed
- "negative results" can be useful if a real difference could have been detected
- journals and research funding bodies require sample sizes calculations a priori
- it is unethical to inconvenience patients if a study could not have detected a useful effect
Sample size calculations are sometimes referred to as power calculations, for reasons that should be obvious when you have finished reading this document.
Definitions
Study | |||
---|---|---|---|
Difference | No difference | ||
True situation | Difference | True positive | False negative |
No difference | False positive | True negative |
Type 1 error
The risk of a false positive result (α).
i.e. the chance of detecting a statistically significant difference when there is no real difference between treatments.
Type 2 error
The risk of a false negative result (β).
i.e. the chance of not detecting a significant difference when there really is a difference.
Power (1-β)
The chance of not getting a false negative result.
i.e. the chance of spotting a difference as being statistically significant if there really is a difference.
Higher power is better, aim for at least 90% power. 80% power is the minimum acceptable.
Relationships
There will be a bigger sample size with:
- High power
- Small difference to detect
- Large standard deviation
- Low chance of false positive (small α)
There will be a smaller sample size with:
- Low power
- Large difference to detect
- Precise outcome measure (small standard deviation)
- High chance of false positive (large α)
Calculating sample sizes
Equations
- Mathematically derived from formulae used for hypothesis tests
- Different formula for each type of test
- Need some information from e.g. previous studies to use the equations
- Generally give number per group
Tables
- Based on equations
- Different tables for different tests
- Generally give number per group
Nomograms
- Based on equations
- An easy to use graph
- Generally give total number for study
- Not much used now that computer programs for sample-size calculation are available
An easier method
A simple way of calculating approximate sample sizes is given another page in this section.
Standardised difference (Δ)
- difference to detect is normally the smallest clinically worthwhile difference
- the standard deviation of measurements will normally have to estimated from previous studies
Remember
- Sample size calculations should be done before the study is started
- Base sample size calculations on the principal outcome measure
- The statistical analysis technique has to be decided on beforehand
- Check if the calculation gives the total number or the number per group
- Sample sizes are very approximate
- Sample size may have to be increased to allow for patients who withdraw
- Other constraints may apply, e.g. money or number of patients available. We may have to calculate the power from the sample size available rather than the sample size from the power required
- Sample size calculations can become very complex and complicated. Consultation with a statistician is vital