Power analysis
Plan information around a meaningful effect
Power analysis links a specified statistical test, effect magnitude, error rates and sample design to the probability of rejecting the null under an alternative. A priori analysis plans sample size; sensitivity analysis identifies detectable effects for a feasible sample. It should incorporate clustering, attrition and multiplicity where relevant. Observed post hoc power computed from the observed effect adds little to its confidence interval.
Use before confirmatory quantitative work when the primary estimand, analysis and smallest effect of practical interest can be specified, or to evaluate what a fixed feasible dataset can detect.
Strengths
- Connects recruitment to the scientific decision
- Exposes unrealistic expectations about detectable effects
Limitations
- Results depend on uncertain effect and variance assumptions
- Simplified calculators can ignore complex dependence or attrition
Know the boundary
Power is a probability under assumed alternatives, not the probability that the hypothesis is true.