Intercoder reliability
Assess consistency of code application
Intercoder reliability evaluates how consistently independent coders apply a defined coding scheme to the same material. Depending on the design, statistics such as Cohen's kappa or Krippendorff's alpha address agreement beyond raw percentages or accommodate particular coding structures. This is a quality procedure for approaches seeking reproducible coding, not a universal requirement for interpretative qualitative research.
Use when stable category application across coders is part of the methodological objective. Define coding units and rules before assessing agreement, and choose a statistic appropriate to the data structure.
Strengths
- Identifies ambiguous codes and training needs
- Supports transparent reproducibility claims for a coding frame
Limitations
- High agreement can coexist with invalid categories
- Chance-corrected statistics depend on prevalence and coding design
Know the boundary
It does not validate interpretations by itself and is not a default criterion for reflexive thematic analysis.