Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine
AbstractMinimal measurement error (reliability) during the collection of interval- and ratio-type data is critically important to sports medicine research. The main components of measurement error are systematic bias (e.g. general learning or fatigue effects on the tests) and random error due to biological or mechanical variation. Both error components should be meaningfully quantified for the sports physician to relate the described error to judgements regarding 'analytical goals' (the requirements of the measurement tool for effective practical use) rather than the statistical significance of any reliability indicators. Methods based on correlation coefficients and regression provide an indication of 'relative reliability'. Since these methods are highly influenced by the range of measured values, researchers should be cautious in: (i) concluding acceptable relative reliability even if a correlation is above 0.9; (ii) extrapolating the results of a test-retest correlation to a new sample of individuals involved in an experiment; and (iii) comparing test-retest correlations between different reliability studies. Methods used to describe 'absolute reliability' include the standard error of measurements (SEM), coefficient of variation (CV) and limits of agreement (LOA). These statistics are more appropriate for comparing reliability between different measurement tools in different studies. They can be used in multiple retest studies from ANOVA procedures, help predict the magnitude of a 'real' change in individual athletes and be employed to estimate statistical power for a repeated-measures experiment. These methods vary considerably in the way they are calculated and their use also assumes the presence (CV) or absence (SEM) of heteroscedasticity. Most methods of calculating SEM and CV represent approximately 68% of the error that is actually present in the repeated measurements for the 'average' individual in the sample. LOA represent the test-retest differences for 95% of a population. The associated Bland-Altman plot shows the measurement error schematically and helps to identify the presence of heteroscedasticity. If there is evidence of heteroscedasticity or non-normality, one should logarithmically transform the data and quote the bias and random error as ratios. This allows simple comparisons of reliability across different measurement tools. It is recommended that sports clinicians and researchers should cite and interpret a number of statistical methods for assessing reliability. We encourage the inclusion of the LOA method, especially the exploration of heteroscedasticity that is inherent in this analysis. We also stress the importance of relating the results of any reliability statistic to 'analytical goals' in sports medicine.
CitationAtkinson, G., Nevill, AM. (1998) 'Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine', Sports medicine, 26 (4) pp. 217-238
PublisherAdis International Limited.
- Measures of reliability in sports medicine and science.
- Authors: Hopkins WG
- Issue date: 2000 Jul
- Reliability and validity of a vertical numerical rating scale supplemented with a faces rating scale in measuring fatigue after stroke.
- Authors: Chuang LL, Lin KC, Hsu AL, Wu CY, Chang KC, Li YC, Chen YL
- Issue date: 2015 Jun 30
- Statistical Primer for Athletic Trainers: The Essentials of Understanding Measures of Reliability and Minimal Important Change.
- Authors: Riemann BL, Lininger MR
- Issue date: 2018 Jan
- Typical error versus limits of agreement.
- Authors: Atkinson G, Nevill A
- Issue date: 2000 Nov
- Test-retest reliability of skeletal muscle oxygenation measurements during submaximal cycling exercise in patients with chronic heart failure.
- Authors: Niemeijer VM, Spee RF, Jansen JP, Buskermolen AB, van Dijk T, Wijn PF, Kemps HM
- Issue date: 2017 Jan