Page 1 of 2

Endgames

ENDGAMES STATISTICAL QUESTION

Uncertainty in sample estimates: sampling error Philip Sedgwick reader in medical statistics and medical education Institute for Medical and Biomedical Education, St George’s, University of London, London, UK

The effects of a shared care obesity management programme on body mass index (BMI) and related outcomes in obese children were investigated using a randomised controlled trial. The intervention consisted of general practice surveillance for childhood obesity, followed by obesity management across primary and tertiary care settings using a shared care model. The intervention was delivered over one year. The control treatment consisted of “usual care.” Participants were children aged 3-10 years who had a BMI above the 95th centile for their age and sex. The setting was Melbourne, Australia. In total, 118 children were recruited through their general practice and randomised to the intervention (n=62) or control (n=56).1 The primary outcome was BMI. Measurements of BMI were transformed to z scores. At the end of follow-up, the intervention group had a smaller mean BMI z score than the control group, although the difference was not significant (2.0 (standard deviation 0.5) v 2.0 (0.4); adjusted mean difference −0.05, 95% confidence interval −0.14 to 0.03; P=0.2). No evidence of a significant difference was found for any secondary outcome, including body fat percentage, waist circumference, physical activity, quality of diet, and health related quality of life. It was concluded that the shared care model of primary and tertiary care management had no effect on BMI and related outcomes in obese children. Which of the following statements, if any, are true? a) The sample estimates were prone to sampling error

b) Sampling error would have occurred as a result of taking a sample from the population c) The sampling error was quantified by the standard error of the mean BMI z scores at follow-up d) Generally, sampling error decreases as sample size increases

Answers

Statements a, b, c, and d are all true.

The aim of the trial was to establish the effects of a shared care obesity management programme on BMI and related outcomes in obese children. The primary outcome was BMI. Patterns of growth change as children mature, and they also differ between

boys and girls. Therefore, to enable children of different ages and sexes to be compared, the BMI measurements were standardised by deriving z scores.2 The sample mean BMI z score at follow-up for each of the treatment groups was an estimate of the population parameter for that treatment—that is, the mean BMI z score that would be seen if all children in the population received that particular treatment. Furthermore, the observed difference in mean BMI z score between treatment groups at 12 months was an estimate of the population parameter of the difference in mean BMI z score between treatments.

A sample was taken so that inferences could be made about the population with regard to the effects of the intervention versus control treatment. The sample was just one of the many possible samples from the population. The nature of sampling meant that the demographic characteristics and other characteristics that may influence someone to participate in, or withdraw from, a trial were unlikely to have been the same for the sample as for the population. There was therefore uncertainty about the inferences of the population parameters based on the sample estimates. The aim of statistics is to quantify the uncertainty of such inferences. Although it is hoped that the sample estimates would be similar in size to the population parameters, it is unlikely that they would be exactly the same. Therefore, the sample estimates were prone to sampling error (a is true). Sampling error is the difference in size between a sample estimate and the population parameter. The word “error” in the statistical sense is not meant to infer that a mistake has been made when sampling. The error is the inaccuracy of the sample values as estimates of the population parameters—a direct result of having obtained a sample to make inferences about a population (b is true). The population parameters are unknown. Therefore, sampling error is a theoretical concept. Sampling error is quantified by the standard error of the mean (c is true). The standard error of the mean (SEM), sometimes shortened to standard error (SE), is derived from the sample data. Generally, the larger the standard error, the less accurate the sample mean as an estimate for the population parameter. In the study above, the mean BMI z score at follow-up in each treatment group was an estimate of the population parameter for that treatment. Although not

[email protected] For personal use only: See rights and reprints http://www.bmj.com/permissions

Subscribe: http://www.bmj.com/subscribe

BMJ 2015;350:h1914 doi: 10.1136/bmj.h1914 (Published 10 April 2015)

Page 2 of 2

ENDGAMES

presented, the standard error of the sample estimate for each treatment group was obtained by dividing the standard deviation of the BMI z scores at 12 months by the square root of the number of participants in the treatment group. Therefore, as sample size increases the standard error generally decreases, representing a more accurate sample estimate for the population parameter. The standard error of the mean and the standard deviation are often confused.3 The standard deviation of the BMI z scores does not provide a measure of sampling error but an indication of the spread of the measurements of the scores in the sample. The standard error of the mean and sampling error are often confused. Sampling error is a theoretical concept that is quantified by the standard error. By itself, the standard error of the mean BMI z score for a treatment group has limited use. It is used to derive the confidence interval—a range of values that quantifies the uncertainty in the sample mean as an estimate of the population parameter. However, when comparing the intervention against the control treatment it would have been good practice and more informative to present a confidence interval for the difference between treatment groups in mean BMI z score rather than compare confidence intervals for the mean BMI z score for each treatment group.4 Although not presented, the standard error for the difference between treatment groups in mean BMI z score at 12 months was derived; this was used to calculate the 95% confidence interval for the population parameter of the difference in mean BMI z score between treatments at follow-up.

For personal use only: See rights and reprints http://www.bmj.com/permissions

In general, as sample size increases and approaches the population size the sample will more accurately represent the population. As sample size increases, sample estimates become closer in size to the population parameter, thereby reducing sampling error (d is true). As described above, sampling error is quantified by the standard error of the mean; as sample size increases, the standard error generally decreases, reflecting the reduction in sampling error. Furthermore, sampling error will be controlled if, as well as increasing sample size, the method used to recruit participants produces a sample that is representative of the population. In the example above, children were recruited using convenience sampling from general practices in Melbourne.5 It is therefore difficult to quantify how representative of the population the sample of children was, but an alternative method of recruitment, such as stratified cluster sampling, would have produced a more representative sample.6 Competing interests: None declared. 1 2 3 4 5 6

Wake M, Lycett K, Clifford SA, et al. Shared care obesity management in 3-10 year old children: 12 month outcomes of HopSCOTCH randomised trial. BMJ 2013;346:f3092. Sedgwick P. Standardising outcome measures using z scores. BMJ 2014;349:g5878. Sedgwick P. Standard deviation or the standard error of the mean. BMJ 2015;350:h831. Sedgwick P. Randomised controlled trials: inferring significance of treatment effects based on confidence intervals. BMJ 2014;349:g5196. Sedgwick P. Convenience sampling. BMJ 2013;347:f6304. Sedgwick P. Stratified cluster sampling. BMJ 2013;347:f7016.

Cite this as: BMJ 2015;350:h1914 © BMJ Publishing Group Ltd 2015

Subscribe: http://www.bmj.com/subscribe