BMJ 2014;348:g4115 doi: 10.1136/bmj.g4115 (Published 20 June 2014)
Page 1 of 2
Endgames
ENDGAMES STATISTICAL QUESTION
What is a non-randomised controlled trial? Philip Sedgwick reader in medical statistics and medical education Institute of Medical and Biomedical Education, St George’s, University of London, London, UK
Researchers investigated whether a low cost public campaign by local health authorities reduced antibiotic prescribing for outpatients. A non-randomised controlled trial study design was used. The intervention was multifaceted and included posters, brochures, and advertisements on local media, plus a newsletter on local antibiotic resistance targeted at doctors and pharmacists. The campaign focused primarily on the prescription of antibiotics for upper respiratory tract infections. The trial was based in Emilia-Romagna in northern Italy. The provinces of Modena and Parma (about 1 150 000 residents) were chosen for implementation of the intervention between November 2011 and February 2012. The provinces where the campaign was not implemented, with no information provided to doctors and pharmacists, formed the control group (about 3 250 000 residents). The primary outcome was the change in prescribing rates of antibiotics for outpatients over five months.1
differences between treatment groups in outcome will be due to differences in treatment.
Which of the following statements, if any, are true? a) The choice of provinces for implementation of the intervention was liable to allocation bias
Because the trial participants were not randomly allocated to treatment, the trial was liable to allocation bias (a is true). Allocation bias is a systematic difference between participants in how they are allocated to treatment.7 Allocation bias could have occurred in the above trial because the researchers chose the health districts in which the intervention was implemented. The researchers no doubt favoured the campaign and wished to show that it was effective. This might have led to them choosing to implement the intervention in health districts that they thought would show the greatest reduction in antibiotic prescribing, thereby increasing the difference in outcomes between treatment groups.
The researchers reported that a low cost community based public campaign targeted at residents, combined with a newsletter on local antibiotic resistance for doctors and pharmacists, was associated with a significantly decreased total rate of antibiotic prescribing. However, the campaign did not affect the participants’ knowledge and attitudes about antibiotic resistance.
b) The association between treatment and outcome was prone to confounding c) The study design promoted internal validity
Answers
Statements a and b are true, whereas c is false.
The randomised controlled trial study design is seen as the “gold standard” for establishing the effectiveness of treatments. Participants are allocated to treatment using simple2 or cluster random allocation,3 possibly using some method of restricted randomisation.4 The aim is to achieve treatment groups that are similar in group size and baseline characteristics, thereby minimising confounding.5 Provided the trial is double blind, ascertainment bias will be minimised.6 Therefore, any
In the above study, a non-randomised controlled trial study design was used. The principal difference in design compared with a randomised controlled trial is that participants in the trial above were not allocated at random to treatment. The aim of the trial was to ascertain whether a multifaceted campaign by local health authorities reduced the rate of antibiotic prescribing for outpatients. The trial participants were residents in Emilia-Romagna, a region in northern Italy that had 42 health districts. The intervention was implemented in the provinces of Modena and Parma (11 health districts). Health districts where the campaign was not implemented formed the control group. The researchers did not explain how provinces were selected for implementation of the intervention. Because the intervention was a public health campaign introduced by health authorities, informed consent on behalf of the participants in the intervention group was probably not deemed necessary. The control group received standard care and may not have been made aware that they were part of a trial. Any potential lack of blinding was possibly not relevant in relation to establishing the effectiveness of the intervention.
Because participants were not randomly allocated to treatment and the treatment groups were not similar in the number of participants, the two treatment groups were unlikely to be similar in baseline characteristics. Therefore, the association between treatment and outcome was prone to confounding (b is true). Hence, the non-randomised controlled trial study design did not
[email protected] For personal use only: See rights and reprints http://www.bmj.com/permissions
Subscribe: http://www.bmj.com/subscribe
BMJ 2014;348:g4115 doi: 10.1136/bmj.g4115 (Published 20 June 2014)
Page 2 of 2
ENDGAMES
promote internal validity (c is false). Internal validity is the extent to which the observed results can be ascribed to differences in treatment and not confounding, thereby allowing the inference of causality to be inferred.8
The non-randomised controlled trial study design is one of a group of designs collectively known as non-randomised studies. The group includes observational study designs, such as prospective and retrospective cohort studies, plus case-control studies. The potential contribution that non-randomised studies can make to the evaluation of the effectiveness of healthcare interventions has generated considerable debate. In particular, it has been suggested that the lack of randomisation in a non-randomised controlled trial, which potentially leads to differences between treatment groups in outcome owing to differences between treatment groups in baseline characteristics, results in the study design having limited value. However, in some trials random allocation may not be appropriate. In the trial above, the researchers did not explain how health districts were selected for implementation of the intervention. Presumably the public health campaign was carried out in those health districts where it was practically convenient or financially feasible because of the number of residents. It was probably necessary for the intervention to be implemented in health districts within the same provinces for practical reasons with regard to mass media, including television, radio, and newspapers.
For personal use only: See rights and reprints http://www.bmj.com/permissions
Although not intrinsic to the design of non-randomised controlled trials, the trial above was community based. An entire community within the population was included in the trial. Such an approach to sampling of study participants promotes external validity—that is, the extent to which the study results can be generalised to the population that the sample is meant to represent.8 This is in contrast to randomised controlled trials, which often have stringent inclusion criteria and can recruit only those patients who are willing to be randomised. This may introduce biases that make the trial participants not representative of the population. Competing interests: None declared. 1 2 3 4 5 6 7 8
Formoso G, Paltrinieri B, Marata AM, Gagliotti C, Pan A, Moro ML, et al; for the LOCAAL Study Group. Feasibility and effectiveness of a low cost campaign on antibiotic prescribing in Italy: community level, controlled, non-randomised trial. BMJ 2013;347:f5391. Sedgwick P. Random sampling versus random allocation. BMJ 2011;343:d7453. Sedgwick P. Treatment allocation in trials: cluster randomisation. BMJ 2014;348:g2820. Sedgwick P. Restricted randomisation. BMJ 2012;344:e1324. Sedgwick P. Confounding in clinical trials. BMJ 2012;345:e7951. Sedgwick P. Bias in clinical trials. BMJ 2011;343:d4176. Sedgwick P. Selection bias versus allocation bias. BMJ 2013;346:f3345. Sedgwick P. Randomised controlled trials: internal versus external validity. BMJ 2014;348:g1742.
Cite this as: BMJ 2014;348:g4115 © BMJ Publishing Group Ltd 2014
Subscribe: http://www.bmj.com/subscribe
Page 1 of 2
Endgames
ENDGAMES STATISTICAL QUESTION
What is a non-randomised controlled trial? Philip Sedgwick reader in medical statistics and medical education Institute of Medical and Biomedical Education, St George’s, University of London, London, UK
Researchers investigated whether a low cost public campaign by local health authorities reduced antibiotic prescribing for outpatients. A non-randomised controlled trial study design was used. The intervention was multifaceted and included posters, brochures, and advertisements on local media, plus a newsletter on local antibiotic resistance targeted at doctors and pharmacists. The campaign focused primarily on the prescription of antibiotics for upper respiratory tract infections. The trial was based in Emilia-Romagna in northern Italy. The provinces of Modena and Parma (about 1 150 000 residents) were chosen for implementation of the intervention between November 2011 and February 2012. The provinces where the campaign was not implemented, with no information provided to doctors and pharmacists, formed the control group (about 3 250 000 residents). The primary outcome was the change in prescribing rates of antibiotics for outpatients over five months.1
differences between treatment groups in outcome will be due to differences in treatment.
Which of the following statements, if any, are true? a) The choice of provinces for implementation of the intervention was liable to allocation bias
Because the trial participants were not randomly allocated to treatment, the trial was liable to allocation bias (a is true). Allocation bias is a systematic difference between participants in how they are allocated to treatment.7 Allocation bias could have occurred in the above trial because the researchers chose the health districts in which the intervention was implemented. The researchers no doubt favoured the campaign and wished to show that it was effective. This might have led to them choosing to implement the intervention in health districts that they thought would show the greatest reduction in antibiotic prescribing, thereby increasing the difference in outcomes between treatment groups.
The researchers reported that a low cost community based public campaign targeted at residents, combined with a newsletter on local antibiotic resistance for doctors and pharmacists, was associated with a significantly decreased total rate of antibiotic prescribing. However, the campaign did not affect the participants’ knowledge and attitudes about antibiotic resistance.
b) The association between treatment and outcome was prone to confounding c) The study design promoted internal validity
Answers
Statements a and b are true, whereas c is false.
The randomised controlled trial study design is seen as the “gold standard” for establishing the effectiveness of treatments. Participants are allocated to treatment using simple2 or cluster random allocation,3 possibly using some method of restricted randomisation.4 The aim is to achieve treatment groups that are similar in group size and baseline characteristics, thereby minimising confounding.5 Provided the trial is double blind, ascertainment bias will be minimised.6 Therefore, any
In the above study, a non-randomised controlled trial study design was used. The principal difference in design compared with a randomised controlled trial is that participants in the trial above were not allocated at random to treatment. The aim of the trial was to ascertain whether a multifaceted campaign by local health authorities reduced the rate of antibiotic prescribing for outpatients. The trial participants were residents in Emilia-Romagna, a region in northern Italy that had 42 health districts. The intervention was implemented in the provinces of Modena and Parma (11 health districts). Health districts where the campaign was not implemented formed the control group. The researchers did not explain how provinces were selected for implementation of the intervention. Because the intervention was a public health campaign introduced by health authorities, informed consent on behalf of the participants in the intervention group was probably not deemed necessary. The control group received standard care and may not have been made aware that they were part of a trial. Any potential lack of blinding was possibly not relevant in relation to establishing the effectiveness of the intervention.
Because participants were not randomly allocated to treatment and the treatment groups were not similar in the number of participants, the two treatment groups were unlikely to be similar in baseline characteristics. Therefore, the association between treatment and outcome was prone to confounding (b is true). Hence, the non-randomised controlled trial study design did not
[email protected] For personal use only: See rights and reprints http://www.bmj.com/permissions
Subscribe: http://www.bmj.com/subscribe
BMJ 2014;348:g4115 doi: 10.1136/bmj.g4115 (Published 20 June 2014)
Page 2 of 2
ENDGAMES
promote internal validity (c is false). Internal validity is the extent to which the observed results can be ascribed to differences in treatment and not confounding, thereby allowing the inference of causality to be inferred.8
The non-randomised controlled trial study design is one of a group of designs collectively known as non-randomised studies. The group includes observational study designs, such as prospective and retrospective cohort studies, plus case-control studies. The potential contribution that non-randomised studies can make to the evaluation of the effectiveness of healthcare interventions has generated considerable debate. In particular, it has been suggested that the lack of randomisation in a non-randomised controlled trial, which potentially leads to differences between treatment groups in outcome owing to differences between treatment groups in baseline characteristics, results in the study design having limited value. However, in some trials random allocation may not be appropriate. In the trial above, the researchers did not explain how health districts were selected for implementation of the intervention. Presumably the public health campaign was carried out in those health districts where it was practically convenient or financially feasible because of the number of residents. It was probably necessary for the intervention to be implemented in health districts within the same provinces for practical reasons with regard to mass media, including television, radio, and newspapers.
For personal use only: See rights and reprints http://www.bmj.com/permissions
Although not intrinsic to the design of non-randomised controlled trials, the trial above was community based. An entire community within the population was included in the trial. Such an approach to sampling of study participants promotes external validity—that is, the extent to which the study results can be generalised to the population that the sample is meant to represent.8 This is in contrast to randomised controlled trials, which often have stringent inclusion criteria and can recruit only those patients who are willing to be randomised. This may introduce biases that make the trial participants not representative of the population. Competing interests: None declared. 1 2 3 4 5 6 7 8
Formoso G, Paltrinieri B, Marata AM, Gagliotti C, Pan A, Moro ML, et al; for the LOCAAL Study Group. Feasibility and effectiveness of a low cost campaign on antibiotic prescribing in Italy: community level, controlled, non-randomised trial. BMJ 2013;347:f5391. Sedgwick P. Random sampling versus random allocation. BMJ 2011;343:d7453. Sedgwick P. Treatment allocation in trials: cluster randomisation. BMJ 2014;348:g2820. Sedgwick P. Restricted randomisation. BMJ 2012;344:e1324. Sedgwick P. Confounding in clinical trials. BMJ 2012;345:e7951. Sedgwick P. Bias in clinical trials. BMJ 2011;343:d4176. Sedgwick P. Selection bias versus allocation bias. BMJ 2013;346:f3345. Sedgwick P. Randomised controlled trials: internal versus external validity. BMJ 2014;348:g1742.
Cite this as: BMJ 2014;348:g4115 © BMJ Publishing Group Ltd 2014
Subscribe: http://www.bmj.com/subscribe
