Industry watch
Tobacco industry-funded research on standardised packaging: there are none so blind as those who will not see! Pascal Diethelm,1 Martin McKee2 1
OxyRomandie, Geneva, Switzerland 2 London School of Hygiene and Tropical Medicine, London, UK Correspondence to Pascal Diethelm, OxyRomandie, 2, rue de la Fontaine, Geneva CH-1204, Switzerland; [email protected] Received 16 April 2014 Accepted 6 June 2014 Published Online First 7 July 2014
To cite: Diethelm P, McKee M. Tob Control 2015;24:e113–e115.
Support for standardised packaging of tobacco products seems to be reaching a tipping point. Australia has already adopted this measure. In New Zealand, legislation is in progress. Ireland and England have now committed to implementing standardised packs while many other countries, including Canada, Norway and Turkey are actively considering them. The tobacco industry, which has invested vast sums in the design of packs that appeal to new smokers, especially adolescents, is worried and, just as it did when bans on smoking in public places were being considered, is engaged in a wideranging and well-funded campaign to undermine the evidence.1 Laverty and colleagues warned about this, describing one such example using Australian data. They showed how the design of the study was such that it would have been virtually impossible to detect a significant effect.2 However, as we will now show, this was not an isolated incident. Their warning would seem to be justified by a working paper published on the website of the Department of Economics of the University of Zürich, entitled ‘The (Possible) Effect of Plain Packaging on the Smoking Prevalence of Minors in Australia: A Trend Analysis’,3 funded by Philip Morris International. Its authors also conclude that there is no evidence that standardised packaging works and Philip Morris has press released it,4 generating headlines such as ‘Plain packs derided as not working’5 and ‘New data proves plain pack cigarettes doesn’t dissuade young smokers’.6 However, as we shall show, once again, it was almost inevitable that the data and methods used would fail to detect any expected effect. The study looks at the ‘prevalence of smoking among Australians aged 14–17 years’, taking monthly prevalence estimates from a marketing survey, of which the authors give no further detail, except that it contains a ‘0–1 “smoker” variable’ derived from personal interviews, a variable which is reported elsewhere to be inappropriate for assessing adolescent smoking.7 Yet, the purpose of standardised packaging is, as the authors acknowledge, to discourage smoking initiation and encourage quitting. Measures of prevalence cannot distinguish those who took up smoking after standardised packaging was introduced from those who took it up previously, with the former likely to comprise only between one-quarter and one-third of the age group. The data used in the study show that the prevalence of smoking in Australia in the 14–17 years age group was 5.6% in 2012. The lack of power of the study is then apparent and has been documented by others.2 7 If it is assumed that the introduction of
standardised packaging would have negligible effect on quitting rates in the 14–17 years age group, a reasonable assumption, as few adolescents readily stop smoking, as ‘they are dependent on nicotine, even before they become regular or daily smokers’,8 and considering a scenario in which standardised packaging reduced smoking uptake by 10%, a result that would amply justify the measure, this would result in a decrease in prevalence of 0.2%, down to 5.4%. According to the power calculations included in their revised working paper, even using the generous 90% confidence level employed by the authors, the study would fail to reliably detect any decline in prevalence that was less than 1.0 percentage point. (Table 2 and 3 in the revised working paper). In particular, it would be hardly more efficient than flipping a coin for the detection of the 0.2% decrease in prevalence mentioned above. Furthermore, the method used by the authors fails to address the question which has been posed. They should have performed a standard test to see whether the introduction of plain packaging has induced a change in the regression slope. Instead, the authors opted for an ad hoc analysis, perhaps realising that availability of only 12 data points after the intervention, and high instability of the observed prevalence due to the small sample size, would considerably reduce the power of the former approach. An even more serious issue is that the arguments of the authors rely entirely on their assumption of linearity, which is unfounded. They make the following observation, without further justification: ‘The individual deviations of the observed data from the fitted line are typically quite large … Globally, however, the fit of such a simple linear time trend is surprisingly good.’ (emphasis ours) They derive the appropriateness of their trend analysis from the fact that the ‘annual and overall sample sizes are big’. Actually, however, the average annual observed prevalence over the entire period (see figure 1), which we have independently computed, shows that the linearity assumption does not hold. For instance, the average prevalence observed in 2011 (8.66%) exceeds the upper limit of the 99.6% CI (8.32%), resulting in a p value of 0.05,i ii
i
The original working paper was dated March 2014. The authors later revised it, including a response to the critique of Laverty et al.2 The original working paper can be accessed at http://www.oxyromandie.ch/files/public/ docs/20140331-uzh-econwp149.pdf. ii We have actually used a confidence interval of 0.996062135=0.951/13, thus the p value of 0.05 for the 13-year period.
Diethelm P, et al. Tob Control 2015;24:e113–e115. doi:10.1136/tobaccocontrol-2014-051734
e113
Industry watch Figure 1 Observed average annual prevalence (squared dots linked by dashed line) versus expected annual prevalence based on the regression line (continuous line) and associated 99.6% CI (dotted line).
that is, under the linearity assumption, such an observation outside the CI range anywhere in the 13-year interval, would occur, on average, less than one time out of 20. This leads to the rejection of the linearity hypothesis. If one looks at the yearly relative change of observed annual prevalence (figure 2), the data used by the authors exhibit very large variations, in spite of the large annual sample sizes. We see a substantial prevalence increase from 2009 to 2011 (20.5%), and a huge decrease of 34.4% from 2011 to 2012. The magnitude of this last decrease, likely an artefact, shows that 2012 data are not a suitable basis for a comparative analysis of a trend in 2013. Furthermore, after such a huge decrease, one could normally expect to see some regression to the mean in 2013, that is, an increase. The fact that this did not happen is in itself remarkable. These results are, moreover, inconsistent with official survey data. A 2012 Australian governmental report concluded ‘The proportion of 12- to 15-year-old students who were current smokers in 2011 was the lowest since the survey series began.’9 (The previous survey was conducted in 2008.) It is thus reasonable to doubt the quality and reliability of the data used by the authors for their analysis of the effect of plain packaging in Australia. Given these serious limitations, it might be expected that the authors would include some caveats. Yet they did not, enabling
their tobacco industry sponsor to proclaim ‘researchers find no evidence plain packaging “experiment” has cut smoking’. Indeed, on the contrary, they insist that their analysis is very robust. In the conclusion, they argue that their approach, ‘if anything, is slightly biased in favour of finding a statistically significant (negative) effect of plain packaging on smoking prevalence of Australians aged 14 to 17 years’. Moreover, they are quoted in the press release by Philip Morris International as saying ‘We used statistical methodology that gave every possible leeway for detecting a possible plain packaging effect. Nevertheless, the data does not support any evidence of an actual effect of the Australian Plain Packaging Act on smoking prevalence of minors.’ (emphasis ours) The headlines that this paper has generated have thus been highly misleading. However, they are entirely in keeping with the narrative being advanced by the tobacco industry and those who speak on its behalf. Crucially, this example provides further justification for the decision of this journal10 (and others11 12) not to consider papers funded by the tobacco industry. These events also reflect poorly on the University of Zürich. In 2001, when the University of Geneva discovered that one of its professors had been secretly employed for decades by Philip Morris, it instructed its faculty members to refuse money from
Figure 2 Yearly evolution (in percent) of observed average annual prevalence.
e114
Diethelm P, et al. Tob Control 2015;24:e113–e115. doi:10.1136/tobaccocontrol-2014-051734
Industry watch the tobacco industry.13 14 The University of Zurich might wish to reflect on the risk of reputational damage of failing to do so.
5
Contributors The initial statistical analysis was done by PD. Both authors contributed equally to writing this paper.
6
Competing interests None.
7
Provenance and peer review Not commissioned; externally peer reviewed.
8 9
REFERENCES 1
2 3
4
Hatchard JL, Fooks GJ, Evans-Reeves KA, et al. A critical evaluation of the volume, relevance and quality of evidence submitted by the tobacco industry to oppose standardised packaging of tobacco products. BMJ Open 2014;4:e00375. Laverty AA, Watt HC, Arnott D, et al. Standardised packaging and tobacco-industry funded research. Lancet 2014;383:1384. Kaul A, Wolf M. The (Possible) effect of plain packaging on the smoking prevalence of minors in Australia: a trend analysis. Department of Economics, Working Paper No. 149. Zürich: University of Zürich, 2014. http://www.econ.uzh.ch/static/wp/ econwp149.pdf (accessed 10.04.2014). Philip Morris International. Researchers Find No Evidence Plain Packaging ‘Experiment’ Has Cut Smoking, Press release, 2014. http://www.pmi.com/eng/ media_center/Pages/plain_packaging_experiment.aspx (accessed 24.06.2014).
10 11 12 13 14
Diethelm P, et al. Tob Control 2015;24:e113–e115. doi:10.1136/tobaccocontrol-2014-051734
Johnston M. Plain packs derided as not working. The New Zealand Herald 01.04.2014. http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid= 11229823 (accessed 10.04.2014). Kassam R. New data proves plain pack cigarettes doesn’t dissuade young smokers. Breitbart.com, 27.03.2014. http://www.breitbart.com/Breitbart-London/2014/03/27/ Australian-data-proves-plain-packs-dont-work (accessed 10.04.2014). Comments on Kaul & Wolf “The ( possible) effect of plain packaging on the smoking prevalence of minors in Australia: a trend analysis”. Cancer Council Victoria, Melbourne, Australia, 26 March 2014. http://www.cancervic.org.au/ downloads/tobacco_control/2013/Cancer_Council_Victoria_comments_on_Kaul_ Wolf.pdf (accessed 25.05.2014). Mermelstein R. Teen smoking cessation. Tob Control 2003;12(Suppl I):i25–34. White V, Bariola E. Chapter 3. Tobacco use among Australian secondary students in 2011, in Australian secondary school students’ use of tobacco, alcohol, and over-the-counter and illicit substances in 2011. Canberra: Drug Strategy Branch Australian Government Department of Health and Ageing, 2012. http://www. nationaldrugstrategy.gov.au/internet/drugstrategy/Publishing.nsf/content/school11 Malone RE. Changing Tobacco Control’s policy on tobacco industry-funded research. Tob Control 2013;22:1–2. Godlee F, Malone R, Timmis A, et al. Journal policy on research funded by the tobacco industry. BMJ 2013;347:f5193. McKee M, Allebeck P. Why the European Journal of Public Health will no longer publish tobacco industry-supported research. Eur J Public Health 2014;24:182. Cohen JE. Universities and tobacco money. BMJ 2001:323:1–2. Hurst SA, Mauron A. A question of method—the ethics of managing conflicts of interest. EMBO Rep 2008;9:119–23.
e115
Copyright of Tobacco Control is the property of BMJ Publishing Group and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Tobacco industry-funded research on standardised packaging: there are none so blind as those who will not see! Pascal Diethelm,1 Martin McKee2 1
OxyRomandie, Geneva, Switzerland 2 London School of Hygiene and Tropical Medicine, London, UK Correspondence to Pascal Diethelm, OxyRomandie, 2, rue de la Fontaine, Geneva CH-1204, Switzerland; [email protected] Received 16 April 2014 Accepted 6 June 2014 Published Online First 7 July 2014
To cite: Diethelm P, McKee M. Tob Control 2015;24:e113–e115.
Support for standardised packaging of tobacco products seems to be reaching a tipping point. Australia has already adopted this measure. In New Zealand, legislation is in progress. Ireland and England have now committed to implementing standardised packs while many other countries, including Canada, Norway and Turkey are actively considering them. The tobacco industry, which has invested vast sums in the design of packs that appeal to new smokers, especially adolescents, is worried and, just as it did when bans on smoking in public places were being considered, is engaged in a wideranging and well-funded campaign to undermine the evidence.1 Laverty and colleagues warned about this, describing one such example using Australian data. They showed how the design of the study was such that it would have been virtually impossible to detect a significant effect.2 However, as we will now show, this was not an isolated incident. Their warning would seem to be justified by a working paper published on the website of the Department of Economics of the University of Zürich, entitled ‘The (Possible) Effect of Plain Packaging on the Smoking Prevalence of Minors in Australia: A Trend Analysis’,3 funded by Philip Morris International. Its authors also conclude that there is no evidence that standardised packaging works and Philip Morris has press released it,4 generating headlines such as ‘Plain packs derided as not working’5 and ‘New data proves plain pack cigarettes doesn’t dissuade young smokers’.6 However, as we shall show, once again, it was almost inevitable that the data and methods used would fail to detect any expected effect. The study looks at the ‘prevalence of smoking among Australians aged 14–17 years’, taking monthly prevalence estimates from a marketing survey, of which the authors give no further detail, except that it contains a ‘0–1 “smoker” variable’ derived from personal interviews, a variable which is reported elsewhere to be inappropriate for assessing adolescent smoking.7 Yet, the purpose of standardised packaging is, as the authors acknowledge, to discourage smoking initiation and encourage quitting. Measures of prevalence cannot distinguish those who took up smoking after standardised packaging was introduced from those who took it up previously, with the former likely to comprise only between one-quarter and one-third of the age group. The data used in the study show that the prevalence of smoking in Australia in the 14–17 years age group was 5.6% in 2012. The lack of power of the study is then apparent and has been documented by others.2 7 If it is assumed that the introduction of
standardised packaging would have negligible effect on quitting rates in the 14–17 years age group, a reasonable assumption, as few adolescents readily stop smoking, as ‘they are dependent on nicotine, even before they become regular or daily smokers’,8 and considering a scenario in which standardised packaging reduced smoking uptake by 10%, a result that would amply justify the measure, this would result in a decrease in prevalence of 0.2%, down to 5.4%. According to the power calculations included in their revised working paper, even using the generous 90% confidence level employed by the authors, the study would fail to reliably detect any decline in prevalence that was less than 1.0 percentage point. (Table 2 and 3 in the revised working paper). In particular, it would be hardly more efficient than flipping a coin for the detection of the 0.2% decrease in prevalence mentioned above. Furthermore, the method used by the authors fails to address the question which has been posed. They should have performed a standard test to see whether the introduction of plain packaging has induced a change in the regression slope. Instead, the authors opted for an ad hoc analysis, perhaps realising that availability of only 12 data points after the intervention, and high instability of the observed prevalence due to the small sample size, would considerably reduce the power of the former approach. An even more serious issue is that the arguments of the authors rely entirely on their assumption of linearity, which is unfounded. They make the following observation, without further justification: ‘The individual deviations of the observed data from the fitted line are typically quite large … Globally, however, the fit of such a simple linear time trend is surprisingly good.’ (emphasis ours) They derive the appropriateness of their trend analysis from the fact that the ‘annual and overall sample sizes are big’. Actually, however, the average annual observed prevalence over the entire period (see figure 1), which we have independently computed, shows that the linearity assumption does not hold. For instance, the average prevalence observed in 2011 (8.66%) exceeds the upper limit of the 99.6% CI (8.32%), resulting in a p value of 0.05,i ii
i
The original working paper was dated March 2014. The authors later revised it, including a response to the critique of Laverty et al.2 The original working paper can be accessed at http://www.oxyromandie.ch/files/public/ docs/20140331-uzh-econwp149.pdf. ii We have actually used a confidence interval of 0.996062135=0.951/13, thus the p value of 0.05 for the 13-year period.
Diethelm P, et al. Tob Control 2015;24:e113–e115. doi:10.1136/tobaccocontrol-2014-051734
e113
Industry watch Figure 1 Observed average annual prevalence (squared dots linked by dashed line) versus expected annual prevalence based on the regression line (continuous line) and associated 99.6% CI (dotted line).
that is, under the linearity assumption, such an observation outside the CI range anywhere in the 13-year interval, would occur, on average, less than one time out of 20. This leads to the rejection of the linearity hypothesis. If one looks at the yearly relative change of observed annual prevalence (figure 2), the data used by the authors exhibit very large variations, in spite of the large annual sample sizes. We see a substantial prevalence increase from 2009 to 2011 (20.5%), and a huge decrease of 34.4% from 2011 to 2012. The magnitude of this last decrease, likely an artefact, shows that 2012 data are not a suitable basis for a comparative analysis of a trend in 2013. Furthermore, after such a huge decrease, one could normally expect to see some regression to the mean in 2013, that is, an increase. The fact that this did not happen is in itself remarkable. These results are, moreover, inconsistent with official survey data. A 2012 Australian governmental report concluded ‘The proportion of 12- to 15-year-old students who were current smokers in 2011 was the lowest since the survey series began.’9 (The previous survey was conducted in 2008.) It is thus reasonable to doubt the quality and reliability of the data used by the authors for their analysis of the effect of plain packaging in Australia. Given these serious limitations, it might be expected that the authors would include some caveats. Yet they did not, enabling
their tobacco industry sponsor to proclaim ‘researchers find no evidence plain packaging “experiment” has cut smoking’. Indeed, on the contrary, they insist that their analysis is very robust. In the conclusion, they argue that their approach, ‘if anything, is slightly biased in favour of finding a statistically significant (negative) effect of plain packaging on smoking prevalence of Australians aged 14 to 17 years’. Moreover, they are quoted in the press release by Philip Morris International as saying ‘We used statistical methodology that gave every possible leeway for detecting a possible plain packaging effect. Nevertheless, the data does not support any evidence of an actual effect of the Australian Plain Packaging Act on smoking prevalence of minors.’ (emphasis ours) The headlines that this paper has generated have thus been highly misleading. However, they are entirely in keeping with the narrative being advanced by the tobacco industry and those who speak on its behalf. Crucially, this example provides further justification for the decision of this journal10 (and others11 12) not to consider papers funded by the tobacco industry. These events also reflect poorly on the University of Zürich. In 2001, when the University of Geneva discovered that one of its professors had been secretly employed for decades by Philip Morris, it instructed its faculty members to refuse money from
Figure 2 Yearly evolution (in percent) of observed average annual prevalence.
e114
Diethelm P, et al. Tob Control 2015;24:e113–e115. doi:10.1136/tobaccocontrol-2014-051734
Industry watch the tobacco industry.13 14 The University of Zurich might wish to reflect on the risk of reputational damage of failing to do so.
5
Contributors The initial statistical analysis was done by PD. Both authors contributed equally to writing this paper.
6
Competing interests None.
7
Provenance and peer review Not commissioned; externally peer reviewed.
8 9
REFERENCES 1
2 3
4
Hatchard JL, Fooks GJ, Evans-Reeves KA, et al. A critical evaluation of the volume, relevance and quality of evidence submitted by the tobacco industry to oppose standardised packaging of tobacco products. BMJ Open 2014;4:e00375. Laverty AA, Watt HC, Arnott D, et al. Standardised packaging and tobacco-industry funded research. Lancet 2014;383:1384. Kaul A, Wolf M. The (Possible) effect of plain packaging on the smoking prevalence of minors in Australia: a trend analysis. Department of Economics, Working Paper No. 149. Zürich: University of Zürich, 2014. http://www.econ.uzh.ch/static/wp/ econwp149.pdf (accessed 10.04.2014). Philip Morris International. Researchers Find No Evidence Plain Packaging ‘Experiment’ Has Cut Smoking, Press release, 2014. http://www.pmi.com/eng/ media_center/Pages/plain_packaging_experiment.aspx (accessed 24.06.2014).
10 11 12 13 14
Diethelm P, et al. Tob Control 2015;24:e113–e115. doi:10.1136/tobaccocontrol-2014-051734
Johnston M. Plain packs derided as not working. The New Zealand Herald 01.04.2014. http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid= 11229823 (accessed 10.04.2014). Kassam R. New data proves plain pack cigarettes doesn’t dissuade young smokers. Breitbart.com, 27.03.2014. http://www.breitbart.com/Breitbart-London/2014/03/27/ Australian-data-proves-plain-packs-dont-work (accessed 10.04.2014). Comments on Kaul & Wolf “The ( possible) effect of plain packaging on the smoking prevalence of minors in Australia: a trend analysis”. Cancer Council Victoria, Melbourne, Australia, 26 March 2014. http://www.cancervic.org.au/ downloads/tobacco_control/2013/Cancer_Council_Victoria_comments_on_Kaul_ Wolf.pdf (accessed 25.05.2014). Mermelstein R. Teen smoking cessation. Tob Control 2003;12(Suppl I):i25–34. White V, Bariola E. Chapter 3. Tobacco use among Australian secondary students in 2011, in Australian secondary school students’ use of tobacco, alcohol, and over-the-counter and illicit substances in 2011. Canberra: Drug Strategy Branch Australian Government Department of Health and Ageing, 2012. http://www. nationaldrugstrategy.gov.au/internet/drugstrategy/Publishing.nsf/content/school11 Malone RE. Changing Tobacco Control’s policy on tobacco industry-funded research. Tob Control 2013;22:1–2. Godlee F, Malone R, Timmis A, et al. Journal policy on research funded by the tobacco industry. BMJ 2013;347:f5193. McKee M, Allebeck P. Why the European Journal of Public Health will no longer publish tobacco industry-supported research. Eur J Public Health 2014;24:182. Cohen JE. Universities and tobacco money. BMJ 2001:323:1–2. Hurst SA, Mauron A. A question of method—the ethics of managing conflicts of interest. EMBO Rep 2008;9:119–23.
e115
Copyright of Tobacco Control is the property of BMJ Publishing Group and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
