Letters to the Editor / Journal of Hospital Infection 89 (2015) 136e139 2. Dubberke ER, Olsen MA. Burden of Clostridium difficile on the healthcare system. Clin Infect Dis 2012;55(Suppl. 2):S88eS92. 3. Public Health England. Published Clostridium difficile data tables April 2007 to March 2013. London: Public Health England; 2013. 4. Patient Safety Domain. Clostridium difficile infection objectives for NHS organisations in 2014/15 and guidance on sanction implementation. London: NHS England; 2014.
S.F. Hill Department of Medical Microbiology, Poole Hospital NHS Foundation Trust, Longfleet Road, Poole BH15 2JB, UK E-mail address: [email protected] Available online 11 December 2014 http://dx.doi.org/10.1016/j.jhin.2014.11.002 ª 2014 The Healthcare Infection Society. Published by Elsevier Ltd. All rights reserved.
Response to K. Page et al., ‘Costing the Australian National Hand Hygiene Initiative’ Madam, The recent article by Page et al. represents an important costing initiative.1 However, I have concerns regarding the methodology that may impact on the validity of some of their findings. First, hospital costs e reportedly accounting for 65% of all programme costs e were estimated retrospectively using a self-administered online survey of infection control practitioners (ICPs). Survey reliability was determined through pilot testing involving only three individuals. Pilot testing results are not described and the survey tool is not provided. No attempt was made to validate the estimates provided by the 38 (76%) responding hospitals. Importantly, these responses underpin much of the costing estimates, especially regarding time spent auditing hand hygiene compliance. Although the authors stated that ‘Validity for time spent auditing was established by cross-checking time estimates with external records from HHA [Hand Hygiene Australia].’; no data were shown. Indeed, HHA is not aware of such data and consequently suggested a prospective standardized assessment of auditing durations to provide real information on this important parameter. Although the authors attempted to control for ‘over-confidence’ in time estimates, they provided no justification for using this method.2 The authors reported that the best estimate of average time spent by ICPs on the National Hand Hygiene Initiative (NHHI) was w5 h per week and that this was ‘. validated during a steering group meeting with a comment from the director of HHA that this seemed ‘about right’’ ‒ this hardly represents scientific validation and misquotes the Director (M.L.G.; who was not contacted to verify the citation). The problems associated with the current data are potentially exacerbated by their extrapolation to the remaining 12 (24%) hospitals that did not respond but which were included in the study.
137
Second, the fact that Queensland-only data were used to estimate the marginal costs of extra alcohol-based hand rubs (ABHRs) associated with the NHHI is a concern. Since ABHR purchasing contracts are state-specific and the degree of each state’s engagement in promoting ABHR prior to the NHHI varied, additional ABHR costs in each state may be quite different. Similarly, the proposal to assume 10‒30% variance in ABHR use is non-validated and should therefore also be considered a potential limitation. Third, the stated running costs for HHA of AU$1.2 million for 2011‒2012 did not represent maintenance phase for the NHHI, since HHA was still recruiting and supporting new sites e hence this funding instead represented a combination of start-up and maintenance costs. During 2011‒2012, while many public hospitals in Australia had joined the NHHI, others were still joining (the numbers of public hospitals participating in 2009, 2010, 2011, 2012, and 2013 were 262, 428, 454, 493, and 504, respectively); furthermore, HHA was also responsible for supporting private hospitals to join the NHHI (the numbers of private hospitals participating in 2009‒2013 were 28, 93, 135, 163, and 248, respectively).3 In 2012‒2013, when HHA was considered to be transitioning to maintenance phase, the annual HHA budget was AU$600,176. One also needs to question the authors’ assumption that the distribution of HHA time/ effort (and therefore budget) should be based on bed numbers rather than on the number of hospitals participating in the NHHI. Although the 50 hospitals studied included 42% of the public hospital beds, many of these organizations had large, well-organized infection control units that required far less input from HHA than smaller, often geographically dispersed, sites. While these data will inform cost-effectiveness analyses of the NHHI, one needs to consider that the time horizon selected for such an analysis is crucial. In contrast to patient-based therapeutic programmes, the NHHI is a culture change programme designed to embed improved hand hygiene within routine Australian clinical care over a longer time period than analysed by Page et al., with benefits reasonably expected to be long-standing.4 While Page et al. provide interesting material, the study has potential methodological and statistical issues which may limit its utility. Conflict of interest statement The author is Director of Hand Hygiene Australia, the body responsible for conducting the National Hand Hygiene Initiative. Funding sources The Australian Commission on Quality and Safety in Health Care.
References 1. Page K, Barnett AG, Campbell M, et al. Costing the Australian National Hand Hygiene Initiative. J Hosp Infect 2014;88:141e148. 2. Speirs-Bridge A, Fidler F, McBride M, Flander L, Cumming G, Burgman M. Reducing overconfidence in the interval judgments of experts. Risk Anal 2010;30:512e523. 3. Hand Hygiene Australia. Latest National Data. 2014; http://www. hha.org.au/LatestNationalData.aspx (accessed 24.09.14). 4. Pittet D, Hugonnet S, Harbarth S, et al. Effectiveness of a hospitalwide programme to improve compliance with hand hygiene. Infection Control Programme. Lancet 2000;356:1307e1312.
138
Letters to the Editor / Journal of Hospital Infection 89 (2015) 136e139 a b
M.L. Graysona,b,c,d,* Hand Hygiene Australia, Australia
Infectious Diseases and Microbiology Department, Austin Health, Melbourne, Australia
c
Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia d
Department of Medicine, University of Melbourne, Melbourne, Australia
* Address: Infectious Diseases Department, Austin Health, 145 Studley Rd, Heidelberg 3084, Victoria, Australia. Tel.: þ61 3 9496 6676; fax: þ61 3 9496 6677. E-mail address: [email protected] Available online 11 December 2014 http://dx.doi.org/10.1016/j.jhin.2014.10.009 ª 2014 The Healthcare Infection Society. Published by Elsevier Ltd. All rights reserved.
Response to Grayson’s Letter to the Editor: ‘Response to K. Page et al., ‘Costing the Australian National Hand Hygiene Initiative’’ Madam, Grayson, in his recent letter to the editor, highlighted that the study by Page et al. represents an important costing initiative, and we agree.1 However, he went on to suggest some possible issues about the data and assumptions in the study. Grayson was both the director of Hand Hygiene Australia (HHA), the body responsible for the implementation and running of the National Hand Hygiene Initiative (NHHI), and a Chief Investigator on the National Health and Medical Research Council (NHMRC)-funded grant used to assess the costeffectiveness of the NHHI. Part of the governance requirements for this research grant was to have a steering committee to oversee the research methods and dissemination. Grayson attended the steering committee meetings and was a helpful member of the research team for the period prior to the results being known. He oversaw and assessed the methods and is co-author on a publication about the research methods.2 Grayson raises concerns about hospital costs being retrospective and collected by a self-administered online survey of infection control practitioners (ICPs). As stated in the paper on p. 147, the cost-effectiveness evaluation was commissioned after the NHHI implementation had begun, making a prospective study impossible. We acknowledge the usual limitations of retrospective data but maintain that the best method was used in the circumstances. We published a peer-reviewed paper on
DOI of original article: http://dx.doi.org/10.1016/j.jhin.2014.10. 009.
the costing methods, consulted with the steering group, and included their feedback.3 Grayson commented on the survey reliability and the pilot testing, stressing that a copy of the survey tool was not included in the published paper. The pilot testing included obtaining detailed responses from three ICPs to check both the content validity of the questions, and the consistency of responses over time. The survey and results have since been shared with the ICPs and presented at seminars in most states and territories. The researchers are happy to share the survey with other individual researchers upon request. Grayson writes ‘No attempt was made to validate the estimates provided by the 38 (76%) responding hospitals.’ This statement is not correct, and the research team went to considerable lengths to validate the results. After each ICP had completed the survey, we held a telephone interview with them individually to review and check their responses. Particular attention was paid to the time estimates attributed to hand hygiene activities. The ICPs are the subject-matter experts and went to great lengths to obtain relevant information, often verifying this with colleagues and written records. We acknowledge that the estimates are uncertain, but this uncertainty is included in the results. Grayson maintains that he was misquoted in his support of the finding that w5 hours per week was spent working on the NHHI by ICPs. This estimate was presented at two separate steering committee meetings and there was both a general consensus, and level of agreement, that this time estimate was plausible. At no point prior to the results being known did Grayson raise concerns about this estimate. Grayson states that: ‘The problems associated with the current data are potentially exacerbated by their extrapolation to the remaining 12 (24%) hospitals that did not respond but which were included in the study.’ It is not clear why these issues would be exacerbated by extrapolation and Grayson does not provide any argument for why this would be the case. We have no reason to believe that the hospitals that did not respond specifically to the cost survey are systematically different from the ones that did respond. Grayson raises concerns about the alcohol-based hand rub (ABHR) costs being calculated from Queensland data. We agree that the marginal ABHR costs in each state and territory may be different, but this is a minor component (6%) relative to the other large costs incurred by the NHHI. We incorporate uncertainty into the ABHR estimates and this is reported in the paper. It is possible that the true value for ABHR lies outside of this range, but its impact on the conclusions of this study will be negligible. Grayson disputes that the running costs for HHA of AU$1.2 million for 2011‒2012 represent only the maintenance phase of the NHHI. He asserts instead that the HHA budget was AU$600,176 for the maintenance period. These represent the values presented to us by HHA on their running costs for this year, and they were subsequently agreed upon during a steering committee meeting. It is worth noting that the AU$1.2 million is our maximum estimate and the value of AU$600,176 is captured by our minimum estimate (AU$519,133) which we set on the annual HHA running costs (see Table I in Page et al.1). Moreover, we maintain that the majority of these costs are fixed salaries and hence will not change much with the number of hospitals being supported. It is for this, and similar, reasons that we were careful to include uncertainty in all estimates.
S.F. Hill Department of Medical Microbiology, Poole Hospital NHS Foundation Trust, Longfleet Road, Poole BH15 2JB, UK E-mail address: [email protected] Available online 11 December 2014 http://dx.doi.org/10.1016/j.jhin.2014.11.002 ª 2014 The Healthcare Infection Society. Published by Elsevier Ltd. All rights reserved.
Response to K. Page et al., ‘Costing the Australian National Hand Hygiene Initiative’ Madam, The recent article by Page et al. represents an important costing initiative.1 However, I have concerns regarding the methodology that may impact on the validity of some of their findings. First, hospital costs e reportedly accounting for 65% of all programme costs e were estimated retrospectively using a self-administered online survey of infection control practitioners (ICPs). Survey reliability was determined through pilot testing involving only three individuals. Pilot testing results are not described and the survey tool is not provided. No attempt was made to validate the estimates provided by the 38 (76%) responding hospitals. Importantly, these responses underpin much of the costing estimates, especially regarding time spent auditing hand hygiene compliance. Although the authors stated that ‘Validity for time spent auditing was established by cross-checking time estimates with external records from HHA [Hand Hygiene Australia].’; no data were shown. Indeed, HHA is not aware of such data and consequently suggested a prospective standardized assessment of auditing durations to provide real information on this important parameter. Although the authors attempted to control for ‘over-confidence’ in time estimates, they provided no justification for using this method.2 The authors reported that the best estimate of average time spent by ICPs on the National Hand Hygiene Initiative (NHHI) was w5 h per week and that this was ‘. validated during a steering group meeting with a comment from the director of HHA that this seemed ‘about right’’ ‒ this hardly represents scientific validation and misquotes the Director (M.L.G.; who was not contacted to verify the citation). The problems associated with the current data are potentially exacerbated by their extrapolation to the remaining 12 (24%) hospitals that did not respond but which were included in the study.
137
Second, the fact that Queensland-only data were used to estimate the marginal costs of extra alcohol-based hand rubs (ABHRs) associated with the NHHI is a concern. Since ABHR purchasing contracts are state-specific and the degree of each state’s engagement in promoting ABHR prior to the NHHI varied, additional ABHR costs in each state may be quite different. Similarly, the proposal to assume 10‒30% variance in ABHR use is non-validated and should therefore also be considered a potential limitation. Third, the stated running costs for HHA of AU$1.2 million for 2011‒2012 did not represent maintenance phase for the NHHI, since HHA was still recruiting and supporting new sites e hence this funding instead represented a combination of start-up and maintenance costs. During 2011‒2012, while many public hospitals in Australia had joined the NHHI, others were still joining (the numbers of public hospitals participating in 2009, 2010, 2011, 2012, and 2013 were 262, 428, 454, 493, and 504, respectively); furthermore, HHA was also responsible for supporting private hospitals to join the NHHI (the numbers of private hospitals participating in 2009‒2013 were 28, 93, 135, 163, and 248, respectively).3 In 2012‒2013, when HHA was considered to be transitioning to maintenance phase, the annual HHA budget was AU$600,176. One also needs to question the authors’ assumption that the distribution of HHA time/ effort (and therefore budget) should be based on bed numbers rather than on the number of hospitals participating in the NHHI. Although the 50 hospitals studied included 42% of the public hospital beds, many of these organizations had large, well-organized infection control units that required far less input from HHA than smaller, often geographically dispersed, sites. While these data will inform cost-effectiveness analyses of the NHHI, one needs to consider that the time horizon selected for such an analysis is crucial. In contrast to patient-based therapeutic programmes, the NHHI is a culture change programme designed to embed improved hand hygiene within routine Australian clinical care over a longer time period than analysed by Page et al., with benefits reasonably expected to be long-standing.4 While Page et al. provide interesting material, the study has potential methodological and statistical issues which may limit its utility. Conflict of interest statement The author is Director of Hand Hygiene Australia, the body responsible for conducting the National Hand Hygiene Initiative. Funding sources The Australian Commission on Quality and Safety in Health Care.
References 1. Page K, Barnett AG, Campbell M, et al. Costing the Australian National Hand Hygiene Initiative. J Hosp Infect 2014;88:141e148. 2. Speirs-Bridge A, Fidler F, McBride M, Flander L, Cumming G, Burgman M. Reducing overconfidence in the interval judgments of experts. Risk Anal 2010;30:512e523. 3. Hand Hygiene Australia. Latest National Data. 2014; http://www. hha.org.au/LatestNationalData.aspx (accessed 24.09.14). 4. Pittet D, Hugonnet S, Harbarth S, et al. Effectiveness of a hospitalwide programme to improve compliance with hand hygiene. Infection Control Programme. Lancet 2000;356:1307e1312.
138
Letters to the Editor / Journal of Hospital Infection 89 (2015) 136e139 a b
M.L. Graysona,b,c,d,* Hand Hygiene Australia, Australia
Infectious Diseases and Microbiology Department, Austin Health, Melbourne, Australia
c
Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia d
Department of Medicine, University of Melbourne, Melbourne, Australia
* Address: Infectious Diseases Department, Austin Health, 145 Studley Rd, Heidelberg 3084, Victoria, Australia. Tel.: þ61 3 9496 6676; fax: þ61 3 9496 6677. E-mail address: [email protected] Available online 11 December 2014 http://dx.doi.org/10.1016/j.jhin.2014.10.009 ª 2014 The Healthcare Infection Society. Published by Elsevier Ltd. All rights reserved.
Response to Grayson’s Letter to the Editor: ‘Response to K. Page et al., ‘Costing the Australian National Hand Hygiene Initiative’’ Madam, Grayson, in his recent letter to the editor, highlighted that the study by Page et al. represents an important costing initiative, and we agree.1 However, he went on to suggest some possible issues about the data and assumptions in the study. Grayson was both the director of Hand Hygiene Australia (HHA), the body responsible for the implementation and running of the National Hand Hygiene Initiative (NHHI), and a Chief Investigator on the National Health and Medical Research Council (NHMRC)-funded grant used to assess the costeffectiveness of the NHHI. Part of the governance requirements for this research grant was to have a steering committee to oversee the research methods and dissemination. Grayson attended the steering committee meetings and was a helpful member of the research team for the period prior to the results being known. He oversaw and assessed the methods and is co-author on a publication about the research methods.2 Grayson raises concerns about hospital costs being retrospective and collected by a self-administered online survey of infection control practitioners (ICPs). As stated in the paper on p. 147, the cost-effectiveness evaluation was commissioned after the NHHI implementation had begun, making a prospective study impossible. We acknowledge the usual limitations of retrospective data but maintain that the best method was used in the circumstances. We published a peer-reviewed paper on
DOI of original article: http://dx.doi.org/10.1016/j.jhin.2014.10. 009.
the costing methods, consulted with the steering group, and included their feedback.3 Grayson commented on the survey reliability and the pilot testing, stressing that a copy of the survey tool was not included in the published paper. The pilot testing included obtaining detailed responses from three ICPs to check both the content validity of the questions, and the consistency of responses over time. The survey and results have since been shared with the ICPs and presented at seminars in most states and territories. The researchers are happy to share the survey with other individual researchers upon request. Grayson writes ‘No attempt was made to validate the estimates provided by the 38 (76%) responding hospitals.’ This statement is not correct, and the research team went to considerable lengths to validate the results. After each ICP had completed the survey, we held a telephone interview with them individually to review and check their responses. Particular attention was paid to the time estimates attributed to hand hygiene activities. The ICPs are the subject-matter experts and went to great lengths to obtain relevant information, often verifying this with colleagues and written records. We acknowledge that the estimates are uncertain, but this uncertainty is included in the results. Grayson maintains that he was misquoted in his support of the finding that w5 hours per week was spent working on the NHHI by ICPs. This estimate was presented at two separate steering committee meetings and there was both a general consensus, and level of agreement, that this time estimate was plausible. At no point prior to the results being known did Grayson raise concerns about this estimate. Grayson states that: ‘The problems associated with the current data are potentially exacerbated by their extrapolation to the remaining 12 (24%) hospitals that did not respond but which were included in the study.’ It is not clear why these issues would be exacerbated by extrapolation and Grayson does not provide any argument for why this would be the case. We have no reason to believe that the hospitals that did not respond specifically to the cost survey are systematically different from the ones that did respond. Grayson raises concerns about the alcohol-based hand rub (ABHR) costs being calculated from Queensland data. We agree that the marginal ABHR costs in each state and territory may be different, but this is a minor component (6%) relative to the other large costs incurred by the NHHI. We incorporate uncertainty into the ABHR estimates and this is reported in the paper. It is possible that the true value for ABHR lies outside of this range, but its impact on the conclusions of this study will be negligible. Grayson disputes that the running costs for HHA of AU$1.2 million for 2011‒2012 represent only the maintenance phase of the NHHI. He asserts instead that the HHA budget was AU$600,176 for the maintenance period. These represent the values presented to us by HHA on their running costs for this year, and they were subsequently agreed upon during a steering committee meeting. It is worth noting that the AU$1.2 million is our maximum estimate and the value of AU$600,176 is captured by our minimum estimate (AU$519,133) which we set on the annual HHA running costs (see Table I in Page et al.1). Moreover, we maintain that the majority of these costs are fixed salaries and hence will not change much with the number of hospitals being supported. It is for this, and similar, reasons that we were careful to include uncertainty in all estimates.
