G Model

ARTICLE IN PRESS

JVAC 15379 1–8

Vaccine xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

Vaccine journal homepage: www.elsevier.com/locate/vaccine

Review

1

Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination?

2

3

4 5 6 7 8

Q1

Kevin Marsh a , Ruth Chapman a , Rebecca F. Baggaley b , Nathalie Largeron c,∗ , Xavier Bresse c a

Evidera, 26-28 Hammersmith Grove, London W6 7HA, United Kingdom London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E HT, United Kingdom c Sanofi Pasteur MSD, 8, rue Jonas Salk, 69367 Lyon Cedex 07, France b

9

10 23

a r t i c l e

i n f o

a b s t r a c t

11 12 13 14 15 16

Article history: Received 31 January 2014 Received in revised form 24 April 2014 Accepted 1 May 2014 Available online xxx

17

22

Keywords: Human papilloma virus Vaccine Cost-effectiveness Health economics

24

1. Introduction

18 19 20 21

25 26 27 28 29 30

Background: Since the original licensing of human papilloma virus (HPV) vaccination for women, evidence is accumulating of its effectiveness in preventing HPV-related conditions in men, and universal vaccination (vaccinating men and women) is now recommended in some countries. Several models of the cost-effectiveness of universal HPV vaccination have been published, but results have been mixed. This article assesses the extent to which economic studies have captured the range of values associated with universal HPV vaccination, and how this influences estimates of its cost-effectiveness. Methods: Eight published economic evaluations of universal HPV vaccination were reviewed to identify which of the values associated with universal HPV vaccination were included in each analysis. Results: Studies of the cost-effectiveness of universal HPV vaccination capture only a fraction of the values generated. Most studies focused on impacts on health and health system cost, and only captured these partially. A range of values is excluded from most studies, including impacts on productivity, patient time and costs, carers and family costs, and broader social values such as the right to access treatment. Further, those studies that attempted to capture these values only did so partially. Discussion: Decisions to invest in universal HPV vaccination need to be based on a complete assessment of the value that it generates. This is not provided by existing economic evaluations. Further work is required to understand this value. First, research is required to understand how HPV-related health outcomes impact on society including, for instance, their impact on productivity. Second, consideration should be given to alternative approaches to capture this broader set of values in a manner useful to decisions-makers, such as multi-criteria decision analysis. © 2014 Published by Elsevier Ltd.

Human papilloma virus (HPV) imposes a significant social burden. Strains of HPV are the cause of cervical cancer and intraepithelial neoplasia (IN) [1,2] and primary causes of anal [3] and vaginal [3] cancer and external genital warts (EGW) [4,5] and recurrent respiratory papillomatosis (RRP) [6]. They also contribute to a substantial proportion of penile [7,8], vulval [3,9] and

Abbreviations: (HPV), Human papilloma virus; (IN), Intraepithelial neoplasia; (EGW), External genital warts; (RRP), Recurrent respiratory papillomatosis; (MSM), Men-who-have-sex-with-men; (ECDC), European Centre for Disease Prevention and Control; (HRQoL), Health-related quality of life; (CUA), Cost utility analysis; (QALY), Quality adjusted life year; (ICER), Incremental cost effectiveness ratio; (CBA), Costbenefit analysis; (MCDA), Multi-criteria decision analysis. ∗ Corresponding author. Tel.: +33 4 37 28 4571. E-mail address: [email protected] (N. Largeron).

head-and-neck cancers (principally oral cavity [10], oropharyngeal [11,12] and laryngeal [10] cancers). HPV-related pre-cancerous lesions and cancers and their treatment have a number of sequelae, including: reduced fertility and sexual dysfunction [13–15]; risk of pregnancy complications due to conisation procedures for treatment of cervical IN [16,17]; anxiety about the risk of recurrence of cancer or neoplasias [18–20]; the psychological impact of surgery for head-and-neck cancers that results in change in appearance, speech impairment or macular dysfunction [21]; changes in sexual function and the general distress associated with cancer treatment [22–26]; and anxiety associated with the stigma of being affected by a sexually transmitted infection [27], reduced fertility [28], or physical changes such as colostomy procedure [29]. There are two vaccines against HPV. Cervarix® is designed to prevent infection from HPV types 16 and 18; and the quadrivalent Gardasil® HPV vaccine is active against HPV types 6, 11, 16 and

http://dx.doi.org/10.1016/j.vaccine.2014.05.007 0264-410X/© 2014 Published by Elsevier Ltd.

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

G Model

ARTICLE IN PRESS

JVAC 15379 1–8

Health equalities

Non-health benefits

HRQoLa

Treatment cost

K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

2

Patient Individual

Family/ carers Other

Government Social a

HRQoL – Health-related quality of life.

Fig. 1. Benefit of universal HPV vaccination to different stakeholder groups (green shading indicates the value types generated).

48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93

18. Gardasil was originally licensed for use among young females for protection against cervical cancer, intraepithelial neoplasia (IN) and EGW [30–33], but since its original licensing it has been demonstrated to also be effective in preventing anal HPV infection and anal IN in both genders, and EGW in men. Universal HPV vaccination (vaccinating males as well as females) thus has numerous benefits. Men directly benefit from protection against a range of HPV-related conditions. Approximately 30% of all new cancer cases attributable to HPV16/18 that occur annually in Europe are estimated to occur in men [34], and unlike cervical cancer there is no screening for early detection of non-cervical HPV-related cancers. Universal vaccination would further prevent cervical cancer and other HPV-related conditions in women indirectly, via herdprotection. HPV infection control through the vaccination of just women would rely on high vaccine coverage and no transmission within men-who-have-sex-with-men (MSM) populations. The high infectiousness of HPV strains means that they are highly unlikely to be eliminated at vaccine coverage levels achievable through the vaccination of girls [35]. So for men to benefit from vaccine protection, universal vaccination, rather than reliance on the benefits of herd-protection, is required. The benefits of universal HPV vaccination recently led Australia to become the first country to offer free HPV vaccines for boys as well as girls [36]. Canada and the US now also recommend vaccinating boys [37–39], although uptake has been slow [40,41]. Austria recently became the first European Union country to include boys in its national HPV vaccination programme [42]. However, questions about the economic value of universal vaccination mean that most European countries do not currently include men in HPV vaccination programmes, instead focusing on expanding coverage among women [43]. Existing economic analysis has demonstrated that HPV vaccination of females is cost-effective [44], but is less conclusive about the cost-effectiveness of universal HPV vaccination [45]. The European Centre for Disease Prevention and Control (ECDC) reviewed this evidence and concluded that, despite the clinical benefits for males of HPV vaccination, including males in current HPV vaccination programmes may not be cost-effective when high coverage of females is achieved [43]. There are, however, a number of concerns with the conclusion of these economic studies. They have been criticised for underestimating the benefits of universal vaccination [46,47]. These values are multiple, including: improvements in health and non-health outcomes, avoided costs, and improved health equalities (Fig. 1). A complete estimate of the value generated by universal HPV vaccination requires that these values are captured. A complete picture

of the health related quality of life (HRQoL) impact of universal vaccination requires models to include: • All the HPV-related diseases potentially avoided as a result of vaccination (see above). This requires modelling of the underlying HPV transmission process. There are many methodological challenges associated with assessing number of infections prevented by HPV vaccination that are discussed in detail elsewhere [48–50], but crucial is a realistic simulation of the effects of herdprotection. • All the HRQoL impacts of avoiding these diseases. Cost utility analysis (CUA) often estimates HRQoL impact using the quality-adjusted life year (QALY), employing instruments such as the EQ-5D, a standardised instrument for use as a measure of health outcome [51]. The EQ-5D measures health across five domains: mobility, capacity for self-care, conduct of usual activities, pain/discomfort and anxiety/depression. It is argued that these domains exclude some important health impacts of cancer, such as fatigue, and this may underestimate the valuation of HPV vaccination [52]. • The HRQoL gains to carers and family members, and society more broadly. As a preventative public health intervention, the benefits of universal vaccination may go beyond the direct individual health effects. The impact on carers is particularly relevant for HPV-related cancers, which can have a significant effect on long-term HRQoL for carers and family members. Universal vaccination also provides value to those for whom it provides peace of mind from knowing that they (and/or their families) have a reduced risk of illness–referred to as ‘utility-in-anticipation’ [47,48]. The value of avoided HPV-related diseases also includes the corresponding avoidance of the costs associated with the management of these diseases. The costs of the health impacts of HPV include health system costs, social services costs, charitably-funded counselling and support services, private practitioner costs, and out-of-pocket expenses such as adult diapers for anal incontinence following anal cancer treatment. The assessment of the cost-effectiveness of universal HPV vaccination also requires an estimate of the cost of vaccination. However, assessments may overestimate the cost of universal HPV vaccination [50] as list price is often used in models, while actual vaccine price paid will be lower than this. Several non-health impacts are also generated by vaccination [46,47]. These include economic benefits, such as improved patient, family and carer productivity (i.e., reduced unemployment, work absenteeism and presenteeism [reduced quality of work]). There are, however, also non-economic benefits of vaccination, such as improved health equalities. Equality in health pertains to the notion that everyone has a fair chance of a full and healthy life [53], something that is often built into the objective of healthcare systems. Judgements about what is equitable have a number of bases (see the Section 4 for further detail). One such source is legal obligations to act in a non-discriminatory way. Such obligations would be pertinent to the evaluation of universal HPV vaccination, which would address a potentially discriminatory variation in access to healthcare between the genders [54]. Beyond the critique of the methodologies adopted by existing studies, the interest of policy makers has changed from the three-dose schedule of HPV vaccination, on which these studies were based, to a two-dose schedule. The European Commission recently granted market authorisation for a two-dose schedule for the two commercialised vaccines [55]. The economic argument needs updating to consider the two-dose schedule. The objective of this article is to assess the extent to which economic studies have captured this range of values associated

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

94 95

96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122

123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157

G Model JVAC 15379 1–8

ARTICLE IN PRESS K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

158 159 160

161

162 163 164 165 166 167 168 169 170 171

with universal HPV vaccination, and how variation in the benefits included influences estimates of the cost-effectiveness of universal HPV vaccination.

2. Methods A recent, comprehensive review [56] identified nine studies of the cost-effectiveness of HPV vaccination in males through a search of PubMed up to end 2011 [57–64]. Eight of these evaluated universal HPV vaccination [57–64], and one evaluated vaccination of MSM only and was therefore dropped from the current analysis [65]. Study characteristics (such as setting, vaccination programme scenarios evaluated) and incremental cost effectiveness ratio (ICER) estimates were extracted. Data were then extracted on the costs and benefits included in the studies, including:

3

3. Results

215

3.1. Study characteristics

216

Table 1 summarises the study characteristics and principal findings of the eight CUAs [56]. All studies evaluated programmes in North American or Western European settings. The ICERs comparing universal vaccination to the equivalent females-only strategy ranged from D 18,331 [62] to D 612,065 [59]. The primary focus of most studies (n = 5 [57,59,61–63]) was not the evaluation of universal vaccination. Rather, in these studies universal vaccination was assessed as a scenario using a model that was primarily designed to assess the cost-effectiveness of vaccinating females. The ICER associated with universal vaccination was lower where the primary focus was assessing universal vaccination (mean D 50,056 [60,62,64]) than where assessing universal vaccination was a secondary objective (mean D 245,331 [57,59,61–63]). All studies were conducted with a three-dose schedule. 3.2. Values captured by economic evaluations of HPV vaccination

172 173 174 175

• • o •

176 177 178 179 180

•

181 182 183 184

•

185 186 187

o

188 189

o

190 191

•

192 193 194

•

195 196

•

197 198

•

199 200 201

• o

202 203 204 205 206 207 208 209

210 211 212 213 214

o

Cases of HPV-related diseases avoided Patient-level effects of HPV HRQoL effects of HPV Direct effects: HPV-related conditions and the HRQoL impact of these conditions and their treatments, including: cervical, anal and penile INs; cervical, vaginal, vulval, anal, penile and some head-and-neck cancers (principally oral cavity, oropharyngeal and laryngeal cancers); and RRP. Sequelae: fertility impacts; changes in appearance following head-and-neck cancer treatment; and consequences of anal incontinence or colostomy fitting following anal cancer treatment. Anxiety and distress regarding: cancer or IN recurrence; and long-term sequelae such as facing the impact on fertility, appearance and potential stigma. Out-of-pocket costs: transportation, incidentals such as adult diapers for anal incontinence following anal cancer treatment. Productivity: unemployment, work absenteeism and presenteeism. Family-level effects of HPV: the psychological burden, outof-pocket costs, and productivity losses experienced by family members and caregivers. Other individual impacts of HPV: utility-in-anticipation–the anxiety of contracting HPV among society more broadly. Fiscal impacts of HPV: Health system and other department costs. Social level effects of HPV: The benefits of fairness and gender equity. Costs associated with HPV vaccination Direct cost of vaccines and associated administration costs, including dynamic efficiencies in the delivery of vaccination: changes in vaccination price; incorporation of the vaccine within existing childhood vaccination schedules; changes in cervical cancer screening; economies of scale with the expansion of vaccination programmes. The cost of treating adverse events, such as rash, headache and rarer but more serious effects such as seizures and bronchospasm.

Two sets of descriptive analyses were performed. First, the frequency with which the benefits and costs listed above were included in each CUA was recorded. Second, the bivariate relationship between each study’s methodological approach and the resultant ICER was analysed.

The studies’ primary focus was on the HRQoL impact and health system costs of universal vaccination, including both the direct health effects of HPV and some of the sequelae (Table 2). Most studies measured HRQoL in terms of quality adjusted life years (QALYs) although one study estimated life-years gained [61]. All studies captured health system costs in terms of both the cost of vaccinations and the cost of the diseases included in the model. With the exception of one study that captured patient out-ofpocket expenses [60], and one study that captured productivity costs [61], none of the other values associated with universal HPV vaccination were captured by the studies. For instance, there was no consideration of: the HRQoL impacts of reduced fertility and anxiety; the impact of HPV on the family, friends and carers of patients; costs to government departments other than health system costs; or social values, such as equality of access to health (Table 2). Kim & Goldie (2009) included what they refer to as ‘direct nonmedical costs’, by which they meant patient costs. However, the study only captured a small portion of the potential patient costs avoided as a result of universal HPV vaccination. Specifically, the model captured patient time and transportation costs required to access vaccination [60]. Zechmeister et al. (2009) included reduced patient productivity. Specifically, the model captured work days lost due to cervical carcinoma [61]. Again, this only represents a small proportion of the potential productivity impact of HPV vaccination. The productivity loss is based on the employment rate, average value per working hour, median annual population gross income and average age of retirement in Austria, and estimated at 40 days sickness leave per case of cervical carcinoma [66]. One further study, Chesson et al., stated that it adopted a societal perspective, suggesting that it captured more costs and benefits than HRQoL and health system cost impacts. However, it is unclear whether a societal perspective was actually adopted by the model [64]. 3.3. Health-related quality of life While the primary focus of studies was on the impact to vaccination on HRQoL and health system costs, they varied in the extent to which they captured all the health impacts of universal HPV vaccination. All eight studies used transmission models, thus more reliably capturing the effects of herd-protection reducing the future incidence of infection. Only one model considered the full spectrum of HPV-related conditions (Table 3 [62]). All studies included cervical cancer and

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

217 218 219 220 221 222 223 224 225 226 227 228 229 230

231

232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265

266

267 268 269 270 271 272 273 274

G Model

ARTICLE IN PRESS

JVAC 15379 1–8

K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

4

Table 1 Summary of cost-effectiveness models evaluating universal HPV vaccinationa Study

Country

Vaccine target population: primary focus

D per QALY gainedb

Comparatorc

Taira et al., 2004 [57]

US

Females

D 315,742

Elbasha et al., 2007 [58]

US

Females

D 29,859

Jit et al., 2008 [59]

UK

Females

D 617,044

Kim & Goldie, 2009 [60] Zechmeister et al., 2009 [61]

US Austria

Universal Females

D 64,907 D 311,000d

Elbasha et al., 2010 [62] Olsen & Jepsen, 2010 [63] Chesson et al., 2011 [64]

US Denmark US

Universal Females Universal

D 18,331 D 18,677 D 66,929

Universal (12y) and universal booster (22y) vs. females only (12y) and females only booster (22y) Universal (12y) and catch up (12-24y, females only) vs. females only (12y) and catch up (12-24y, females only) Universal (12y) vs. females only (12y) and catch up (12-15y females only) Universal (12y) vs. females only (12y) Universal (12y) and universal booster (22y) vs. females only (12y) and females only booster (22y) Universal (9-26y) vs. females only (9-26y) Universal (12y) vs. no vaccination Females (12-26y) and males (12y) vs. females only (12-26y)

a

Table adapted from Jiang et al., 2013 [56]. Calculated using the exchange rate D 1 = $1.4 = £0.85 (November 2010 [83]). c Expressed in terms of life years gained rather than quality-adjusted life years gained. d The ICER from Zechmeister et al., 2009 [61] is the cost per life year gained ICER: Incremental cost-effectiveness ratio; y: years of age. ICER associated with the analysis accounting for most of HPV-associated diseases b

296

IN. All but two studies included EGW [57,61]. Six studies included HRQoL effects for cancer survivors [57–60,62,64] but no study included the indirect effects experienced by patients who have recovered from HPV-related conditions, such as impacts on fertility or emotional sequelae such as anxiety and distress. Seven studies expressed the ICER in terms of QALYs gained, while one estimated life-years gained, which ignores the value of preventing HPV-related morbidity [61]. However, a number of limitations were identified with the approaches used to estimate utility. First, it was assumed that utilities for vulvar, vaginal, penile, anal and head-and-neck cancers were the same as those of cervical disease [60,62], or the same utility value was used for all cancers, distinguishing just the same stage at cancer diagnosis (local, regional, distant) and survival outcome (survival, death) [64]. Second, the gaps in the literature meant that it was necessary to use utility estimates generated using different valuation methods. For example, Chesson et al. (2011) [64] calculated QALYs lost per episode of EGW as the average of values from three studies that employed different utility methods: Institute of Medicine committee-consensus Health Utility Index expert panel [67]; the EQ-5D (EQ-5D index and EQ visual analogue scale scores); and the CECA10 [68].

297

3.4. Treatment costs

275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295

all had ICERs below D 70,000. There were, however, two studies that achieved low ICERs based on a low number of health states [58,63]. Potential reasons for these low ICERs include: using a no vaccination scenario as the comparator [63]; high estimates of the costs of cervical cancer treatment; and high probabilities of sexual HPV transmission [58,63]. There is insufficient variation in the perspective adopted to identify its impact on the ICER. Further, it is not clear that those studies that stated they adopted a societal perspective actually did so [64], and where a societal perspective was adopted only a small proportion of non health-related benefits were captured by models. Thus, it would not be expected that perspective would have an impact on the ICER. Zechmeister et al. (2009) is not included in Fig. 2, as it generated estimates of cost per life-year gained, rather than cost per QALY gained [61]. This study allows ICERs to be generated based on both direct and indirect costs, but demonstrated only a small difference between these estimates. The cost per life-year gained of universal HPV vaccination was estimated at D 311,000 when just based on health gains and health system costs, and D 299,000 when indirect costs (productivity impacts) are included. As noted earlier, this small impact is likely due to the incomplete manner in which productivity impacts are included in the model. 4. Discussion

303

All models included the cost of the vaccines and direct healthcare system costs associated with HPV-related health outcomes (Table 2). However, all models included estimates of the cost of vaccine at the current retail price, and not at the tender market price in relevant countries such the UK, and none allowed for dynamic efficiencies to reduce vaccine prices, nor a two-dose schedule.

304

3.5. Impact of evaluation methods on results

298 299 300 301 302

305 306 307 308 309 310 311 312 313 314

Fig. 2 shows the results of the studies that estimated the cost per QALY gained for universal HPV vaccination, demonstrating how the perspective adopted and the number of health states included in models impacts these results. It shows how the ICER varies with the number of HPV-related health states (cervical IN, anal cancer, junior onset RRP, etc.) included in the model, and how these states are valued. The small sample size and variation in methods, however, means that it is difficult to draw firm conclusions from this figure. While studies including more health states tended to have low ICERs. Those studies that included more than five health states

A recent review of the economic literature has concluded that universal HPV vaccination may not be cost-effective [43]. This review may, however, be drawing inaccurate conclusions. First, the evidence reviewed was concerned with the cost-effectiveness of a three-dose schedule of HPV vaccination, while the use of vaccination is moving towards the use of a two-dose schedule. Second, the limited scope of existing economic studies may mean that they are underestimating the economic value of universal HPV vaccination. Studies tend to focus on the HRQoL gains and health system cost savings from HPV vaccination. They do not capture all health impacts, and ignore improvements in productivity, non-health system costs, utility-in-anticipation, and reductions in health inequality. A minority of studies of universal HPV vaccination include values beyond health gains and health system costs, such as patient time, out-of-pocket expenses, and productivity costs. However, the precise methods for capturing these costs are inadequately explained. Further, it seems that these efforts only capture a small proportion of these non-health values.

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337

338

339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356

G Model

ARTICLE IN PRESS

JVAC 15379 1–8

K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

€700,000

5

Jit 2008 Health system perspective

€600,000 ICERa (€/QALY gained)

Societal perspective

€500,000 €400,000

Taira 2004

€300,000 €200,000 €100,000 €0

Chesson 2011 Kim 2009 Elbasha 2010

Olsen 2010 Elbasha 2007

0

5

10

15

20

25

Number of modelled HPV-related health statesb a

ICER: incremental cost-effectiveness ratio; QALY: quality adjusted life year.

b

The health states included in Figure 2 are more detailed than the diseased included in Table 3. For example, the health states

distinguish between local, regional and distant cervical cancer.

Fig. 2. Relationship between perspective, number of modelled HPV-related health states and ICER estimates of reviewed cost-effectiveness models.

357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399

This critique is consistent with previous reviews of economic studies of HPV vaccination studies [46,56] and is not specific to studies of universal HPV vaccination. Similar limitations have been identified economic evaluations of vaccines against other infections [47,69,70], as well as economic studies of public health interventions more generally [71]. Furthermore, other studies suggest that responding to this critique and adopting a broader perspective could change the assessment of universal HPV. For instance, Beutels et al. (2003) [48] argue that, “from a societal point of view varicella vaccination results in substantial net savings, thanks to avoided time losses of parents and other caretakers. Hence the inclusion of time losses solely changes the decision. It is therefore unfortunate that these costs, by lack of reliable and widely accepted methodologies are usually not deemed credible.” It is important that economic analysis of universal HPV vaccination adopts a broader perspective than is the case with the existing literature. Without this information, there is a risk that vaccination is inappropriately restricted to females, at the expense of significant social costs. There are a number of ways in which estimates of the value of universal HPV vaccination could be made more complete. First, a broader perspective could be adopted while maintaining the CUA frameworks conventionally adopted by health economists. Further research is required to facilitate this approach. First, work is required to estimate the impact of HPV-related diseases on utility, out-of-pocket costs to patients and families, and nonhealth system costs. Research on the indirect costs of cancer suggests that HPV-related conditions may have significant nonhealth impacts, such as lowering productivity. For instance, only 40% of the total measurable costs of cancer are directly related to the healthcare system, with the remainder being cost of informal care (unpaid care provided by relatives or friends, 38%); lost earnings after premature death (18%), and lost productivity, temporary or permanent, due to illness (8%) [72]. Similarly, Insinga et al. estimated that the indirect costs of cervical cancer in the US exceed the direct medical costs by a factor of several times [73], and the Association of Cancer Patients in France estimated that 47% of cancer patients incur costs not covered by the healthcare system or insurance, and that 30% lost their job within two years of their diagnosis [74]. Further work to understand these costs and incorporate them into economic models would produce a more complete estimate of the cost-effectiveness of universal HPV vaccination.

Second, a better understanding of HPV-related health conditions would improve current models. While all the models reviewed captured the effects of infection transmission, it is important that these models are kept up to date to reflect the changing incidence of HPV-related diseases. For instance, models may need to be reparameterised and re-calibrated to reflect the rising prevalence of head-and-neck cancers [75,76]. More epidemiological research would improve our understanding of the health gains associated with avoided HPV, in particular, HPV-attributable head-and-neck cancers and the sequelae of HPV-related cancers. Third, further work would be required to determine the opportunity cost of investments from this broader perspective. In countries such as the UK and Holland, a willingness to pay threshold is used as a proxy for opportunity cost, with the decision rule being that an intervention should be adopted if its cost-effectiveness ratio is lower than the threshold. To assess the cost-effectiveness of interventions from a societal perspective would require the reestimation of this threshold by considering the value of current uses of the health budget from a societal perspective. An alternative decision rule is adopted by cost-benefit analysis (CBA). CBA converts all cost and benefits into monetary units, facilitating their comparison, and concludes that an intervention is good value for money if its benefits exceed its costs. A CBA would be able to capture the impacts of universal HPV vaccination to which the measures of utility currently employed in CUA are insensitive, such as utility-in-anticipation. However, CBA makes a number of assumptions that raise concerns about the validity of its conclusions. In particular, it is necessary to assume that the monetary values used in the analysis are a valid reflection of opportunity cost. However, there is a large literature on the challenges associated with both market prices or values generated through non-market valuation techniques [77]. A limitation of both CUA and CBA is that a complete picture of the value of public health interventions requires going beyond economic value to also consider values such as reductions in health inequalities [71]. Despite the importance of ideas of equity to the design of many healthcare systems, pragmatic tools integrating them into health technology assessments (HTAs) are still underdeveloped [78]. Further research is required to improve our evaluation methods to capture health inequality affects in HTAs [79]. First, public debate is required to determine which notions of equity are relevant to decision making. Second, methods to measure health equity, and how this changes with interventions, need

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442

Family/carer

HRQoLc

a b c

× √ √ √

Fertility Impact

Other sequelae √ √ √ √

×b × × × × × × ×

Anxiety × × × × × × × ×

× √ × √

× × × √

× × × × √

× × × ×

× × ×

Government

HRQoL

Out-of-Pocket Costs

Fertility impact

Other sequelae

Anxiety

× × × × × × × ×

× × × × × × × ×

× × × × × × × ×

× × × × × × × ×

Social level HRQoL

Productivity

Health system √ √ √ √ √ √ √ √

× × × × × × × ×

Other dept.d

Anxiety

× × × × × × × ×

× × × × × × × ×

Health Equality

× × × × × × × ×

Included. Not included or not reported as being included. HRQoL: Health-related quality of life. Dept.: Department

K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

d

Direct health effects √a √ √ √

Productivity

Table 3 Summary of HPV-related conditions included in cost-effectiveness models evaluating universal HPV vaccination. Study

Intraepithelial Neoplasia

Cancers Cervical

Taira et al., 2004 [57] Elbasha et al., 2007 [58] Jit et al., 2008 [59] Kim et al., 2009 [60] Zechmeister et al., 2009 [61] Elbasha et al., 2010 [62] Olsen et al., 2010 [63] Chesson et al., 2011 [64] a b c

Cervical √a √ √ √ √ √ √ √

Anal

Penile

×b × × × × √

× × × × × √

× ×

× ×

Recurrent Respiratory Papillomatosis Vaginal

Vulval

Anal

Penile

c

√ √ √ √ √ √ √ √

× × × √

× × × √

× × × √

× × × √

× √

× √

× √

× √

× √

× √

× √

× √

Included. Not included or not reported as being included. OC: Oral cavity; O-PH: Oropharyngeal; LAR: Laryngeal; JO: Junior onset; AO: Adult onset.

External Genital Warts

Head-and-Neck OC

O-PH

LAR

JO

AO

× × × × × √

× × × √

× × × × × √

× × × √

× × × × × √

× ×

× √

× ×

× √

× √

× √

× ×

× √ √ √ × √ √ √

ARTICLE IN PRESS

Taira et al., 2004 [57] Elbasha et al., 2007 [58] Jit et al., 2008 [59] Kim et al., 2009 [60] Zechmeister et al., 2009 [61] Elbasha et al., 2010 [62] Olsen et al., 2010 [63] Chesson et al., 2011 [64]

Out-of-Pocket Cost

G Model

HPV patient

JVAC 15379 1–8

6

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

Table 2 Summary of costs and values included in cost-effectiveness models evaluating universal HPV vaccination.

G Model JVAC 15379 1–8

ARTICLE IN PRESS K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

486

to be agreed. Third, social value judgments are required about which sources of health inequality are fair and unfair. Fourth, judgments are required to combine or trade off changes in health gain and unfair health inequality. It is beyond the scope of this study to address these challenges. Rather, we limit ourselves to supporting the position that equity is not a matter for a priori definition, but something that should be discovered by those responsible for decisions [78]. For instance, conventional CUA sits within the utilitarian tradition. However, the notions of equity referenced in discussions about healthcare – equity of need, deservingness, capacity to benefit, degree of incapacity or health state imply moving away from this utilitarian tradition. But rather than adopting any other overarching theory of justice, a more pragmatic approach should be adopted that acknowledges the validity of the currencies proposed by the multiple schools of thought and allows these competing objectives to be weighted by decision makers. Moving beyond the utilitarian tradition of conventional analysis and acknowledging the validity of multiple “currencies” points us towards alternative methods, such as multi-criteria decision analysis (MCDA). MCDA has received much attention for its ability to define, measure, and weight multiple criteria, including efficacy and equity [53]. It can support healthcare decision making through: facilitating a transparent judgement of the value of multiple criteria; the consideration of multiple measures as long as they can be transferred into preference values; and the explicit involvement of all stakeholders [80]. As a result, the use of MCDA in healthcare has increased exponentially in recent years [81], including the proposal of an MCDA framework for valuing vaccines [82]. The application of MCDA to healthcare is, however, still in its infancy [80]. It is recommended, therefore, that further work be undertaken to develop an MCDA framework for incorporating equity considerations into healthcare decision making. This should include the piloting of the MCDA framework on interventions where equity considerations are relevant, including universal HPV vaccination. In conclusion, current studies of the cost-effectiveness of universal HPV vaccination suffer from a number of limitations. In particular, they overlook a number of the values generated by vaccination. Further work is required to overcome these limitations. This should include not only the generation of evidence on, for instance, the health and productivity impact of HPV, but should also consider the appropriateness of current frameworks for the assessment of vaccination.

487

Conflict of interest

443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485

491

NL and XB are employees of Sanofi Pasteur MSD. KM and RC are employees of Evidera that received funding from Sanofi Pasteur MSD to conduct this study. RFB was also employed by Evidera during the writing of this manuscript.

492

Author contribution

488 489 490

493

494

All authors have approved the final article. Acknowledgements

496

We would like to thank Rosanna Tarricone, Isabelle Borget and Juergen Wasem for helpful comments.

497

References

495

498 499 500

[1] Smith JS, et al. Human papillomavirus type distribution in invasive cervical cancer and high-grade cervical lesions: a meta-analysis update. Int J Cancer 2007;121(3):621–32.

7

[2] Bosch FX, et al. Epidemiology and natural history of human papillomavirus infections and type-specific implications in cervical neoplasia. Vaccine 2008;26(Suppl. 1):K1–16. [3] De Vuyst H, et al. Prevalence and type distribution of human papillomavirus in carcinoma and intraepithelial neoplasia of the vulva, vagina and anus: a meta-analysis. Int J Cancer 2009;124(7):1626–36. [4] Menton JF, et al. Molecular epidemiology of sexually transmitted human papillomavirus in a self referred group of women in Ireland. Virol J 2009;6:112. [5] Aubin F, et al. Human papillomavirus genotype distribution in external acuminata condylomata: a large French national study (EDiTH IV). Clin Infect Dis 2008;47(5):610–5. [6] Donne AJ, et al. The role of HPV type in recurrent respiratory papillomatosis. Int J Pediatr Otorhinolaryngol 2010;74(1):7–14. [7] Backes DM, et al. Systematic review of human papillomavirus prevalence in invasive penile cancer. Cancer Cause Control 2009;20(4):449–57. [8] Miralles-Guri C, et al. Human papillomavirus prevalence and type distribution in penile carcinoma. J Clin Pathol 2009;62(10):870–8. [9] Insinga RP, et al. A systematic review of the prevalence and attribution of human papillomavirus types among cervical: vaginal, and vulvar precancers and cancers in the United States. Cancer Epidemiol Biomarkers Prev 2008;17(7):1611–22. [10] Kreimer AR, et al. Human papillomavirus types in head and neck squamous cell carcinomas worldwide: a systematic review. Cancer Epidemiol Biomarkers Prev 2005;14(2):467–75. [11] Hansson BG, et al. Strong association between infection with human papillomavirus and oral and oropharyngeal squamous cell carcinoma: a population-based case-control study in southern Sweden. Acta Otolaryngol 2005;125(12):1337–44. [12] D‘Souza G, et al. Case-control study of human papillomavirus and oropharyngeal cancer. N Engl J Med 2007;356(19):1944–56. [13] Schover LR. Sexuality and fertility after cancer. Hematology Am Soc Hematol Q2 Educ Program 2005:523–7. [14] Corney RH, et al. Psychosexual dysfunction in women with gynaecological cancer following radical pelvic surgery. Brit J Obstet Gynaecol 1993;100(1):73–8. [15] Alexander-Sefre F, et al. Surgical morbidity associated with radical trachelectomy and radical hysterectomy. Gynecol Oncol 2006;101(3):450–4. [16] Soergel P, et al. The cost efficiency of HPV vaccines is significantly underestimated due to omission of conisation-associated prematurity with neonatal mortality and morbidity. Hum Vaccin Immunother 2012;8(2):243–51. [17] Albrechtsen S, et al. Pregnancy outcome in women before and after cervical conisation: population based cohort study. BMJ 2008;337:a1343. [18] Glaser AW, et al. Patient-reported outcomes of cancer survivors in England 1–5 years after diagnosis: a cross-sectional survey. BMJ Open 2013;3(4). [19] Skaali T, et al. Fear of recurrence in long-term testicular cancer survivors. Psychooncology 2009;18(6):580–8. [20] Mitchell AJ, et al. Depression and anxiety in long-term cancer survivors compared with spouses and healthy controls: a systematic review and metaanalysis. Lancet Oncol 2013;14(8):721–32. [21] Bonanno A, Esmaeli B. Cancer and facial disfigurement: reducing survivors’ stigma in social interaction. Clin J Oncol Nurs 2012;16(2):153–9. [22] Kobayashi M, et al. Psychological distress and quality of life in cervical cancer survivors after radiotherapy: do treatment modalities, disease stage, and selfesteem influence outcomes? Int J Gynecol Cancer 2009;19(7):1264–8. [23] Yap JK, Davies M. Fertility preservation in female cancer survivors. J Obstet Gynaecol 2007;27(4):390–400. [24] Mortensen GL, Jakobsen JK. Patient perspectives on quality of life after penile cancer. Dan Med J 2013;60(7):A4655. [25] Maddineni SB, Lau MM, Sangar VK. Identifying the needs of penile cancer sufferers: a systematic review of the quality of life, psychosexual and psychosocial literature in penile cancer. BMC Urol 2009;9:8. [26] Philip EJet al. Psychological correlates of sexual dysfunction in female rectal and anal cancer survivors: analysis of baseline intervention data. J Sex Med Q3 2013. [27] Shepherd MA, Gerend MA. The blame game: cervical cancer, knowledge of its link to human papillomavirus and stigma. Psychol Health; 2013. [28] Crawshaw M. Male coping with cancer-fertility issues: putting the ‘social’ into biopsychosocial approaches. Reprod Biomed Online 2013;27(3):261–70. [29] Manderson L. Boundary breaches: the body, sex and sexuality after stoma surgery. Soc Sci Med 2005;61(2):405–15. [30] Garland SM, et al. Quadrivalent vaccine against human papillomavirus to prevent anogenital diseases. N Engl J Med 2007;356(19):1928–43. [31] Group FIS. Quadrivalent vaccine against human papillomavirus to prevent high-grade cervical lesions. N Engl J Med 2007;356(19):1915–27. [32] Wheeler CM, et al. The impact of quadrivalent human papillomavirus (HPV; types 6: 11, 16, and 18) L1 virus-like particle vaccine on infection and disease due to oncogenic nonvaccine HPV types in sexually active women aged 16–26 years. J Infect Dis 2009;199(7):936–44. [33] Brown DR, et al. The impact of quadrivalent human papillomavirus (HPV; types 6: 11, 16, and 18) L1 virus-like particle vaccine on infection and disease due to oncogenic nonvaccine HPV types in generally HPV-naive women aged 16-26 years. J Infect Dis 2009;199(7):926–35. [34] Hartwig S, et al. Estimation of the epidemiological burden of human papillomavirus-related cancers and non-malignant diseases in men in Europe: a review. BMC Cancer 2012;12:30. [35] Malagon T, et al. The impact of differential uptake of HPV vaccine by sexual risks on health inequalities: a model-based analysis. Vaccine 2013;31(13):1740–7.

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586

G Model JVAC 15379 1–8 8 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650

ARTICLE IN PRESS K. Marsh et al. / Vaccine xxx (2014) xxx–xxx

[36] Australian Cancer Research Foundation. World-first HPV vaccination plan will protect young Australian men from cancer. Available at: http://acrf.com.au /2012/world-first-hpv-vaccination-plan-will-protect-young-australian-menfrom-cancer/ Accessed August 28, 2013. 2012. [37] Public Health Agency of Canada. Update on human papillomavirus (HPV) vaccines: recommendations. Available at: http://www.phac-aspc.gc.ca/ publicat/ccdr-rmtc/12vol38/acs-dcc-1/index-eng.php#a5 Accessed August 28, 2013. Canada Communicable Disease Report, 2012. 38. [38] Prince Edward Island Canada, Department of Health and Wellness. Press Release: HPV vaccine program extends to include boys. Available at: http://www.gov.pe.ca/health/index.php3?number = news&newsnumber = 8937&dept = &lang = E Accessed August 28, 2013. 2013. [39] CDC, Centers for Disease Control and Prevention. Recommendations on the use of quadrivalent human papillomavirus vaccine in males–Advisory Committee on Immunization Practices (ACIP). Available at: http://www.cdc.gov/ mmwr/preview/mmwrhtml/mm6050a3.htm Accessed 28 August, 2013. 2011. [40] Reiter PL, Gilkey MB, Brewer NT. HPV vaccination among adolescent males: results from the National Immunization Survey-Teen. Vaccine 2013;31(26):2816–21. [41] Laz TH, Rahman M, Berenson AB. Human papillomavirus vaccine uptake among 18- to 26-year-old women in the United States: National Health Interview Survey, 2010. Cancer 2013;119(7):1386–92. [42] Welfare Society Territory (WEST), Austria to vaccinate boys and girls against HPV. Available at: http://www.west-info.eu/austria-to-vaccinate -boys-and-girls-against-hpv/. Accessed April 23, 2014, 2014. [43] ECDC. European Centre for Disease Prevention and Control. Introduction of HPV vaccines in European Union countries – an update. Stockholm. ECDC; 2012. Available at: http://ecdc.europa.eu/en/publications/publications/ 20120905 gui hpv vaccine update.pdf Accessed August 28, 2013. 2012. [44] Seto K, et al. The cost effectiveness of human papillomavirus vaccines: a systematic review. Drugs 2012;72(5):715–43. [45] Canfell K, et al. Modeling preventative strategies against human papillomavirus-related disease in developed countries. Vaccine 2012;30(Suppl. 5):F157–67. [46] Barnighausen T, et al. Economic evaluation of vaccination: capturing the full benefits, with an application to human papillomavirus. Clin Microbiol Infect 2012;18(Suppl. 5):70–6. [47] Drummond M, Chevat C, Lothgren M. Do we fully understand the economic value of vaccines? Vaccine 2007;25(32):5945–57. [48] Beutels P, et al. Methodological issues and new developments in the economic evaluation of vaccines. Expert Rev Vaccines 2003;2(5):649–60. [49] Brisson M, Edmunds WJ. Economic evaluation of vaccination programs: the impact of herd-immunity. Med Dec Making 2003;23(1):76–82. [50] Beutels P, Jit M. A brief history of economic evaluation for human papillomavirus vaccination policy. Sex Health 2010;7(3):352–8. [51] EuroQol, EQ-5D User Guide. Available at: http://www.euroqol.org/about-eq -5d/publications/user-guide.html. Accessed November 7 2013. 2013. [52] Wailoo, A., S. Davis, and J. Tosh, The incorporation of health benefits in cost utility analysis using the EQ-5D. Report by the Decision Support Unit. Available at: http://www.nicedsu.org.uk/PDFs%20of%20reports/DSU%20EQ5D%20final %20report%20-%20submitted.pdf. Accessed November 12 2013. 2010. [53] Baltussen Ret al. Balancing efficiency, equity and feasibility of HIV treatment in South Africa – development of programmatic guidance. Cost Eff Resour Alloc 2013;11(1):26. [54] Luyten J, Engelen B, Beutels P. The Sexual Ethics of HPV Vaccination for Boys. HEC Forum; 2013. [55] European Commission, Community register of medicinal products for human use. Available at: http://ec.europa.eu/health/documents/community-register/ html/h357.htm. Accessed April 23, 2014, 2014. [56] Jiang Y, et al. A critical review of cost-effectiveness analyses of vaccinating males against human papillomavirus. Hum Vaccine Immunother 2013;9(11). [57] Taira AV, Neukermans CP, Sanders GD. Evaluating human papillomavirus vaccination programs. Emerg Infect Dis 2004;10(11):1915–23. [58] Elbasha EH, Dasbach EJ, Insinga RP. Model for assessing human papillomavirus vaccination strategies. Emerg Infect Dis 2007;13(1):28–41.

[59] Jit M, Choi YH, Edmunds WJ. Economic evaluation of human papillomavirus vaccination in the United Kingdom. BMJ 2008;337:a769. [60] Kim JJ, Goldie SJ. Cost effectiveness analysis of including boys in a human papillomavirus vaccination programme in the United States. BMJ 2009;339: b3884. [61] Zechmeister I, et al. Cost-effectiveness analysis of human papillomavirusvaccination programs to prevent cervical cancer in Austria. Vaccine 2009;27(37):5133–41. [62] Elbasha EH, Dasbach EJ. Impact of vaccinating boys and men against HPV in the United States. Vaccine 2010;28(42):6858–67. [63] Olsen J, Jepsen MR. Human papillomavirus transmission and cost-effectiveness of introducing quadrivalent HPV vaccination in Denmark. Int J Technol Assess Health Care 2010;26(2):183–91. [64] Chesson HW, et al. The cost-effectiveness of male HPV vaccination in the United States. Vaccine 2011;29(46):8443–50. [65] Kim JJ. Targeted human papillomavirus vaccination of men who have sex with men in the USA: a cost-effectiveness modelling analysis. Lancet Infect Dis 2010;10(12):845–52. [66] Statistik Austria. Jahrbuch der Gesundheitsstatistik. Wien: Statistik Austria; 2007. [67] Stratton, K.R., J. Durch, and R.S. Lawrence, Institute of Medicine (U.S.). Committee to Study Priorities for Vaccine Development. Vaccines for the 21st century: a tool for decision making. Washington: National Academy Press. Available at: http://www.nap.edu/catalog.php?record id = 5501 Accessed 4 October 2013. 2000. [68] Woodhall S, et al. Estimation of the impact of genital warts on health-related quality of life. Sex Transm Infect 2008;84(3):161–6. [69] Barnighausen T, et al. Rethinking the benefits and costs of childhood vaccination: the example of the Haemophilus influenzae type b vaccine. Vaccine 2011;29(13):2371–80. [70] Bloom, D.E. and D. Canning, The value of vaccination. World Economics, 2005. 6(3). [71] Marsh Ket al. Estimating cost-effectiveness in public health: a summary of modelling and valuation methods. Health Econ Rev 2012;2(1):17. [72] Luengo-Fernandez R, et al. Economic burden of cancer across the European Union: a population-based cost analysis. Lancet Oncol 2013;14(12): 1165–74. [73] Insinga RP. Annual productivity costs due to cervical cancer mortality in the United States. Womens Health Issues 2006;16(5):236–42. [74] Rapport 2013 de l’Observatoire sociétal des cancers, Ligue nationale contre le cancer. [75] Sturgis EM, Cinciripini PM. Trends in head and neck cancer incidence in relation to smoking prevalence: an emerging epidemic of human papillomavirusassociated cancers? Cancer 2007;110(7):1429–35. [76] Mehanna H, et al. Prevalence of human papillomavirus in oropharyngeal and nonoropharyngeal head and neck cancer – systematic review and metaanalysis of trends by time and region. Head Neck 2013;35(5):747–55. [77] Dolan, P. and R. Metcalfe, Comparing willingness to pay and subjective wellbeing in the context of non-market goods. CEP discussion paper No. 890, 2008. [78] Culyer, A. and Y. Bombard, An equity checklist: a framework for health technology assessments. CHE Research Paper 62, 2011. [79] Asaria M, Cookson R, Griffin S. Incorporating health inequality impacts into CEA. Elsevier Online Encyclopaedia of Health Economics; 2013. [80] Marsh K, et al. Assessing the value of healthcare interventions using multicriteria decision analysis: a review of the literature. Pharmacoeconomics 2014;32(4):345–65. [81] Diaby V, Campbell K, Goeree R. Multi-criteria decision analysis (MCDA) in health care: A bibliometric analysis. Operations Research for Health Care 2013;2:20–4. [82] (IOM), I.o.M., Ranking Vaccines A Prioritization Framework - Phase I: Demonstration of Concept and a Software Blueprint. Available at: http://www.iom .edu/Reports/2012/Ranking-Vaccines-A-Prioritization-Framework-Phase-I .aspx. Accessed 2nd January 2014. 2012. [83] Reuters Currencies Quote. Available at: http://uk.reuters.com/business/ currencies/ Accessed 3 October 2013. 2013.

Please cite this article in press as: Marsh K, et al. Mind the gaps: What’s missing from current economic evaluations of universal HPV vaccination? Vaccine (2014), http://dx.doi.org/10.1016/j.vaccine.2014.05.007

651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715