Clin Genet 2016: 89: 133–138 Printed in Singapore. All rights reserved
© 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd CLINICAL GENETICS doi: 10.1111/cge.12592
Social and Behavioural Research in Clinical Genetics Section Editor: Aad Tibben, email: [email protected]
Evaluating stakeholder’s perspective on referred out genetic testing in Canada: a discrete choice experiment Blumenschein P., Lilley M., Bakal J.A., Christian S. Evaluating stakeholder’s perspective on referred out genetic testing in Canada: a discrete choice experiment. Clin Genet 2016: 89: 133–138. © John Wiley & Sons A/S. Published by John Wiley & Sons Ltd, 2015 The expanding number and increasing utility of clinical genetic tests is creating a growing burden on the Canadian healthcare system. Administrators are faced with the challenge of determining which genetic tests should be publicly funded. A discrete choice experiment (DCE) was utilized to assess the importance stakeholders place on five attributes of a genetic test. One hundred ninety individuals completed the DCE questions. Analysis of the data revealed that medical benefit of a test had the greatest impact on a respondent’s decision to select a test for funding. The detection rate of the test ranked second in importance followed by severity of the condition, aim of the test, and cost. With limited resources available for referred out molecular genetic testing within a public healthcare setting such as Canada’s, funding guidelines are critical. Our findings provide further evidence for the value of a decision-making framework and the relative importance of specific test attributes within such a framework. Conflict of interest
The authors have no conflict of interest to declare.
New technologies, increased clinical utility and a greater awareness have resulted in higher demand for molecular genetic testing (1). Not all testing can be offered within any one jurisdiction resulting in a large number of samples being referred out for analysis. Each province or territory is responsible for the administration and financing of referred out molecular genetic testing. This has led to expanding costs which must be balanced against the need for fiscal responsibility within public health care systems. Little guidance is available to health care administrators on how molecular genetic testing should be prioritized. In Canada, allocation of resources for molecular genetic testing is regional and there is little consistency between jurisdictions with regard to the process or criteria used to make funding decisions. (2, 3)
P. Blumenscheina , M. Lilleya , J.A. Bakalb and S. Christiana a Genetic
Laboratory Services, Alberta Health Services, Edmonton, Canada and b Patient Health Outcomes Research and Clinical Effectiveness Unit, University of Alberta, Edmonton, Canada
Key words: discrete choice experiment – genetic testing – health care policy – resource allocation Corresponding author: Margaret Lilley, 826 Medical Sciences Building University of Alberta, Edmonton, AB T6G 2H7 Canada. Tel.: +780 407 1015 fax: +780 407 1761 e-mail: Margaret.Lilley@ albertahealthservices.ca Received 22 January 2015, revised and accepted for publication 27 March 2015
Discrete choice experiment (DCE) is a method used to elicit the preferences of a respondent. This approach is commonly used in economics and marketing as it allows for the analysis of preferences for goods or services with multiple attributes (4). It has more recently been used in the field of healthcare. In 2014, Severin et al. used this approach to assess the value stakeholders attach to various attributes of a genetic test in a European population (5). The DCE results revealed the relative importance of the test attributes in the decision-making process for funding a genetic test. This study used DCE to evaluate Canadian healthcare professionals’ views regarding molecular genetic testing and prioritization of funding. Stakeholders’ views on the relative importance of different test attributes could be
133
Blumenschein et al. useful information for health care administrators when creating policies for referred out testing. Methods Study design and participants
This was a cross-sectional study involving clinical genetic counsellors, medical geneticists, neurologists, endocrinologists, and hematologists from across Canada. In January 2014, participants were invited to complete an online survey which was developed using The Survey System Version 10 (6). The survey included demographic questions, a series of DCE questions and a question regarding minimum likelihood of disease cut-offs. Members of the Canadian Association of Genetic Counsellors (CAGC) and the Canadian College of Medical Geneticists (CCMG) were invited to participate through an e-mail sent out by their respective listservs. Members of the Colleges of Physicians and Surgeon of Alberta, Manitoba and Nova Scotia indicating a specialty in neurology, endocrinology or hematology were mailed invitations to participate with the option of responding via an electronic link or by mail. A self-addressed, postage paid envelope was included with the invitation. These three provinces were selected for the study because they have similar funding systems for referred out molecular genetic testing. All three provinces have formal application processes, have a committee to review the test requests, and allow non-genetics specialists to request funding (2). Neurologists, endocrinologists and hematologists were selected based on the authors’ experience with referred out molecular genetic testing in Alberta where, apart from geneticists, they were the most frequent requestors. In addition, Alberta physicians who had submitted an application for referred out molecular genetic testing were contacted by e-mail and invited to participate in the study. This allowed us to invite physicians who request referred out testing but were not members of the above specialties. Survey tool
and they had to decide which of the two tests should be publically funded. They could pick only one test to fund. A sample question is provided in Fig. 1. Each attribute included three levels which were defined at the beginning of the survey (Table 1). The survey design allowed a D-efficiency of 0.69. This D-efficiency indicates that this survey presented 69% of all possible test attribute combinations. In order to compare all possible combinations of attributes, the survey would have been impractical in length. Limiting the number of combinations and the number of questions, accordingly, was done to improve the response rate and reduce respondent fatigue. In the electronic version, DCE questions were presented in a random order to reduce potential bias related to decision fatigue. Three versions of the paper questionnaire with questions ordered differently were used to reduce bias. The survey was only available in English. The survey tool used was based largely on the tool developed and validated by Severin et al. (with permission from the authors) (5). The Severin survey included an attribute described as ‘risk of having the condition’ which was replaced in this survey with ‘detection rate of the test’. This modification was based on the authors experience with referred out molecular genetic testing. The cost levels were also modified to reflect estimates for familial mutation testing, single gene testing and large gene panels in current (2014) Canadian dollars. Likelihood question
The likelihood that an individual has a given condition can be an important factor in funding for molecular genetic testing. The detection rate of a given molecular testing can vary from 99% to less than 5% depending on the number of genes and the types of mutations that have been reported. Participants were asked to indicate on a sliding scale (0–100%) what they felt was an appropriate minimum likelihood of disease to justify funding a genetic test. They were told to assume that the test has a high detection rate and that genetic test results would impact medical management. This question was added to help with the comparison of our data with results by Severin et al. as they included the risk of disease as an attribute in their DCE questions (5). Statistical analysis
Demographics
The first part of the survey instrument gathered information regarding age, gender, province of practice, specialty and if the participant had previously ordered referred out molecular genetic testing. Discrete choice experiment
Nineteen DCE questions were used to assess the importance placed on five attributes of a molecular genetic test. Attributes included severity of the condition, aim, benefit, detection rate and cost of the test. The primary benefit of the test may be to the patient directly or to their family members. Participants were asked to imagine that they were the decision maker in a health care organization
134
Demographic variables were summarized using counts and chi-squared tests or Fisher exact tests where appropriate. The test descriptor variables in the DCE questions were effect coded to account for possible non-linear effects on choice, for both the selected and unselected tests. The chosen test was given a response of 1 and the non-selected test was assigned a 0. The DCE questions were evaluated using a series of mixed logit models which take into account the respondents variability in value judgments and comparison of parameters in each question. The models can then be evaluated using adjusted odds ratios based on the estimated coefficients. Additional demographic parameters were dummy coded and included in the models as interaction terms to
Evaluating stakeholder’s perspective on referred out genetic testing
Fig. 1. Sample discrete choice experiment (DCE) question.
evaluate effects. All analyzes were conducted using SAS 9.4 (Cary, NC). Results
In total, 993 individuals were invited to participate in the study. One hundred ninety individuals responded to the survey (19%). Fifty-three (28%) paper surveys were completed and the remaining 137 (72%) surveys were completed electronically. Respondent demographics
Areas of practice included Genetics (n = 110, 58%), Neurology (n = 42, 23%), Hematology (n = 13, 7%), Endocrinology (n = 10, 5%) and Other (n = 15, 8%). The greatest number of respondents practiced in Alberta and Ontario, 78 (41%) and 46 (24%), respectively. Alberta was disproportionately represented as a greater number of Alberta physicians were invited to participate compared to other provinces. There may also be an Alberta bias given the location of the authors. Demographics are summarized in Table 2. Preferred genetic testing attributes
Analysis of the DCE responses showed that medical benefit of a test had the greatest impact on respondents’ decision to select a test to be funded. Proven medical benefit showed a 26-fold increase in odds of being chosen over no medical benefit (Table 3). The detection rate of the test ranked second in importance with a sevenfold difference between the largest and smallest rate, followed by severity of the condition, and aim of the test with
around a twofold increase between smallest and largest levels. The cost of the test showed the smallest amount of difference with less than a twofold difference in preference between the low cost ($300) and high cost ($7000) levels. These comparisons are based on the assumption that the levels chosen for each attribute are of a similar magnitude of importance. Tests that offered a detection rate of less than 5% were significantly less probably to be selected compared to tests with a detection rate of 60% and 98% [odds ratio (OR) 7.8; 95% confidence interval [CI], 6.18, 9.91 and 7.2; 95% CI 5.74, 9.02, respectively]. However, the odds ratio for tests with a 60% detection rate did not significantly differ from those with a detection rate of 98%. Other attributes showed a more linear relationship with greater support for testing with increasing severity of the condition, increasing benefit of testing and decreasing cost. Minimum likelihood of disease
One hundred forty-six (77%) participants responded to the likelihood of disease question and responses ranged from
© 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd CLINICAL GENETICS doi: 10.1111/cge.12592
Social and Behavioural Research in Clinical Genetics Section Editor: Aad Tibben, email: [email protected]
Evaluating stakeholder’s perspective on referred out genetic testing in Canada: a discrete choice experiment Blumenschein P., Lilley M., Bakal J.A., Christian S. Evaluating stakeholder’s perspective on referred out genetic testing in Canada: a discrete choice experiment. Clin Genet 2016: 89: 133–138. © John Wiley & Sons A/S. Published by John Wiley & Sons Ltd, 2015 The expanding number and increasing utility of clinical genetic tests is creating a growing burden on the Canadian healthcare system. Administrators are faced with the challenge of determining which genetic tests should be publicly funded. A discrete choice experiment (DCE) was utilized to assess the importance stakeholders place on five attributes of a genetic test. One hundred ninety individuals completed the DCE questions. Analysis of the data revealed that medical benefit of a test had the greatest impact on a respondent’s decision to select a test for funding. The detection rate of the test ranked second in importance followed by severity of the condition, aim of the test, and cost. With limited resources available for referred out molecular genetic testing within a public healthcare setting such as Canada’s, funding guidelines are critical. Our findings provide further evidence for the value of a decision-making framework and the relative importance of specific test attributes within such a framework. Conflict of interest
The authors have no conflict of interest to declare.
New technologies, increased clinical utility and a greater awareness have resulted in higher demand for molecular genetic testing (1). Not all testing can be offered within any one jurisdiction resulting in a large number of samples being referred out for analysis. Each province or territory is responsible for the administration and financing of referred out molecular genetic testing. This has led to expanding costs which must be balanced against the need for fiscal responsibility within public health care systems. Little guidance is available to health care administrators on how molecular genetic testing should be prioritized. In Canada, allocation of resources for molecular genetic testing is regional and there is little consistency between jurisdictions with regard to the process or criteria used to make funding decisions. (2, 3)
P. Blumenscheina , M. Lilleya , J.A. Bakalb and S. Christiana a Genetic
Laboratory Services, Alberta Health Services, Edmonton, Canada and b Patient Health Outcomes Research and Clinical Effectiveness Unit, University of Alberta, Edmonton, Canada
Key words: discrete choice experiment – genetic testing – health care policy – resource allocation Corresponding author: Margaret Lilley, 826 Medical Sciences Building University of Alberta, Edmonton, AB T6G 2H7 Canada. Tel.: +780 407 1015 fax: +780 407 1761 e-mail: Margaret.Lilley@ albertahealthservices.ca Received 22 January 2015, revised and accepted for publication 27 March 2015
Discrete choice experiment (DCE) is a method used to elicit the preferences of a respondent. This approach is commonly used in economics and marketing as it allows for the analysis of preferences for goods or services with multiple attributes (4). It has more recently been used in the field of healthcare. In 2014, Severin et al. used this approach to assess the value stakeholders attach to various attributes of a genetic test in a European population (5). The DCE results revealed the relative importance of the test attributes in the decision-making process for funding a genetic test. This study used DCE to evaluate Canadian healthcare professionals’ views regarding molecular genetic testing and prioritization of funding. Stakeholders’ views on the relative importance of different test attributes could be
133
Blumenschein et al. useful information for health care administrators when creating policies for referred out testing. Methods Study design and participants
This was a cross-sectional study involving clinical genetic counsellors, medical geneticists, neurologists, endocrinologists, and hematologists from across Canada. In January 2014, participants were invited to complete an online survey which was developed using The Survey System Version 10 (6). The survey included demographic questions, a series of DCE questions and a question regarding minimum likelihood of disease cut-offs. Members of the Canadian Association of Genetic Counsellors (CAGC) and the Canadian College of Medical Geneticists (CCMG) were invited to participate through an e-mail sent out by their respective listservs. Members of the Colleges of Physicians and Surgeon of Alberta, Manitoba and Nova Scotia indicating a specialty in neurology, endocrinology or hematology were mailed invitations to participate with the option of responding via an electronic link or by mail. A self-addressed, postage paid envelope was included with the invitation. These three provinces were selected for the study because they have similar funding systems for referred out molecular genetic testing. All three provinces have formal application processes, have a committee to review the test requests, and allow non-genetics specialists to request funding (2). Neurologists, endocrinologists and hematologists were selected based on the authors’ experience with referred out molecular genetic testing in Alberta where, apart from geneticists, they were the most frequent requestors. In addition, Alberta physicians who had submitted an application for referred out molecular genetic testing were contacted by e-mail and invited to participate in the study. This allowed us to invite physicians who request referred out testing but were not members of the above specialties. Survey tool
and they had to decide which of the two tests should be publically funded. They could pick only one test to fund. A sample question is provided in Fig. 1. Each attribute included three levels which were defined at the beginning of the survey (Table 1). The survey design allowed a D-efficiency of 0.69. This D-efficiency indicates that this survey presented 69% of all possible test attribute combinations. In order to compare all possible combinations of attributes, the survey would have been impractical in length. Limiting the number of combinations and the number of questions, accordingly, was done to improve the response rate and reduce respondent fatigue. In the electronic version, DCE questions were presented in a random order to reduce potential bias related to decision fatigue. Three versions of the paper questionnaire with questions ordered differently were used to reduce bias. The survey was only available in English. The survey tool used was based largely on the tool developed and validated by Severin et al. (with permission from the authors) (5). The Severin survey included an attribute described as ‘risk of having the condition’ which was replaced in this survey with ‘detection rate of the test’. This modification was based on the authors experience with referred out molecular genetic testing. The cost levels were also modified to reflect estimates for familial mutation testing, single gene testing and large gene panels in current (2014) Canadian dollars. Likelihood question
The likelihood that an individual has a given condition can be an important factor in funding for molecular genetic testing. The detection rate of a given molecular testing can vary from 99% to less than 5% depending on the number of genes and the types of mutations that have been reported. Participants were asked to indicate on a sliding scale (0–100%) what they felt was an appropriate minimum likelihood of disease to justify funding a genetic test. They were told to assume that the test has a high detection rate and that genetic test results would impact medical management. This question was added to help with the comparison of our data with results by Severin et al. as they included the risk of disease as an attribute in their DCE questions (5). Statistical analysis
Demographics
The first part of the survey instrument gathered information regarding age, gender, province of practice, specialty and if the participant had previously ordered referred out molecular genetic testing. Discrete choice experiment
Nineteen DCE questions were used to assess the importance placed on five attributes of a molecular genetic test. Attributes included severity of the condition, aim, benefit, detection rate and cost of the test. The primary benefit of the test may be to the patient directly or to their family members. Participants were asked to imagine that they were the decision maker in a health care organization
134
Demographic variables were summarized using counts and chi-squared tests or Fisher exact tests where appropriate. The test descriptor variables in the DCE questions were effect coded to account for possible non-linear effects on choice, for both the selected and unselected tests. The chosen test was given a response of 1 and the non-selected test was assigned a 0. The DCE questions were evaluated using a series of mixed logit models which take into account the respondents variability in value judgments and comparison of parameters in each question. The models can then be evaluated using adjusted odds ratios based on the estimated coefficients. Additional demographic parameters were dummy coded and included in the models as interaction terms to
Evaluating stakeholder’s perspective on referred out genetic testing
Fig. 1. Sample discrete choice experiment (DCE) question.
evaluate effects. All analyzes were conducted using SAS 9.4 (Cary, NC). Results
In total, 993 individuals were invited to participate in the study. One hundred ninety individuals responded to the survey (19%). Fifty-three (28%) paper surveys were completed and the remaining 137 (72%) surveys were completed electronically. Respondent demographics
Areas of practice included Genetics (n = 110, 58%), Neurology (n = 42, 23%), Hematology (n = 13, 7%), Endocrinology (n = 10, 5%) and Other (n = 15, 8%). The greatest number of respondents practiced in Alberta and Ontario, 78 (41%) and 46 (24%), respectively. Alberta was disproportionately represented as a greater number of Alberta physicians were invited to participate compared to other provinces. There may also be an Alberta bias given the location of the authors. Demographics are summarized in Table 2. Preferred genetic testing attributes
Analysis of the DCE responses showed that medical benefit of a test had the greatest impact on respondents’ decision to select a test to be funded. Proven medical benefit showed a 26-fold increase in odds of being chosen over no medical benefit (Table 3). The detection rate of the test ranked second in importance with a sevenfold difference between the largest and smallest rate, followed by severity of the condition, and aim of the test with
around a twofold increase between smallest and largest levels. The cost of the test showed the smallest amount of difference with less than a twofold difference in preference between the low cost ($300) and high cost ($7000) levels. These comparisons are based on the assumption that the levels chosen for each attribute are of a similar magnitude of importance. Tests that offered a detection rate of less than 5% were significantly less probably to be selected compared to tests with a detection rate of 60% and 98% [odds ratio (OR) 7.8; 95% confidence interval [CI], 6.18, 9.91 and 7.2; 95% CI 5.74, 9.02, respectively]. However, the odds ratio for tests with a 60% detection rate did not significantly differ from those with a detection rate of 98%. Other attributes showed a more linear relationship with greater support for testing with increasing severity of the condition, increasing benefit of testing and decreasing cost. Minimum likelihood of disease
One hundred forty-six (77%) participants responded to the likelihood of disease question and responses ranged from
