CSIRO PUBLISHING
Australian Journal of Primary Health, 2015, 21, 254–258 http://dx.doi.org/10.1071/PY13129
Research
Recruiting general practitioners for surveys: reflections on the difficulties and some lessons learned Anne Parkinson A,F, Louisa Jorm B,C, Kirsty A. Douglas D, Alison Gee A, Ginny M. Sargent A,E, Sanja Lujic B and Ian S. McRae A A
Australian Primary Health Care Research Institute, The Australian National University, Building 63, Corner Mills and Eggleston Roads, Canberra, ACT 0200, Australia. B Centre for Health Research, Building 3, Campbelltown Campus, University of Western Sydney, Locked Bag 1797, Penrith, NSW 2751, Australia. C Sax Institute, PO Box K617, Haymarket, NSW 1240, Australia. D Australian National University Medical School, College of Medicine, Biology and Environment, Building 42, Linnaeus Way, Canberra, ACT 0200, Australia. E National Centre for Epidemiology and Population Health, The Australian National University, Building 62, Corner Mills and Eggleston Roads, Canberra, ACT 0200, Australia. F Corresponding author. Email: [email protected]
Abstract. Surveys of GPs are essential to facilitate future planning and delivery of health services. However, recruitment of GPs into research has been disappointing with response rates declining over recent years. This study identified factors that facilitated or hampered GP recruitment in a recent survey of Australian GPs where a range of strategies were used to improve recruitment following poor initial responses. GP response rates for different stages of the survey were examined and compared with reasons GPs and leaders of university research networks cited for non-participation. Poor initial response rates were improved by including a questionnaire in the mail-out, changing the mail-out source from an unknown research team to locally known network leaders, approaching a group of GPs known to have research and training interests, and offering financial compensation. Response rates increased from below 1% for the first wave to 14.5% in the final wave. Using a known and trusted network of professionals to endorse the survey combined with an explicit compensation payment significantly enhanced GP response rates. To obtain response rates for surveys of GPs that are high enough to sustain external validity requires an approach that persuades GPs and their gatekeepers that it is worth their time to participate. Additional keywords: general practice, health-care surveys, primary health care, survey recruitment. Received 12 September 2013, accepted 21 January 2014, published online 3 March 2014
Introduction Understanding GPs’ knowledge, attitudes, and behaviours is essential for future planning and delivery of health services (Bonevski et al. 2011). However, recruitment and retention of GPs into research has been disappointing in Australia and elsewhere (Veitch et al. 2001; Mant et al. 2004; Harris et al. 2005; James and Talbot 2005; Jones et al. 2012). Survey response rates of GPs are lower than those of the general population (Asch et al. 1997), and there has been a decline in response rates over recent years (Aitken et al. 2008; GravaGubins and Scott 2008; Cook et al. 2009; Crouch et al. 2011). This is of concern as the generalisability of survey results to inform policy and practice decision-making might be compromised (Scott et al. 2011). The literature on maximising GP response rates is consistent with Dillman’s Total Design Approach: provision of a financial Journal compilation La Trobe University 2015
incentive, respondent friendly questionnaires, four contacts by mail, one by phone, reply paid envelopes and personalisation (Field et al. 2002; Harris et al. 2005; VanGeest et al. 2007; Edwards et al. 2009). In the Australian context, Harris et al. (2005) recommend the value of follow up by telephone, multiple reminders, incentives, and personal contact before the survey with either the GP or practice manager. Therefore, surveying GPs is labour intensive and costly, especially for the large numbers often required. Common barriers to GP participation and retention identified in the literature include: GPs having little input into research design; concern about misuse of patient data; scepticism about the value of the research; survey overload; lack of time and results that are not locally relevant (Yallop et al. 2006; GravaGubins and Scott 2008; Jones et al. 2012). The effect of non-GP staff on response rates is another hurdle mail-based www.publish.csiro.au/journals/py
Recruiting GPs for surveys
What is known about the topic? *
Surveys of GPs are necessary to facilitate future planning and delivery of health services; however, recruitment of GPs into research has been disappointing, with response rates declining over recent years.
What does this paper add? *
We show that survey design must consider trade-offs between the use of targeted sample frames, which might be biased but will generate higher GP response rates, and more complete sample frames where high response is difficult.
approaches face because these ‘gatekeepers’ might choose not to pass on a survey (VanGeest et al. 2007; Thorpe et al. 2009; Scott et al. 2011; Ziegenfuss et al. 2012). This study reports on response patterns to a recent survey of Australian GPs and their patients about primary care (the survey reported on in this paper was part of an international study, Quality and Costs of Primary Care in Europe (QUALICOPC), to evaluate primary care systems in 35 countries (Schäfer et al. 2011)). The approach taken in the first instance attempted to replicate the approach taken by other countries to obtain a nationally representative sample of GP practices. However, initial GP response rates were poor and a range of strategies was used to improve recruitment. This paper identifies factors that facilitated or hampered survey recruitment, and the costs and risks to representativeness of the various strategies used. Methods Survey data collection The Australian arm of the QUALICOPC study was a cooperative venture between the Australian National University and the University of Western Sydney (Ethical approvals 2011/584 and H9493, respectively). The recruitment target for Australia was 220 GPs. The survey comprised two stages. The first asked GPs about the organisation and procedures of their practice, and aspects of workload (30–40 min). The second asked GPs to allow fieldworkers to seek survey responses from 10 patients in their waiting rooms regarding patients’ health-care experiences (15 min). Standard international QUALICOPC questionnaires were adapted to Australian structures, and pre-tested with a small group of GPs. In line with the literature, strategies to encourage participation throughout the study included personalised invitations, endorsement from professional bodies, reply paid envelopes and comprehensive information sheets. In the first recruitment round, a sample of 2496 GPs was selected from the Australasian Medical Publishing Company (AMPCo) database of mailing addresses, which covers the majority of GPs in Australia. We randomly selected one GP per practice, then stratified practices into five groups based on practice size (1, 2, 3, 4, or 5 or more GPs), and proportionally sampled GPs from those strata. Following QUALICOPC practice, the initial mail-out included a ‘registration of interest’ form for GPs to
Australian Journal of Primary Health
255
request a questionnaire or to refuse the invitation to participate. A reminder letter and paper copy of the questionnaire were then sent to non-respondents advising continuing professional development (CPD) points could be claimed and also offering an online option. The low initial response prompted a second round of recruitment. Fourteen Australian university primary health-care research and training networks were invited to participate in the project, with seven agreeing to participate. One university (Network 7) opted to utilise their ‘Medicare Local’ (a practicebased network), as they had developed a close working relationship. A sample of 716 GPs from seven research and training networks were invited to participate. A covering letter tailored by each network on university stationery endorsed the study and encouraged GPs to participate. A paper copy of the questionnaire was included. The invitation offered GPs a $100 gift card as compensation for their time in conjunction with CPD points on completion of both the GP and patient surveys. Reasons for non-participation were sought from a random sample of 20 GPs in the first stages of the survey, and from managers of both participating and non-participating university networks.
Results The survey was undertaken between March 2012 and April 2013. The characteristics of GP respondents who completed the questionnaire are shown in Table 1. The gender split was broadly similar to that reported for practicing Australian GPs, as was the urban/rural structure (DOHA 2012), although younger GPs were under-represented amongst respondents. The targeted (second round) samples were significantly more likely to be in metropolitan areas where most networks were based, and while differences were not significant, the targeted sample was more likely to be aged between 45 and 64 years compared with the general (first round) sample.
First recruitment round Survey responses are shown in Table 2. The initial survey mailout to 2496 GPs resulted in 21 GP completed questionnaires. Only 13 of these GPs completed the patient fieldwork component. An additional 87 GPs either requested no further participation, or were no longer at the address to which the survey was sent. Following the distribution of a reminder letter with a paper copy of the questionnaire attached and provision for online completion, a further 57 GPs (2.4% of the initial sample) returned completed questionnaires, with seven completed online. The patient fieldwork component was completed by 39 of these GPs (1.6% of the initial sample).
Reasons given for non-response Follow-up telephone calls to 20 randomly selected GPs who had not returned surveys after the first recruitment round revealed most had not seen the invitation or they were too busy to participate in the survey.
256
Australian Journal of Primary Health
A. Parkinson et al.
Table 1. Characteristics of GP respondents compared with National data for Australian GPs GP respondents, mail-out 1 (%)
GP respondents, mail-out 2 (%)
Australian GPs (DOHA 2012) (%)
Gender Male Female
64.3 35.7
63.1 36.9
58.3 41.7
Age (years)
Australian Journal of Primary Health, 2015, 21, 254–258 http://dx.doi.org/10.1071/PY13129
Research
Recruiting general practitioners for surveys: reflections on the difficulties and some lessons learned Anne Parkinson A,F, Louisa Jorm B,C, Kirsty A. Douglas D, Alison Gee A, Ginny M. Sargent A,E, Sanja Lujic B and Ian S. McRae A A
Australian Primary Health Care Research Institute, The Australian National University, Building 63, Corner Mills and Eggleston Roads, Canberra, ACT 0200, Australia. B Centre for Health Research, Building 3, Campbelltown Campus, University of Western Sydney, Locked Bag 1797, Penrith, NSW 2751, Australia. C Sax Institute, PO Box K617, Haymarket, NSW 1240, Australia. D Australian National University Medical School, College of Medicine, Biology and Environment, Building 42, Linnaeus Way, Canberra, ACT 0200, Australia. E National Centre for Epidemiology and Population Health, The Australian National University, Building 62, Corner Mills and Eggleston Roads, Canberra, ACT 0200, Australia. F Corresponding author. Email: [email protected]
Abstract. Surveys of GPs are essential to facilitate future planning and delivery of health services. However, recruitment of GPs into research has been disappointing with response rates declining over recent years. This study identified factors that facilitated or hampered GP recruitment in a recent survey of Australian GPs where a range of strategies were used to improve recruitment following poor initial responses. GP response rates for different stages of the survey were examined and compared with reasons GPs and leaders of university research networks cited for non-participation. Poor initial response rates were improved by including a questionnaire in the mail-out, changing the mail-out source from an unknown research team to locally known network leaders, approaching a group of GPs known to have research and training interests, and offering financial compensation. Response rates increased from below 1% for the first wave to 14.5% in the final wave. Using a known and trusted network of professionals to endorse the survey combined with an explicit compensation payment significantly enhanced GP response rates. To obtain response rates for surveys of GPs that are high enough to sustain external validity requires an approach that persuades GPs and their gatekeepers that it is worth their time to participate. Additional keywords: general practice, health-care surveys, primary health care, survey recruitment. Received 12 September 2013, accepted 21 January 2014, published online 3 March 2014
Introduction Understanding GPs’ knowledge, attitudes, and behaviours is essential for future planning and delivery of health services (Bonevski et al. 2011). However, recruitment and retention of GPs into research has been disappointing in Australia and elsewhere (Veitch et al. 2001; Mant et al. 2004; Harris et al. 2005; James and Talbot 2005; Jones et al. 2012). Survey response rates of GPs are lower than those of the general population (Asch et al. 1997), and there has been a decline in response rates over recent years (Aitken et al. 2008; GravaGubins and Scott 2008; Cook et al. 2009; Crouch et al. 2011). This is of concern as the generalisability of survey results to inform policy and practice decision-making might be compromised (Scott et al. 2011). The literature on maximising GP response rates is consistent with Dillman’s Total Design Approach: provision of a financial Journal compilation La Trobe University 2015
incentive, respondent friendly questionnaires, four contacts by mail, one by phone, reply paid envelopes and personalisation (Field et al. 2002; Harris et al. 2005; VanGeest et al. 2007; Edwards et al. 2009). In the Australian context, Harris et al. (2005) recommend the value of follow up by telephone, multiple reminders, incentives, and personal contact before the survey with either the GP or practice manager. Therefore, surveying GPs is labour intensive and costly, especially for the large numbers often required. Common barriers to GP participation and retention identified in the literature include: GPs having little input into research design; concern about misuse of patient data; scepticism about the value of the research; survey overload; lack of time and results that are not locally relevant (Yallop et al. 2006; GravaGubins and Scott 2008; Jones et al. 2012). The effect of non-GP staff on response rates is another hurdle mail-based www.publish.csiro.au/journals/py
Recruiting GPs for surveys
What is known about the topic? *
Surveys of GPs are necessary to facilitate future planning and delivery of health services; however, recruitment of GPs into research has been disappointing, with response rates declining over recent years.
What does this paper add? *
We show that survey design must consider trade-offs between the use of targeted sample frames, which might be biased but will generate higher GP response rates, and more complete sample frames where high response is difficult.
approaches face because these ‘gatekeepers’ might choose not to pass on a survey (VanGeest et al. 2007; Thorpe et al. 2009; Scott et al. 2011; Ziegenfuss et al. 2012). This study reports on response patterns to a recent survey of Australian GPs and their patients about primary care (the survey reported on in this paper was part of an international study, Quality and Costs of Primary Care in Europe (QUALICOPC), to evaluate primary care systems in 35 countries (Schäfer et al. 2011)). The approach taken in the first instance attempted to replicate the approach taken by other countries to obtain a nationally representative sample of GP practices. However, initial GP response rates were poor and a range of strategies was used to improve recruitment. This paper identifies factors that facilitated or hampered survey recruitment, and the costs and risks to representativeness of the various strategies used. Methods Survey data collection The Australian arm of the QUALICOPC study was a cooperative venture between the Australian National University and the University of Western Sydney (Ethical approvals 2011/584 and H9493, respectively). The recruitment target for Australia was 220 GPs. The survey comprised two stages. The first asked GPs about the organisation and procedures of their practice, and aspects of workload (30–40 min). The second asked GPs to allow fieldworkers to seek survey responses from 10 patients in their waiting rooms regarding patients’ health-care experiences (15 min). Standard international QUALICOPC questionnaires were adapted to Australian structures, and pre-tested with a small group of GPs. In line with the literature, strategies to encourage participation throughout the study included personalised invitations, endorsement from professional bodies, reply paid envelopes and comprehensive information sheets. In the first recruitment round, a sample of 2496 GPs was selected from the Australasian Medical Publishing Company (AMPCo) database of mailing addresses, which covers the majority of GPs in Australia. We randomly selected one GP per practice, then stratified practices into five groups based on practice size (1, 2, 3, 4, or 5 or more GPs), and proportionally sampled GPs from those strata. Following QUALICOPC practice, the initial mail-out included a ‘registration of interest’ form for GPs to
Australian Journal of Primary Health
255
request a questionnaire or to refuse the invitation to participate. A reminder letter and paper copy of the questionnaire were then sent to non-respondents advising continuing professional development (CPD) points could be claimed and also offering an online option. The low initial response prompted a second round of recruitment. Fourteen Australian university primary health-care research and training networks were invited to participate in the project, with seven agreeing to participate. One university (Network 7) opted to utilise their ‘Medicare Local’ (a practicebased network), as they had developed a close working relationship. A sample of 716 GPs from seven research and training networks were invited to participate. A covering letter tailored by each network on university stationery endorsed the study and encouraged GPs to participate. A paper copy of the questionnaire was included. The invitation offered GPs a $100 gift card as compensation for their time in conjunction with CPD points on completion of both the GP and patient surveys. Reasons for non-participation were sought from a random sample of 20 GPs in the first stages of the survey, and from managers of both participating and non-participating university networks.
Results The survey was undertaken between March 2012 and April 2013. The characteristics of GP respondents who completed the questionnaire are shown in Table 1. The gender split was broadly similar to that reported for practicing Australian GPs, as was the urban/rural structure (DOHA 2012), although younger GPs were under-represented amongst respondents. The targeted (second round) samples were significantly more likely to be in metropolitan areas where most networks were based, and while differences were not significant, the targeted sample was more likely to be aged between 45 and 64 years compared with the general (first round) sample.
First recruitment round Survey responses are shown in Table 2. The initial survey mailout to 2496 GPs resulted in 21 GP completed questionnaires. Only 13 of these GPs completed the patient fieldwork component. An additional 87 GPs either requested no further participation, or were no longer at the address to which the survey was sent. Following the distribution of a reminder letter with a paper copy of the questionnaire attached and provision for online completion, a further 57 GPs (2.4% of the initial sample) returned completed questionnaires, with seven completed online. The patient fieldwork component was completed by 39 of these GPs (1.6% of the initial sample).
Reasons given for non-response Follow-up telephone calls to 20 randomly selected GPs who had not returned surveys after the first recruitment round revealed most had not seen the invitation or they were too busy to participate in the survey.
256
Australian Journal of Primary Health
A. Parkinson et al.
Table 1. Characteristics of GP respondents compared with National data for Australian GPs GP respondents, mail-out 1 (%)
GP respondents, mail-out 2 (%)
Australian GPs (DOHA 2012) (%)
Gender Male Female
64.3 35.7
63.1 36.9
58.3 41.7
Age (years)
