Family Practice, 2015, 101–105 doi:10.1093/fampra/cmu081 Advance Access publication 30 November 2014
Qualitative Research
Exploring GPs’ experiences of using diagnostic tools for cancer: a qualitative study in primary care Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
Trish Greena,*, Tanimola Martinsb,William Hamiltonb, Greg Rubinc, Kathy Elliottd and Una Macleoda Centre for Health and Population Sciences, Hull York Medical School, University of Hull, bUniversity of Exeter Medical School, cSchool of Medicine, Pharmacy and Health, University of Durham, and dkaeconsulting, London, UK. a
*Correspondence to Trish Green, Centre for Health and Population Sciences, Hull York Medical School, University of Hull, Hertford Building, Cottingham Road, Hull, HU6 7RX, UK. E-mail: [email protected]
Abstract Background. The UK has an estimated 5–10 000 extra cancer deaths each year when compared to other European countries and diagnostic delays are thought to make a significant contribution to this. One of the initiatives in England intended to support primary care professionals has been the development of cancer risk assessment tools (RATs). These tools assist in identifying and quantifying the risk of cancer in symptomatic primary care patients. Objective. To explore GPs’ experiences of incorporating the RATs for lung and bowel cancers into their clinical practice and in so doing, identify constraints and facilitators to the wider dissemination of the tools in primary care. Methods. We conducted semi-structured interviews over the telephone with 11 project managers who implemented the study and 23 GPs who used the tool. The interviews were digitally recorded, professionally transcribed verbatim and analysed through the construction of a ‘thematic framework’. Results. The training and support package was fundamental to the successful integration of the RATs into GPs’ daily routines. Ongoing support from cancer networks alongside acknowledgement of the clinical expertize of the GPs by those implementing the study enhanced GPs’ uptake of the tool in practice. Conclusion. Findings suggest that the embedding of clinical decision support tools into clinical practice is more likely to be achieved when they are perceived to support but not supersede the clinical judgement of their users. This element of our findings is a focal point of this article. Key words: Clinical decision support tools, early detection of cancer, general practitioners, primary health care, qualitative research.
Introduction Diagnosing cancer when at an early stage is important as stage of disease at diagnosis is related to survival for many cancers. The UK, however, has relatively poor cancer outcomes when compared with other European countries (1–3) with 5–10 000 extra cancer deaths a year (4,5). Diagnostic delay is considered a major contributor to
© The Author 2014. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: [email protected]
this and primary care has been identified as one arena wherein delay might occur. In the UK, patients with symptoms suggestive of cancer generally present to their GP in the first instance; it thus follows that part of the solution to delayed diagnosis could lie within the UK’s primary care system. To date however, no interventions have been shown to reduce time to diagnosis of cancer in primary care (6). A key task for GPs is to recognize potential cancer symptoms and 101
Family Practice, 2015, Vol. 32, No. 1
102
Methods RATs materials Desk-based RATs consist of tables containing the risk score for each symptom in isolation, for repeat presentation of the same symptom, and in combination with one other symptom (5). The tables were reproduced on mousemats and A5 desk easels, which functioned as aids for GPs when considering the likelihood of an individual patient aged 40 or over having either lung or colorectal cancer, given the symptom or combination of symptoms they presented with. Seven Cancer Networks, three of them London Networks working together, were involved in the study, which provided ~175 GP practices; the RATs materials were distributed to all 175.
Interviews A total of 47 invitations to interview were sent out across the seven networks via email and telephone and of these 34 individuals consented: 11 Project Managers; 8 GP Cancer Leads (GP cancer leads were part of the clinical leadership teams in Cancer Networks, with responsibility for supporting activities to assist with cancer diagnosis in practices in their locality using peer support and educational techniques) and 15 GPs. GP Cancer Leads were involved in the study in a supportive role and also used the RATs in their practice, making the total number of RATs users 23. All interviews were conducted by the first author between July and October, 2011. Prior to interview, all participants provided signed consent to its being recorded and our use of the data for analysis and reporting. Consent was also checked verbally at the time of each telephone interview. Topic guides for the interviews were designed to enable the capture of focussed data through a mixture of specific and open-ended questions. Project managers’ interviews focused on how the RATs were introduced to practices; GPs’ interviews explored experiences of the implementation process and their use of the RATs in practice. Interviews lasted ~30 minutes and ‘topic-relevant content (16)’ was ensured through the use of a semi-structured topic guide that enabled the elicitation of a broad range of responses. All interviews were professionally transcribed verbatim. Participants were made aware that data would not be reported in a way that would identify them individually. Practices were given some financial support via the Cancer Networks to conduct this project. We have attributed an acronym and number system to participants as follows: PM/1–11; GPL/1–8; GP/1–15. The study was reviewed and approved by the Hull York Medical School Ethics Committee.
Data analysis Because of the applied nature of our inquiry, the interview schedule was constructed pragmatically in order to determine firstly how the RATs study was implemented, secondly how GPs perceived this process and thirdly, GPs’ experiences of using the RATs. All transcripts were imported into the software package N-Vivo, which assisted the analytic process (17). To increase the clarity of the coding and the breadth of the analysis, transcriptions were double-read, co-coded and cross-checked (UM, TG). Emergent themes and sub-categories were identified, from which a coding framework (18) was created to represent different aspects of GPs’ experiences of the implementation process and GPs’ use of the tools in practice. Following Spencer et al.’s ‘code and retrieve’ model (19), each coded group was condensed and summarized and the emerging concepts fed into initial categories. The analytical process we describe here initially responded to the questions asked at interview. As our analysis progressed, a more nuanced understanding of the findings from our data emerged regarding how integration of the tools was achieved. We shall argue here that these findings are crucial for the future development and usability of RATs, and indeed for that of CDS tools in primary care more generally.
Results The data discussed in this article emerge from a qualitative study that was embedded within an evaluation that incorporated quantitative and qualitative methodologies. The quantitative evaluation demonstrated increased diagnostic activity and additional cancer diagnoses over the time of the study, which suggests that the RATs were effective in helping GPs with referral decisions for some patients (20). However, the quantitative data cannot tell us why the tools were
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
initiate appropriate investigation. To ensure that primary care does not contribute to delays, work needs to be directed at each of the following: improving recognition of symptoms, ensuring appropriate referral/investigation pathways are available and accessible, and ensuring responsive systems exist in specialist care. This article is concerned with the first aspect of this process—recognition of symptoms by GPs. Most cancer symptoms are also symptoms of benign conditions—and these benign conditions are much more common (7). When patients present with symptoms to primary care, GPs must judge the individual patient’s likelihood of cancer and select those whose symptom profile is reasonably suggestive of cancer to warrant further investigation or specialist referral. Support for cancer symptom recognition is available for GPs in the format of national guidance published by the National Institute for Health and Care Excellence [NICE (8)]. An urgent GP referral pathway for suspected cancer, the 2-week referral scheme, is well established. This describes symptom patterns for cancer that warrant urgent investigation (sometimes called ‘red flag’ symptoms), but many patients with cancer do not have symptoms that meet the criteria for its use. As the NICE guidance was based on contemporary available evidence largely derived from secondary care populations, interest in recent years has moved to obtaining evidence on presenting symptoms from primary care populations. Hamilton identified symptoms predictive of cancer in primary care cohort studies, which led to the development of risk estimates for patients aged over 40 presenting to primary care with symptoms of possible cancer; for single symptoms, pairs of symptoms and repeat attendances with the same symptom. These were presented as colour-coded risk assessment tools (RATs), for colorectal and lung cancers (5,9,10). They are now available for thirteen cancer sites. Although clinical decision support (CDS) is not a new concept (11), the development of integrated tools for the diagnosis of cancer is relatively novel. One relevant consideration is the way in which a newly introduced CDS tool becomes integrated into routine practice. Much of the theoretical underpinning for this is derived from the work of May and colleagues (12–15) who have developed normalization process theory to seek to understand and explain what it is that underscores the successful, or otherwise, incorporation of complex interventions into clinical practice. Undoubtedly, the ultimate purpose of any diagnostic tool is for it to become a part of clinical routines, that is, to become normalized, and instigate behaviour changes that prompt actions at a practice level that have a positive effect on patients’ health outcomes. This article reports on if and how the desk-based RATs for lung and colorectal cancer were integrated into routine practice within the context of this study.
GPs’ experiences of using diagnostic tools for cancer effective, or whether all participating GPs found them useful. We are also unable to gauge how GPs used the RATs in order for them to prove successful, or otherwise. We would argue that the answers to these kinds of queries hold the key to whether the RATs, and indeed CDS tools more generally, would succeed if disseminated beyond the parameters of a research project. The overall aim of the qualitative study was thus to explore GPs’ experiences of using the RATs to ascertain their acceptability and usability within the context of primary care practice. As will become clear in what follows, to meet our aim it was also necessary to consider the ways in which the study was implemented. We discuss our findings here under three main headings: firstly, the process of implementation; secondly, identifying signs and symptoms; and thirdly, changing practice.
The process of implementation
It was very professionally done at the beginning; we were well supported. It was not a particularly straightforward project. The team must’ve put in the groundwork, basically, and that’s really good, because we do get stuff coming through that’s just not being thought through and is a real waste. It’s a real disincentive then to get involved. (GP/10) I had quite a lot of requests to go down, um, to explain to them how to use the tool. It actually gave them an opportunity if they had any questions to ask them there and then and they felt more confident in using the tool, because when they got the presentations, they weren’t clear on quite a lot of the things. (PM/7)
The supportive activities that accompanied the implementation process emerged as key to GPs’ acceptance and perhaps more crucially, accurate reading of the tools. This approach resonates with Murray et al.’s (14) suggestion that clinicians need to not only familiarize themselves with decision making tools, but also grasp the principles that underpin them. What our findings so far suggest is that the RATs training and continued interaction between the study team and RATs users created a supportive environment that was seemingly integral to the successful streamlining of the tools into practice for the duration of the project. Clearly, the proven efficacy of a diagnostic tool when in its development phase does not guarantee acceptability on the part of its intended users. Rather, what becomes apparent from our discussion is that the implementation process could prove to be what ‘makes or breaks’ it in terms of successful uptake by practitioners.
Identifying signs and symptoms Guidance from NICE is in place to assist GPs in the referral of red flag cancer symptoms and the 2-week wait rule (2WW) ensures the
timeliness of secondary care consultations. As we described earlier, however, the nuanced nature of cancer symptoms presents one of the main stumbling blocks for the referral of some patients to secondary care: The barn door ones are a piece of cake, it’s just the grey area in the middle that we struggle with, patients coming along with symptoms that we aren’t certain about. (GPL/1)
The notion of ‘barn door’ symptoms was often juxtaposed with patients whose symptoms did not meet 2WW criteria; this latter group of patients caused study participants a great deal of anxiety as their pathway to investigation poses more difficulties: It [RATs] helped us with patients, not the red flag symptoms, but the kind of yellow flag symptoms, the under-symptoms that you think, you know, should I be worried about this patient or not? Whereas the NICE guidance really tells you much more about the red flag symptoms, which we should all have embedded in us already. (GP/5)
Our data revealed that some GPs used the RATs to support their referral decisions when patients’ symptoms did not meet 2WW referral criteria: You could actually say, although it doesn’t fulfil the two-week wait guidelines, the RAT score is high for this patient therefore we would like to bump it up to a two-week wait. (GP/4)
Findings suggest that using the RATs encouraged GPs to think differently about their patients’ symptoms; effectively to consider cancer first and then, as consultation proceeded, to either rule out cancer as a potential diagnosis or take additional diagnostic action, perhaps helping to reduce some of the uncertainty characteristic of decision making in general practice (21). A major benefit of the RATs was then that they alerted GPs to the potential significance of more opaque signs and symptoms of cancer. Significantly, GPLs made similar comments: It made me understand that certain symptoms when put with others actually do have more relevance than perhaps I thought they did. I think that’s a learning point in itself. (GPL/4) What the tool does really is make you ask the right questions if you think of cancer ... it made you better at making the decision, so I don’t think our two week referrals shot up dramatically, but everyone thought a little bit more about cancer when they were taking [patients’] histories. (GPL/5)
Evidently, the RATs assisted in the recognition of symptoms that previously might not have raised alarm, which is of course an intended function of the tools. GPs also perceived that sharing the tool during consultation helped them explain to patients why they would be referred, or not, to secondary care: I would use it alongside a patient, so if someone comes in and says, I have got a cough, I would be able to glance down to my mousemat and say, ‘and when you’ve been coughing, you know, have you been bringing up any blood?’, and just making myself think about looking at whether they’ve had any spirometry done and, you know, just using it as a prompt in my mind as to making sure I’m covering all bases. Having those charts was just very, very useful from that point of view. (GP/9)
Our findings indicate that for the majority of GPs we interviewed, RATs were useful in consultation, heightened GPs’ awareness of potential cancer symptoms, reminded and alerted them to potential cancer risks and affected their referral thresholds. As such, the majority of RATs users perceived the tools to be valuable additions to practice.
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
The introduction of the study to participating practices involved teams from the cancer networks distributing the RATs materials (mousemats/desk easels) with accompanying literature (5) and providing training on how the tools were to be used, that is, how to read each of the three tables. In most instances, training was carried out remotely via webcasts that GPs accessed in situ. Once trained, GPs were tasked to cascade the information to colleagues unable to attend training sessions. Networks also emailed updates and newsletters to the practices. Murray et al. (14) endorse these types of communication device because they were found to reinforce the importance of participation and instilled a sense of collective action for study participants. GPs were also encouraged to contact their respective network team if they encountered any problems, requests that were often responded to in person. Several project managers outlined the value of visits to practices to ensure GPs understood how to use the tools effectively:
103
Family Practice, 2015, Vol. 32, No. 1
104
Changing practice Although by dint of their use, the RATs will inevitably change practice, they were not designed to supersede GPs’ clinical judgment (the instructions on the tools state ‘use your clinical judgment’); indeed as GP/3 asserted, ‘the clinical judge is the doctor’. Interview data highlight how the implementation process was underpinned by this philosophy, which seemed to hold another key to the successful embedding of the RATs during the study period: We have stressed throughout the campaign that you shouldn’t use it above and beyond your clinical suspicion, you want GPs to still be using their clinical acumen, you can’t just drive by protocols, things don’t always fit a box. (GPL/2)
Just having those on our desk has made us think a little bit wider about what symptoms may be suggestive of cancer, looking at combining symptoms, rather than thinking about single symptoms on their own. (GP/5) It’s a fantastic aide memoire and prompt to have, you know, physically in front of you while you’re in the middle of a consultation. (GP/9)
The majority of GPs spoke positively regarding the aide memoire quality of the RATs materials, indicative perhaps of the tools’ ease of integration into daily practice and consultation, as well as its being a stimulus for discussion among colleagues and patients. It might be as well that through their engagement with the RATs, GPs became what Mansell et al. would describe as ‘sensitised to cancer (6)’, which in itself is probably helpful: I’ve been a GP for about 14, 15 years now ... there aren’t many things that change your practice, so it’s good when you come across things that do. (GP/8)
It is well accepted that experience and intuition play a part in the diagnosis of cancer (21,22). As well as this, GPs at times refer patients who do not meet guidelines, and conversely sometimes do not refer patients who do meet guidelines (22). What our data demonstrate is that the majority of GPs perceived that the RATs supported their clinical decision-making. As such, they were perhaps more easily incorporated as part of the mix that makes up the diagnostic process: You rely on your gut instinct to refer, but that’s where I could find this being more of . . . an algorithm that would help you with patients that you feel need the two week rule, but haven’t got quite the diagnostic criteria that fits. (GP/4) Sometimes it worked as a very reassuring tool; when you have someone who’s absolutely convinced they’ve got lung cancer or they’ve got bowel cancer, I would then get [the RAT] out in consultation and discuss it with the patient and say, this is all the reasons why I don’t have a bad feeling about your symptoms. (GP/2)
Evidently, the RATs were accepted by GPs as an additional element of the decision-making process. Although there were instances when GPs found the RATs to be at odds with referral guidelines, which in its turn created some difficulties with secondary care colleagues, overall the majority of participants felt that the tools complemented NICE and other cancer referral guidance and so strengthened and validated their referral decisions.
I just found the tables didn’t help hurry the process on at all. (GP/1) We’ve been doing all these things before this RAT thing came anyway. (GP/14) What I’m not sure it does for me is sort of fits into the consultation in a natural way of making a decision about the management of that patient as yet. It’s one more thing to fit into a busy ten minute consultation. (GPL/1)
There were, then, some participants who were not convinced that the RATs were valuable additions to general practice. GPs across the study sample, those ‘for’ and those ‘against’ RATs, also talked about the amount of information they received on a daily basis in the form of guidelines, letters, emails and so forth. This coupled with a less than enthusiastic response from potential RATs users, could prove problematic for the implementation of tools that intend to bring about new ways of working, not least in terms of keeping up the momentum needed to ensure GPs engage with new diagnostic aids.
Discussion Using the RATs led the majority of the GPs and GPLs we interviewed to make changes to their given ways of practice which, as data demonstrate, was perceived as beneficial. In several instances, use of the tools confirmed a decision to refer that, although GPs stated would have been made anyway, was perceived to give more credibility to that decision. For some participants, using the tools urged referrals that might not have been made; similarly at times the RATs were used to confirm a GP’s decision not to refer. Significantly then, the RATs seemed to meet the need for support in lung and colorectal cancer symptom recognition in primary care. In addition it was clear that the tools encouraged debate among study participants and their peers. Moreover, while GPs welcomed them as instructive tools, the RATs did not supersede clinical judgement; rather the tools were perceived as additives to GPs’ skills sets. The RATs were developed to address primary care delays in cancer symptom recognition and referral and our findings respond to the call from Mansell et al. who have argued the need for additional research ‘to determine which type of intervention is most effective in reducing the time for referral from primary to secondary care (6)’. The acceptability of the RATs was enhanced by them being derived from a primary care source and because there was a strong primary care involvement in the delivery of the study. Indeed, one of our major findings is that GPs felt well supported throughout and, importantly, felt their clinical expertize was acknowledged and valued by the team implementing the study. It is unclear from our data whether or not the tool was ‘normalized’ into practice (12–15). Indeed the study was not set up to address that question as this would necessitate further investigation into the extent of the RATs’ use beyond the study’s timeframe. What we can say is that where the tool appeared to be successfully integrated into practice for the duration of the study, the essential elements were firstly, GPs received appropriate and sufficient training and support during the implementation process, secondly their professional skills and expertize were valued and thirdly, the RATs were perceived as workable and useful additions to GPs’ armoury.
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
GPL/2’s comments resonate with our earlier discussion regarding 2WW criteria. Seemingly the value of the RATs was in highlighting symptoms that might not have raised alarm previously which, in and of itself, will change GP practice. Indeed, the majority of GPs defined RATs in these terms:
So far we have considered the use of RATs from the perspective of users who perceived their incorporation as beneficial. There was, however, a minority of participants who were not persuaded that the RATs added to practice. In order to provide a balanced discussion, we include their reservations here. The following data illustrate the main issues raised:
GPs’ experiences of using diagnostic tools for cancer We conducted a qualitative study alongside a quantitative evaluation and one of the strengths of our analysis is our ability to hypothesize why, as the quantitative data indicate, GPs’ use of the RATs increased diagnostic activity and led to additional cancer diagnoses. As with all qualitative data, however, our discussions are based on a relatively small sample size and so are not generalizable. Nevertheless, we do suggest that our findings can augment the strategies needed for successful implementation of effective CDS tools into primary care practice. A final point to be made, however, is that this study was well resourced and thus able to provide ongoing support to practices throughout the study period. We would suggest this degree of support is reflected in the outcomes of the quantitative and qualitative data. This has implications for the wider dissemination of CDS tools and, at a time of increasing cutbacks in NHS England, raises questions as to whether resources will be made available in primary care for the kinds of support GPs who participated in this study needed and received.
Funding: the evaluation was funded by the National Cancer Action Team (NCAT). The funders have no involvement in the authors’ work. Ethical approval: the study was reviewed and approved by the Hull York Medical School Medical Education Ethics Committee. Conflict of interest: Professor William Hamilton is the clinical lead for the current revision of the NICE 2005 guidance. His contribution to this article is in a personal capacity and is not to be interpreted as representing the view of the Guideline Development Group or of NICE itself. Professor Hamilton is an Associate Editor of the Journal of Family Practice.
Acknowledgements The authors would like to thank Claire Morris in the National Cancer Action Team (NCAT), all participating GPs and the following people on behalf of the seven Local Cancer Networks: Nafeesah Mian, North Central London and West Essex Cancer Commissioning Cancer Network; Frances Haste, North East London Cancer Network; Jas Barn and Dr Louise Merriman, North Trent Cancer Network; Chris Losty, South West London Cancer Network; Matthew Greensmith, Humber and Yorkshire Cancer Network; Keith Smith, Sussex Cancer Network; Emma Walsh, Three Counties Cancer Network. The authors are grateful for the support and expertize of the administration team at SEDA, Hull York Medical School.
References 1. Murray CJL, Richards MA, Newton JN et al. UK health performance: findings of the Global Burden of Disease Study 2010. Lancet 2013; 381: 997–1020. 2. Verdecchia A, Francisci S, Brenner H et al.; EUROCARE-4 Working Group. Recent cancer survival in Europe: a 2000–02 period analysis of EUROCARE-4 data. Lancet Oncol 2007; 8: 784–96.
3. Coleman MP, Forman D, Bryant H, et al. Cancer survival in Australia, Canada, Denmark, Norway, Sweden, and the UK, 1995–2007 (the International Cancer Benchmarking Partnership): an analysis of populationbased cancer registry data. Lancet 2011; 377: 127–38. 4. Richards MA. The size of the prize for earlier diagnosis of cancer in England. Br J Cancer 2009; 101 (suppl 2): S125–9. 5. Hamilton W. The CAPER studies: five case-control studies aimed at identifying and quantifying the risk of cancer in symptomatic primary care patients. Br J Cancer 2009; 101 (suppl 2): S80–6. 6. Mansell G. Shapley M. Jordan J. Jordan K. Interventions to reduce primary care delay in cancer referral: a systematic review. Br J Gen Pract 2011; 63: e821–35. 7. Macleod U, Mitchell ED, Burgess C, Macdonald S, Ramirez AJ. Risk factors for delayed presentation and referral of symptomatic cancer: evidence for common cancers. Br J Cancer 2009; 101 (suppl 2): S92–101. 8. National Institute for Health and Clinical Excellence (NICE). Referral Guidelines for Suspected Cancer. London: NICE, 2005. 9. Hamilton W, Peters TJ, Round A, Sharp D. What are the clinical features of lung cancer before the diagnosis is made? A population based case–control study. Thorax 2005; 60: 1059–65. 10. Hamilton W, Round A, Sharp D, Peters TJ. Clinical features of colorectal cancer before diagnosis: a population-based case-control study. Br J Cancer 2005; 93: 399–405. 11. Adams S, Levenson S. Clinical prediction rules. BMJ 2012; 344: d8312. 12. May CA. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res 2006; 6: 86. 13. May CA, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology 2009; 43: 535–54. 14. Murray E, Treweek S, Pope C et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010; 8: 63. 15. May C, Finch T, Mair F et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res 2007; 7: 148. 16. Dahlhaus A, Vanneman N, Guethlin C, Behrend J, Siebenhofer A. German general practitioners’ views on their involvement and role in cancer care: a qualitative study. J Fam Pract 2014 Apr;31:209–14. 17. Bazely P, Jackson K. Qualitative Data Analysis with NVivo. London: Sage, 2013. 18. Ritchie J, Spencer L, O’Connor, W. Carrying out qualitative analysis. In: Ritchie J, Lewis J. (eds.) Qualitative Research Practice. London: Sage, 2003. pp. 219–62. 19. Spencer L, Ritchie J, O’Connor W. Analysis: practices, principles and processes. In: Ritchie J, Lewis J. (eds.) Qualitative Research Practice. London: Sage, 2003. pp. 199–218. 20. Hamilton W, Green T, Martins T, Elliott K, Rubin G, Macleod U. Evaluation of risk assessment tools for suspected cancer in general practice. Br J Gen Pract 2013; 63: e30–6. 21. Stolper E, Van Royen P, Van de Wiel M et al. Consensus on gut feelings in general practice. BMC Fam Pract 2009: 1471–2296. 22. Jiwa M, Hamilton W. Referral of suspected colorectal cancer: have guidelines made a difference? Br J Gen Pract 2004; 54: 608–10.
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
Declaration
105
Qualitative Research
Exploring GPs’ experiences of using diagnostic tools for cancer: a qualitative study in primary care Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
Trish Greena,*, Tanimola Martinsb,William Hamiltonb, Greg Rubinc, Kathy Elliottd and Una Macleoda Centre for Health and Population Sciences, Hull York Medical School, University of Hull, bUniversity of Exeter Medical School, cSchool of Medicine, Pharmacy and Health, University of Durham, and dkaeconsulting, London, UK. a
*Correspondence to Trish Green, Centre for Health and Population Sciences, Hull York Medical School, University of Hull, Hertford Building, Cottingham Road, Hull, HU6 7RX, UK. E-mail: [email protected]
Abstract Background. The UK has an estimated 5–10 000 extra cancer deaths each year when compared to other European countries and diagnostic delays are thought to make a significant contribution to this. One of the initiatives in England intended to support primary care professionals has been the development of cancer risk assessment tools (RATs). These tools assist in identifying and quantifying the risk of cancer in symptomatic primary care patients. Objective. To explore GPs’ experiences of incorporating the RATs for lung and bowel cancers into their clinical practice and in so doing, identify constraints and facilitators to the wider dissemination of the tools in primary care. Methods. We conducted semi-structured interviews over the telephone with 11 project managers who implemented the study and 23 GPs who used the tool. The interviews were digitally recorded, professionally transcribed verbatim and analysed through the construction of a ‘thematic framework’. Results. The training and support package was fundamental to the successful integration of the RATs into GPs’ daily routines. Ongoing support from cancer networks alongside acknowledgement of the clinical expertize of the GPs by those implementing the study enhanced GPs’ uptake of the tool in practice. Conclusion. Findings suggest that the embedding of clinical decision support tools into clinical practice is more likely to be achieved when they are perceived to support but not supersede the clinical judgement of their users. This element of our findings is a focal point of this article. Key words: Clinical decision support tools, early detection of cancer, general practitioners, primary health care, qualitative research.
Introduction Diagnosing cancer when at an early stage is important as stage of disease at diagnosis is related to survival for many cancers. The UK, however, has relatively poor cancer outcomes when compared with other European countries (1–3) with 5–10 000 extra cancer deaths a year (4,5). Diagnostic delay is considered a major contributor to
© The Author 2014. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: [email protected]
this and primary care has been identified as one arena wherein delay might occur. In the UK, patients with symptoms suggestive of cancer generally present to their GP in the first instance; it thus follows that part of the solution to delayed diagnosis could lie within the UK’s primary care system. To date however, no interventions have been shown to reduce time to diagnosis of cancer in primary care (6). A key task for GPs is to recognize potential cancer symptoms and 101
Family Practice, 2015, Vol. 32, No. 1
102
Methods RATs materials Desk-based RATs consist of tables containing the risk score for each symptom in isolation, for repeat presentation of the same symptom, and in combination with one other symptom (5). The tables were reproduced on mousemats and A5 desk easels, which functioned as aids for GPs when considering the likelihood of an individual patient aged 40 or over having either lung or colorectal cancer, given the symptom or combination of symptoms they presented with. Seven Cancer Networks, three of them London Networks working together, were involved in the study, which provided ~175 GP practices; the RATs materials were distributed to all 175.
Interviews A total of 47 invitations to interview were sent out across the seven networks via email and telephone and of these 34 individuals consented: 11 Project Managers; 8 GP Cancer Leads (GP cancer leads were part of the clinical leadership teams in Cancer Networks, with responsibility for supporting activities to assist with cancer diagnosis in practices in their locality using peer support and educational techniques) and 15 GPs. GP Cancer Leads were involved in the study in a supportive role and also used the RATs in their practice, making the total number of RATs users 23. All interviews were conducted by the first author between July and October, 2011. Prior to interview, all participants provided signed consent to its being recorded and our use of the data for analysis and reporting. Consent was also checked verbally at the time of each telephone interview. Topic guides for the interviews were designed to enable the capture of focussed data through a mixture of specific and open-ended questions. Project managers’ interviews focused on how the RATs were introduced to practices; GPs’ interviews explored experiences of the implementation process and their use of the RATs in practice. Interviews lasted ~30 minutes and ‘topic-relevant content (16)’ was ensured through the use of a semi-structured topic guide that enabled the elicitation of a broad range of responses. All interviews were professionally transcribed verbatim. Participants were made aware that data would not be reported in a way that would identify them individually. Practices were given some financial support via the Cancer Networks to conduct this project. We have attributed an acronym and number system to participants as follows: PM/1–11; GPL/1–8; GP/1–15. The study was reviewed and approved by the Hull York Medical School Ethics Committee.
Data analysis Because of the applied nature of our inquiry, the interview schedule was constructed pragmatically in order to determine firstly how the RATs study was implemented, secondly how GPs perceived this process and thirdly, GPs’ experiences of using the RATs. All transcripts were imported into the software package N-Vivo, which assisted the analytic process (17). To increase the clarity of the coding and the breadth of the analysis, transcriptions were double-read, co-coded and cross-checked (UM, TG). Emergent themes and sub-categories were identified, from which a coding framework (18) was created to represent different aspects of GPs’ experiences of the implementation process and GPs’ use of the tools in practice. Following Spencer et al.’s ‘code and retrieve’ model (19), each coded group was condensed and summarized and the emerging concepts fed into initial categories. The analytical process we describe here initially responded to the questions asked at interview. As our analysis progressed, a more nuanced understanding of the findings from our data emerged regarding how integration of the tools was achieved. We shall argue here that these findings are crucial for the future development and usability of RATs, and indeed for that of CDS tools in primary care more generally.
Results The data discussed in this article emerge from a qualitative study that was embedded within an evaluation that incorporated quantitative and qualitative methodologies. The quantitative evaluation demonstrated increased diagnostic activity and additional cancer diagnoses over the time of the study, which suggests that the RATs were effective in helping GPs with referral decisions for some patients (20). However, the quantitative data cannot tell us why the tools were
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
initiate appropriate investigation. To ensure that primary care does not contribute to delays, work needs to be directed at each of the following: improving recognition of symptoms, ensuring appropriate referral/investigation pathways are available and accessible, and ensuring responsive systems exist in specialist care. This article is concerned with the first aspect of this process—recognition of symptoms by GPs. Most cancer symptoms are also symptoms of benign conditions—and these benign conditions are much more common (7). When patients present with symptoms to primary care, GPs must judge the individual patient’s likelihood of cancer and select those whose symptom profile is reasonably suggestive of cancer to warrant further investigation or specialist referral. Support for cancer symptom recognition is available for GPs in the format of national guidance published by the National Institute for Health and Care Excellence [NICE (8)]. An urgent GP referral pathway for suspected cancer, the 2-week referral scheme, is well established. This describes symptom patterns for cancer that warrant urgent investigation (sometimes called ‘red flag’ symptoms), but many patients with cancer do not have symptoms that meet the criteria for its use. As the NICE guidance was based on contemporary available evidence largely derived from secondary care populations, interest in recent years has moved to obtaining evidence on presenting symptoms from primary care populations. Hamilton identified symptoms predictive of cancer in primary care cohort studies, which led to the development of risk estimates for patients aged over 40 presenting to primary care with symptoms of possible cancer; for single symptoms, pairs of symptoms and repeat attendances with the same symptom. These were presented as colour-coded risk assessment tools (RATs), for colorectal and lung cancers (5,9,10). They are now available for thirteen cancer sites. Although clinical decision support (CDS) is not a new concept (11), the development of integrated tools for the diagnosis of cancer is relatively novel. One relevant consideration is the way in which a newly introduced CDS tool becomes integrated into routine practice. Much of the theoretical underpinning for this is derived from the work of May and colleagues (12–15) who have developed normalization process theory to seek to understand and explain what it is that underscores the successful, or otherwise, incorporation of complex interventions into clinical practice. Undoubtedly, the ultimate purpose of any diagnostic tool is for it to become a part of clinical routines, that is, to become normalized, and instigate behaviour changes that prompt actions at a practice level that have a positive effect on patients’ health outcomes. This article reports on if and how the desk-based RATs for lung and colorectal cancer were integrated into routine practice within the context of this study.
GPs’ experiences of using diagnostic tools for cancer effective, or whether all participating GPs found them useful. We are also unable to gauge how GPs used the RATs in order for them to prove successful, or otherwise. We would argue that the answers to these kinds of queries hold the key to whether the RATs, and indeed CDS tools more generally, would succeed if disseminated beyond the parameters of a research project. The overall aim of the qualitative study was thus to explore GPs’ experiences of using the RATs to ascertain their acceptability and usability within the context of primary care practice. As will become clear in what follows, to meet our aim it was also necessary to consider the ways in which the study was implemented. We discuss our findings here under three main headings: firstly, the process of implementation; secondly, identifying signs and symptoms; and thirdly, changing practice.
The process of implementation
It was very professionally done at the beginning; we were well supported. It was not a particularly straightforward project. The team must’ve put in the groundwork, basically, and that’s really good, because we do get stuff coming through that’s just not being thought through and is a real waste. It’s a real disincentive then to get involved. (GP/10) I had quite a lot of requests to go down, um, to explain to them how to use the tool. It actually gave them an opportunity if they had any questions to ask them there and then and they felt more confident in using the tool, because when they got the presentations, they weren’t clear on quite a lot of the things. (PM/7)
The supportive activities that accompanied the implementation process emerged as key to GPs’ acceptance and perhaps more crucially, accurate reading of the tools. This approach resonates with Murray et al.’s (14) suggestion that clinicians need to not only familiarize themselves with decision making tools, but also grasp the principles that underpin them. What our findings so far suggest is that the RATs training and continued interaction between the study team and RATs users created a supportive environment that was seemingly integral to the successful streamlining of the tools into practice for the duration of the project. Clearly, the proven efficacy of a diagnostic tool when in its development phase does not guarantee acceptability on the part of its intended users. Rather, what becomes apparent from our discussion is that the implementation process could prove to be what ‘makes or breaks’ it in terms of successful uptake by practitioners.
Identifying signs and symptoms Guidance from NICE is in place to assist GPs in the referral of red flag cancer symptoms and the 2-week wait rule (2WW) ensures the
timeliness of secondary care consultations. As we described earlier, however, the nuanced nature of cancer symptoms presents one of the main stumbling blocks for the referral of some patients to secondary care: The barn door ones are a piece of cake, it’s just the grey area in the middle that we struggle with, patients coming along with symptoms that we aren’t certain about. (GPL/1)
The notion of ‘barn door’ symptoms was often juxtaposed with patients whose symptoms did not meet 2WW criteria; this latter group of patients caused study participants a great deal of anxiety as their pathway to investigation poses more difficulties: It [RATs] helped us with patients, not the red flag symptoms, but the kind of yellow flag symptoms, the under-symptoms that you think, you know, should I be worried about this patient or not? Whereas the NICE guidance really tells you much more about the red flag symptoms, which we should all have embedded in us already. (GP/5)
Our data revealed that some GPs used the RATs to support their referral decisions when patients’ symptoms did not meet 2WW referral criteria: You could actually say, although it doesn’t fulfil the two-week wait guidelines, the RAT score is high for this patient therefore we would like to bump it up to a two-week wait. (GP/4)
Findings suggest that using the RATs encouraged GPs to think differently about their patients’ symptoms; effectively to consider cancer first and then, as consultation proceeded, to either rule out cancer as a potential diagnosis or take additional diagnostic action, perhaps helping to reduce some of the uncertainty characteristic of decision making in general practice (21). A major benefit of the RATs was then that they alerted GPs to the potential significance of more opaque signs and symptoms of cancer. Significantly, GPLs made similar comments: It made me understand that certain symptoms when put with others actually do have more relevance than perhaps I thought they did. I think that’s a learning point in itself. (GPL/4) What the tool does really is make you ask the right questions if you think of cancer ... it made you better at making the decision, so I don’t think our two week referrals shot up dramatically, but everyone thought a little bit more about cancer when they were taking [patients’] histories. (GPL/5)
Evidently, the RATs assisted in the recognition of symptoms that previously might not have raised alarm, which is of course an intended function of the tools. GPs also perceived that sharing the tool during consultation helped them explain to patients why they would be referred, or not, to secondary care: I would use it alongside a patient, so if someone comes in and says, I have got a cough, I would be able to glance down to my mousemat and say, ‘and when you’ve been coughing, you know, have you been bringing up any blood?’, and just making myself think about looking at whether they’ve had any spirometry done and, you know, just using it as a prompt in my mind as to making sure I’m covering all bases. Having those charts was just very, very useful from that point of view. (GP/9)
Our findings indicate that for the majority of GPs we interviewed, RATs were useful in consultation, heightened GPs’ awareness of potential cancer symptoms, reminded and alerted them to potential cancer risks and affected their referral thresholds. As such, the majority of RATs users perceived the tools to be valuable additions to practice.
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
The introduction of the study to participating practices involved teams from the cancer networks distributing the RATs materials (mousemats/desk easels) with accompanying literature (5) and providing training on how the tools were to be used, that is, how to read each of the three tables. In most instances, training was carried out remotely via webcasts that GPs accessed in situ. Once trained, GPs were tasked to cascade the information to colleagues unable to attend training sessions. Networks also emailed updates and newsletters to the practices. Murray et al. (14) endorse these types of communication device because they were found to reinforce the importance of participation and instilled a sense of collective action for study participants. GPs were also encouraged to contact their respective network team if they encountered any problems, requests that were often responded to in person. Several project managers outlined the value of visits to practices to ensure GPs understood how to use the tools effectively:
103
Family Practice, 2015, Vol. 32, No. 1
104
Changing practice Although by dint of their use, the RATs will inevitably change practice, they were not designed to supersede GPs’ clinical judgment (the instructions on the tools state ‘use your clinical judgment’); indeed as GP/3 asserted, ‘the clinical judge is the doctor’. Interview data highlight how the implementation process was underpinned by this philosophy, which seemed to hold another key to the successful embedding of the RATs during the study period: We have stressed throughout the campaign that you shouldn’t use it above and beyond your clinical suspicion, you want GPs to still be using their clinical acumen, you can’t just drive by protocols, things don’t always fit a box. (GPL/2)
Just having those on our desk has made us think a little bit wider about what symptoms may be suggestive of cancer, looking at combining symptoms, rather than thinking about single symptoms on their own. (GP/5) It’s a fantastic aide memoire and prompt to have, you know, physically in front of you while you’re in the middle of a consultation. (GP/9)
The majority of GPs spoke positively regarding the aide memoire quality of the RATs materials, indicative perhaps of the tools’ ease of integration into daily practice and consultation, as well as its being a stimulus for discussion among colleagues and patients. It might be as well that through their engagement with the RATs, GPs became what Mansell et al. would describe as ‘sensitised to cancer (6)’, which in itself is probably helpful: I’ve been a GP for about 14, 15 years now ... there aren’t many things that change your practice, so it’s good when you come across things that do. (GP/8)
It is well accepted that experience and intuition play a part in the diagnosis of cancer (21,22). As well as this, GPs at times refer patients who do not meet guidelines, and conversely sometimes do not refer patients who do meet guidelines (22). What our data demonstrate is that the majority of GPs perceived that the RATs supported their clinical decision-making. As such, they were perhaps more easily incorporated as part of the mix that makes up the diagnostic process: You rely on your gut instinct to refer, but that’s where I could find this being more of . . . an algorithm that would help you with patients that you feel need the two week rule, but haven’t got quite the diagnostic criteria that fits. (GP/4) Sometimes it worked as a very reassuring tool; when you have someone who’s absolutely convinced they’ve got lung cancer or they’ve got bowel cancer, I would then get [the RAT] out in consultation and discuss it with the patient and say, this is all the reasons why I don’t have a bad feeling about your symptoms. (GP/2)
Evidently, the RATs were accepted by GPs as an additional element of the decision-making process. Although there were instances when GPs found the RATs to be at odds with referral guidelines, which in its turn created some difficulties with secondary care colleagues, overall the majority of participants felt that the tools complemented NICE and other cancer referral guidance and so strengthened and validated their referral decisions.
I just found the tables didn’t help hurry the process on at all. (GP/1) We’ve been doing all these things before this RAT thing came anyway. (GP/14) What I’m not sure it does for me is sort of fits into the consultation in a natural way of making a decision about the management of that patient as yet. It’s one more thing to fit into a busy ten minute consultation. (GPL/1)
There were, then, some participants who were not convinced that the RATs were valuable additions to general practice. GPs across the study sample, those ‘for’ and those ‘against’ RATs, also talked about the amount of information they received on a daily basis in the form of guidelines, letters, emails and so forth. This coupled with a less than enthusiastic response from potential RATs users, could prove problematic for the implementation of tools that intend to bring about new ways of working, not least in terms of keeping up the momentum needed to ensure GPs engage with new diagnostic aids.
Discussion Using the RATs led the majority of the GPs and GPLs we interviewed to make changes to their given ways of practice which, as data demonstrate, was perceived as beneficial. In several instances, use of the tools confirmed a decision to refer that, although GPs stated would have been made anyway, was perceived to give more credibility to that decision. For some participants, using the tools urged referrals that might not have been made; similarly at times the RATs were used to confirm a GP’s decision not to refer. Significantly then, the RATs seemed to meet the need for support in lung and colorectal cancer symptom recognition in primary care. In addition it was clear that the tools encouraged debate among study participants and their peers. Moreover, while GPs welcomed them as instructive tools, the RATs did not supersede clinical judgement; rather the tools were perceived as additives to GPs’ skills sets. The RATs were developed to address primary care delays in cancer symptom recognition and referral and our findings respond to the call from Mansell et al. who have argued the need for additional research ‘to determine which type of intervention is most effective in reducing the time for referral from primary to secondary care (6)’. The acceptability of the RATs was enhanced by them being derived from a primary care source and because there was a strong primary care involvement in the delivery of the study. Indeed, one of our major findings is that GPs felt well supported throughout and, importantly, felt their clinical expertize was acknowledged and valued by the team implementing the study. It is unclear from our data whether or not the tool was ‘normalized’ into practice (12–15). Indeed the study was not set up to address that question as this would necessitate further investigation into the extent of the RATs’ use beyond the study’s timeframe. What we can say is that where the tool appeared to be successfully integrated into practice for the duration of the study, the essential elements were firstly, GPs received appropriate and sufficient training and support during the implementation process, secondly their professional skills and expertize were valued and thirdly, the RATs were perceived as workable and useful additions to GPs’ armoury.
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
GPL/2’s comments resonate with our earlier discussion regarding 2WW criteria. Seemingly the value of the RATs was in highlighting symptoms that might not have raised alarm previously which, in and of itself, will change GP practice. Indeed, the majority of GPs defined RATs in these terms:
So far we have considered the use of RATs from the perspective of users who perceived their incorporation as beneficial. There was, however, a minority of participants who were not persuaded that the RATs added to practice. In order to provide a balanced discussion, we include their reservations here. The following data illustrate the main issues raised:
GPs’ experiences of using diagnostic tools for cancer We conducted a qualitative study alongside a quantitative evaluation and one of the strengths of our analysis is our ability to hypothesize why, as the quantitative data indicate, GPs’ use of the RATs increased diagnostic activity and led to additional cancer diagnoses. As with all qualitative data, however, our discussions are based on a relatively small sample size and so are not generalizable. Nevertheless, we do suggest that our findings can augment the strategies needed for successful implementation of effective CDS tools into primary care practice. A final point to be made, however, is that this study was well resourced and thus able to provide ongoing support to practices throughout the study period. We would suggest this degree of support is reflected in the outcomes of the quantitative and qualitative data. This has implications for the wider dissemination of CDS tools and, at a time of increasing cutbacks in NHS England, raises questions as to whether resources will be made available in primary care for the kinds of support GPs who participated in this study needed and received.
Funding: the evaluation was funded by the National Cancer Action Team (NCAT). The funders have no involvement in the authors’ work. Ethical approval: the study was reviewed and approved by the Hull York Medical School Medical Education Ethics Committee. Conflict of interest: Professor William Hamilton is the clinical lead for the current revision of the NICE 2005 guidance. His contribution to this article is in a personal capacity and is not to be interpreted as representing the view of the Guideline Development Group or of NICE itself. Professor Hamilton is an Associate Editor of the Journal of Family Practice.
Acknowledgements The authors would like to thank Claire Morris in the National Cancer Action Team (NCAT), all participating GPs and the following people on behalf of the seven Local Cancer Networks: Nafeesah Mian, North Central London and West Essex Cancer Commissioning Cancer Network; Frances Haste, North East London Cancer Network; Jas Barn and Dr Louise Merriman, North Trent Cancer Network; Chris Losty, South West London Cancer Network; Matthew Greensmith, Humber and Yorkshire Cancer Network; Keith Smith, Sussex Cancer Network; Emma Walsh, Three Counties Cancer Network. The authors are grateful for the support and expertize of the administration team at SEDA, Hull York Medical School.
References 1. Murray CJL, Richards MA, Newton JN et al. UK health performance: findings of the Global Burden of Disease Study 2010. Lancet 2013; 381: 997–1020. 2. Verdecchia A, Francisci S, Brenner H et al.; EUROCARE-4 Working Group. Recent cancer survival in Europe: a 2000–02 period analysis of EUROCARE-4 data. Lancet Oncol 2007; 8: 784–96.
3. Coleman MP, Forman D, Bryant H, et al. Cancer survival in Australia, Canada, Denmark, Norway, Sweden, and the UK, 1995–2007 (the International Cancer Benchmarking Partnership): an analysis of populationbased cancer registry data. Lancet 2011; 377: 127–38. 4. Richards MA. The size of the prize for earlier diagnosis of cancer in England. Br J Cancer 2009; 101 (suppl 2): S125–9. 5. Hamilton W. The CAPER studies: five case-control studies aimed at identifying and quantifying the risk of cancer in symptomatic primary care patients. Br J Cancer 2009; 101 (suppl 2): S80–6. 6. Mansell G. Shapley M. Jordan J. Jordan K. Interventions to reduce primary care delay in cancer referral: a systematic review. Br J Gen Pract 2011; 63: e821–35. 7. Macleod U, Mitchell ED, Burgess C, Macdonald S, Ramirez AJ. Risk factors for delayed presentation and referral of symptomatic cancer: evidence for common cancers. Br J Cancer 2009; 101 (suppl 2): S92–101. 8. National Institute for Health and Clinical Excellence (NICE). Referral Guidelines for Suspected Cancer. London: NICE, 2005. 9. Hamilton W, Peters TJ, Round A, Sharp D. What are the clinical features of lung cancer before the diagnosis is made? A population based case–control study. Thorax 2005; 60: 1059–65. 10. Hamilton W, Round A, Sharp D, Peters TJ. Clinical features of colorectal cancer before diagnosis: a population-based case-control study. Br J Cancer 2005; 93: 399–405. 11. Adams S, Levenson S. Clinical prediction rules. BMJ 2012; 344: d8312. 12. May CA. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res 2006; 6: 86. 13. May CA, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology 2009; 43: 535–54. 14. Murray E, Treweek S, Pope C et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010; 8: 63. 15. May C, Finch T, Mair F et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res 2007; 7: 148. 16. Dahlhaus A, Vanneman N, Guethlin C, Behrend J, Siebenhofer A. German general practitioners’ views on their involvement and role in cancer care: a qualitative study. J Fam Pract 2014 Apr;31:209–14. 17. Bazely P, Jackson K. Qualitative Data Analysis with NVivo. London: Sage, 2013. 18. Ritchie J, Spencer L, O’Connor, W. Carrying out qualitative analysis. In: Ritchie J, Lewis J. (eds.) Qualitative Research Practice. London: Sage, 2003. pp. 219–62. 19. Spencer L, Ritchie J, O’Connor W. Analysis: practices, principles and processes. In: Ritchie J, Lewis J. (eds.) Qualitative Research Practice. London: Sage, 2003. pp. 199–218. 20. Hamilton W, Green T, Martins T, Elliott K, Rubin G, Macleod U. Evaluation of risk assessment tools for suspected cancer in general practice. Br J Gen Pract 2013; 63: e30–6. 21. Stolper E, Van Royen P, Van de Wiel M et al. Consensus on gut feelings in general practice. BMC Fam Pract 2009: 1471–2296. 22. Jiwa M, Hamilton W. Referral of suspected colorectal cancer: have guidelines made a difference? Br J Gen Pract 2004; 54: 608–10.
Downloaded from http://fampra.oxfordjournals.org/ at Karolinska Institutet on November 16, 2015
Declaration
105
