International Journal of Nursing Studies 52 (2015) 716–726
Contents lists available at ScienceDirect
International Journal of Nursing Studies journal homepage: www.elsevier.com/ijns
Nurse practitioners versus doctors diagnostic reasoning in a complex case presentation to an acute tertiary hospital: A comparative study Alison M. Pirret a,b,*, Stephen J. Neville a, Steven J. La Grow c a b c
School of Nursing, College of Health, Massey University, New Zealand Critical Care Complex, Middlemore Hospital, Auckland, New Zealand College of Health, Massey University, New Zealand
A R T I C L E I N F O
A B S T R A C T
Article history: Received 20 December 2013 Received in revised form 20 August 2014 Accepted 25 August 2014
Background: Nurse practitioners perform a diagnostic role previously delivered by doctors. Multiple studies demonstrate nurse practitioners are as effective as doctors when managing chronic conditions and minor illnesses and injuries. No studies have focused on how nurse practitioners compare to doctors in their management of complex cases presenting for the first time. Objective: This study assessed how nurse practitioners’ diagnostic reasoning abilities when managing a complex case compared to those of doctors’? Design: A comparative research design. Participants: Purposeful sampling recruited 30 nurse practitioners and 16 doctors working in multiple specialties in New Zealand. All doctors were completing postgraduate specialist training programmes. Specialties included older adults, emergency care, primary health care/general practice, cardiology, respiratory and palliative care. Methods: A complex case scenario assessed by an expert panel and think aloud protocol was used to assess diagnostic reasoning abilities. The ability of 30 nurse practitioners to determine diagnoses, identify the problem, and propose actions was compared to that of 16 doctors. Correct responses were determined by an expert panel. Data gained from the case scenario using think aloud protocol were quantified for analysis. Results: 61.9% of doctors identified the correct diagnoses, 56.3% the problem and 34.4% the actions as determined by the expert panel. This compares to 54.7% of nurse practitioners identifying the correct diagnoses, 53.3% the problem and 35.8% the actions. Analysis revealed no difference between these groups (diagnoses 95% CI: 1.76 to 0.32, p = 0.17, problem x2 = 0.00, p = 1.0, or actions 95% CI: 1.23 to 1.58, p = 0.80). Conclusion: Nurse practitioners’ diagnostic reasoning abilities compared favourably to those of doctors in terms of diagnoses made, problems identified and action plans proposed from a complex case scenario. In times of global economic restraints this adds further support to alternative models of care. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Complex case Diagnostic accuracy Diagnostic reasoning Nurse practitioners Registrars
* Corresponding author at: School of Nursing, College of Health, Massey University, Private Bag 102904, North Shore, Auckland 0745, New Zealand. Tel.: +64 94140800; fax: +64 94418165. E-mail address: [email protected] (A.M. Pirret). http://dx.doi.org/10.1016/j.ijnurstu.2014.08.009 0020-7489/ß 2014 Elsevier Ltd. All rights reserved.
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
What is already known about the topic? Multiple studies demonstrate nurse practitioners achieve similar patient outcomes to doctors. Most studies comparing nurse practitioner patient outcomes to doctors compare their management of chronic conditions and minor illnesses and injuries Few studies compare nurse practitioner diagnostic reasoning abilities to those of doctors. Most studies comparing nurse practitioner diagnostic reasoning abilities compare them to house officers. What the paper adds Nurse practitioners analysis of a single complex case compare favourably to that of registrars in terms of diagnoses made, problems identified and action plans proposed from a complex case scenario. The findings of the study suggest nurse practitioners’ diagnostic reasoning developed from education and experience enables them to diagnose and manage complex patients presenting for the first time. 1. Introduction Nurse practitioners in New Zealand (NZ) were introduced to increase patients’ access to healthcare, improve patient outcomes (Ministry of Health, 2002) and provide a solution to doctor shortages (Forde, 2008; Ministry of Health, 2009). Combining advanced nursing practice with skills from medicine, nurse practitioners diagnose, assess and manage patients and can order diagnostic tests and prescribe; historically these roles were considered exclusive to medicine (Forde, 2008; Ministry of Health, 2002). In addition nurse practitioners promote health, encourage self-care and look beyond the diagnosis to consider nonmedical interventions (Ministry of Health, 2002). 1.1. International differences in the use of the nurse practitioner title There are international differences as to how the title nurse practitioner is used. Countries, such has NZ, Australia and the United States (US) have a rigorous assessment process and require a Master’s degree (Carryer et al., 2007; Kleinpell et al., 2008). New Zealand and Australia and some US states allow nurse practitioners to practice independently without supervision from a physician (Carryer et al., 2007; Kleinpell et al., 2008; Lowes, 2014). The US recommend nurse practitioner education programmes move to a Doctorate degree by 2015 (American Association of Colleges of Nursing, 2012), however laws and regulations pertaining to nurse practitioner scope of practice (including prescribing authority) remain inconsistent from state to state (Poghosyan et al., 2012). In Canada, legislation, regulations and standardisation of the nurse practitioner title are in place in most provinces and territories (Sangster-Gormley et al., 2011) unlike the United Kingdom (UK) where no legislation protects the title nurse practitioner. Many nurses working in advanced nursing practice roles in the UK have a Master’s degree but
717
nurses are still able to do a one-week course and use the title advanced nurse practitioner (Coombes, 2008). Over recent years, the Royal College of Nursing has lobbied for a registered trade title for nurse practitioner similar to NZ (Coombes, 2008) but this has not yet occurred (Santry, 2010). 1.2. Nurse practitioner and physician patient outcomes Multiple studies demonstrate nurse practitioners achieve similar patient outcomes to medical doctors (Dierick-van Daele et al., 2009; Horrocks et al., 2002; Laurant et al., 2008) however many of these studies focus on patients referred by the doctor to nurse practitioners for management of chronic conditions, patients presenting for the first time with minor illnesses or injuries, and nurse practitioners working alongside general practitioners (Horrocks et al., 2002; Laurant et al., 2008; Newhouse et al., 2011). Few studies compare nurse practitioner diagnostic reasoning abilities to those of doctors. Sakr et al. (1999) showed UK emergency care nurse practitioners and house officers had similar rates of diagnostic error (9.2% compared to 10.7%) when assessing and treating patients presenting with minor illnesses and injuries (Sakr et al., 1999). This finding was further supported in a study of Dutch emergency care nurse practitioners and senior house officers; this study also found no difference in the rate of missed injuries and inappropriate management of patients presenting with minor illnesses and injuries (van der Linden et al., 2010). In a study of UK primary care nurse practitioners and general practitioners, Offredy (2002) attributed the differences between the diagnostic accuracy of the two groups to general practitioners having more knowledge and experience than nurse practitioners. This was related to nurse practitioners’ lack of familiarity with the case presentations due to the restrictions general practitioners placed on the type of consultations they performed. Although all the nurse practitioners in the study had completed the Royal College of Nursing nurse practitioner degree programme, their limited scope of practice means these results may not reflect the international context. Whilst research demonstrates nurse practitioners compare favourably to doctors in their management of minor illnesses/injuries and chronic conditions, to the best of our knowledge no research compares nurse practitioners’ and doctors’ diagnostic reasoning abilities pertaining to a complex case. 1.3. Diagnostic reasoning theory Familiar case presentations automatically use intuitive processing (Croskerry, 2009; Djulbegovic et al., 2012) which is developed through experience. It allows rapid diagnosis however it is influenced by environmental information, pattern recognition and the use of mental short cuts known as heuristics (Croskerry, 2009; Stanovich, 2010), which increase the risk of diagnostic error. If the patient presentation is not initially recognised, time permits or the clinician is uncertain, analytic processing
718
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
is triggered (Croskerry, 2009; Pelaccia et al., 2011; Stolper et al., 2011). Analytic processing involves slower, step-by-step, conscious, logical and defensible processes (Coderre et al., 2010; Croskerry, 2009; Ely et al., 2011; Heiberg Engel, 2008; Szaflarski, 1997). This approach requires explicit knowledge of pathophysiology, diseases/conditions and clinical manifestations (Croskerry, 2009; Szaflarski, 1997) developed through learning that continue to develop as clinicians mature (Croskerry, 2009). 1.4. Using complex cases to trigger analytic processing Analytic processing is triggered by complex cases. Ilgen et al. (2011) assessed the diagnostic accuracy of medical students and residents using both simple and complex cases. They defined a simple case as a typical presentation of a single diagnosis and a complex case as one that introduces features of multiple diagnoses (Ilgen et al., 2011). These definitions resonate those used by Mamede et al. (2008) measuring the effect of reflective practice on the diagnostic accuracy of 42 internal medical residents in Brazil. Their simple cases included a single diagnosis that was frequently encountered by residents. Their complex cases comprised of either a combination of different acute clinical conditions, patients with co-morbidities, an atypical disease presentation or a case rarely seen by residents (Mamede et al., 2008). Diagnostic accuracy differs in simple and complex cases. Mamede et al. (2008) found reflective practice achieved a mean accuracy of 73% in simple cases and 42% in complex cases. Ilgen et al. (2011) in their study of 51 medical students (novices), 26 first or second year residents (intermediates) and 38 third year residents (experienced) found participants achieved high scores in simple cases but lower scores in complex cases. In simple cases novices scored a mean of 69.9%, intermediates 80.8% and experienced 89.5%. In the complex cases novices scored a mean of 31%, intermediates 47% and experienced 55% (Ilgen et al., 2011). In a study to test the relationship between speed and accuracy of Canadian emergency care residents, Sherbino et al. (2012) deliberately made the cases difficult so analytic processing could be triggered. Following a pilot study, they removed cases that were too easy (achieving a diagnostic accuracy approaching 100%) and those that were too difficult (achieved a diagnostic accuracy approaching 0%). Findings from their study showed residents achieved a mean accuracy score of 49% (Sherbino et al., 2012). As intuitive processing is faster than analytic processing, speed in generating a diagnosis has historically been associated with a higher rate of diagnostic error. This is thought to be related to premature closure or using intuitive processing when the case requires analytic processing (Elstein, 2009; Lucchiari and Pravettoni, 2012; Norman and Eva, 2010). Premature closure is the acceptance of a diagnosis before sufficient verification has occurred and failure to consider other plausible alternatives once it has been reached (Levy et al., 2007). Sherbino et al. (2012) measured residents’ diagnostic accuracy using both intuitive and analytic processing; fast
times indicated intuitive processing while slower times signified analytic processing. They found the response times varied but the most difficult case was associated with the longest time. The ability of nurse practitioners to diagnoses complex cases has been challenged. Gorman (2009) views the doctor in the future as a health professional who has largely a cognitive function, translating patients’ signs and symptoms into a diagnosis; this role, he argues, cannot be substituted by nurse practitioners. Gorman sees medicine as having a strong diagnostic role at the front door of healthcare facilities, referring to nurse practitioners and other health professional-led intervention clinics when required. This view suggests doctors are better suited to diagnosis and treatment whereas nurse practitioners are better suited to ongoing interventions once the diagnosis is made. 2. Research aim This study compared nurse practitioners’ and doctors’ diagnostic reasoning abilities related to a complex case. The study answered the research question, how do nurse practitioners’ diagnostic reasoning abilities of a complex case compare to those of doctors’? This question was based on the underlying assumption, that as nurse practitioners focus on health promotion and disease prevention, when compared to doctors, their diagnostic abilities when analysing a complex case may be inferior. The primary outcome for this study was the number of correct diagnoses, problem and actions identified. In this study, the term diagnostic reasoning indicates the cognitive process involving data collection, identification of diagnoses and problems and the formulation of an action plan (Baid, 2006; Carneiro, 2003; Stausberg and Person, 1999). The term diagnosis denotes labelling of the disease or illness and the term problem means abnormal findings or problems needing intervention (Brykczynski, 1989, 1999; Elstein et al., 1993; Frauman and Skelly, 1999; Hoffman et al., 2009; Muller-Staub et al., 2008). The term action plan refers to applying interventions, prescribing and referring in response to identified diagnoses and problems (Baid, 2006; Carneiro, 2003; Weiss, 2011). 3. Method Using a comparative research design, this study used a complex case scenario and think aloud protocol to compare nurse practitioners’ and doctors’ diagnostic reasoning abilities. Think aloud protocol is a qualitative method (Arocha and Patel, 2008; Bucknall and Aitken, 2010; Hoffman et al., 2009; Lundgren-Laine and Salantera, 2010) used to analyse how clinicians’ use their knowledge to generate diagnoses and the complex relationships between knowledge transition and generation of diagnoses (Joseph and Patel, 1990). In this study, the ability of 30 nurse practitioners to determine diagnoses, identify the problem, and propose actions related to a complex case was compared to that of 16 doctors. Correct responding was determined by an expert panel consisting of a Professor of General Practice,
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Box 1. Summary of case scenario. A 67-year-old man presents with worsening productive cough, purulent sputum and left sided chest pain. He has a 2-week history of flu-like illness, backache, posterior chest pain, productive cough and increased shortness of breath. He visited his GP 1 week ago and was prescribed amoxicillin 500 mg three times daily but has had no improvement. He has multi-morbidities including insulin dependent Type II diabetes, hypertension and hyperlipidaemia. He has stopped taking his insulin as he does not think he needs it. He is an ex-smoker with a 40-pack year history. He has a reduced appetite with a 5 kg weight loss over past few weeks. He is experiencing generalised abdominal pain and has black hard stools. He is hypertensive with a slightly raised respiratory rate. Left sided chest pain is present on inspiration and coughing, with crackles left base on auscultation and dull left base on percussion. He feels slightly clammy to touch but peripherally warm. Chest X-ray indicates opacity in the left lower lobe. Chest X-ray report indicates no consolidation and suggests changes in left lower lobe relate to old scarring. It also reports hyperinflation of the lungs consistent with a degree of COPD.
an Associate Professor of Rheumatology and a Diabetes nurse practitioner. The decision to use an expert panel was based on previous research showing the diagnostic accuracy of doctors was inferior when compared to medical specialists (Allen et al., 1998) hence it was crucial nurse practitioners’ and doctors’ diagnostic reasoning abilities were compared to what experts considered best practice. The expert panel was also used to gain consensus on the case scenario’s complexity and ability to trigger participants’ analytic processing. The medical experts on the expert panel were academically and clinically involved in training doctors and experienced at using simple and complex cases to assess doctors’ diagnostic reasoning abilities. They
considered the case scenario complex as it contained multiple diagnoses within a single case presentation. A summary of the case scenario is outlined in Box 1. The case scenario was based on a real case presenting to an acute tertiary hospital. It was obtained by the researcher (who is also a nurse practitioner) after the patient was identified by a ward nurse manager as being a complex case with diagnostic uncertainty. Data from the presenting patient’s health history and physical examination were obtained from the clinical notes. Data pertaining to laboratory and radiology results were downloaded from the hospital data management system. All patient identifying data were removed. The data were transcribed and inserted into a word document for review by the expert panel. The hospital provided written approval for this case scenario to be used in this study. As patient identifying data were removed, the hospital research office and Massey University Ethics Committee did not require patient consent for the case scenario to be used. The Delphi technique gained consensus from the expert panel. For the correct diagnoses, problems and actions, two rounds of Delphi were used to achieve a 96% consensus. The panel did not agree on the need for spirometry to confirm the diagnoses of chronic obstructive pulmonary disease. The diagnoses, problem and action plan the expert panel identified are outlined in Box 2. One round of Delphi achieved a 100% consensus on the complexity of the case scenario and its ability to trigger analytic reasoning. 3.1. Participant selection As the study incorporated quantitative analysis, prospective power analysis calculated the desired sample size. This calculation was based on the study’s underlying assumption that when compared to doctors, nurse practitioners may have inferior diagnostic abilities when analysing a complex case. Previous research identified a 15.7% difference between junior and senior residents when analysing a complex case (Ilgen et al., 2011); hence for this study the percentage difference between the two groups
Box 2. Diagnoses, problems and actions the expert panel expected of doctors.
Expected diagnoses/problems
Expected action plan
? lower respiratory tract infection ? pleural effusion ? pulmonary embolism ? lung cancer ? chronic obstructive pulmonary disease Gastric bleeding ? Gastric ulcer
Computerised tomography or with pulmonary angiogram Sputum culture Change antibiotics to include a macrolide Lung function tests – spirometry
Hypertension Reasonably well controlled Type II diabetes mellitus Well controlled hyperlipidaemia Poor adherence to medications
719
Gastroscopy Proton pump inhibitor Stop both aspirin Stop diclofenac Test Haemophilus-Pylori Refer hospital for specialist team review/? hospital admission Recheck and monitor blood pressure and if required review antihypertensive medications Diabetic referral/Diabetic education
720
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Table 1 Prospective and retrospective power analysis for statistical tests. Statistical test
Effect size
Prospective Power
Sample size
Power
Sample size
Two tailed between group independent t-test One tailed between group independent t-test Two tailed within group independent t-test Two tailed Pearson’s product moment correlation coefficient Chi-square (with one degree of freedom)
0.80 0.80 0.80 0.80 0.80
0.86 0.92 0.99 0.99
60 60 30 19
0.75 0.85 0.99 0.98 0.97
46 46 30 16 46
was expected to be greater with a mean difference (MD) >1.0 when determining diagnoses and >1.3 when planning actions. Purposive sampling was used to recruit 30 nurse practitioners and 30 doctors working in adult practice areas in NZ with a patient population reflected in the case scenario using think aloud. Only doctors completing postgraduate specialist training programmes were recruited. Nurse practitioners working in areas, such as child and mental health, and specialist areas, such as ophthalmology and organ transplant, were excluded from the study. Multiple professional networks failed to recruit the planned number of doctors, particularly in the general practice specialty; as a result only 16 doctors were recruited. The calculations for the prospective and retrospective power analyses are outlined in Table 1. As most of the nurse practitioners meeting the selection criteria were recruited into the study, no randomisation was performed. 3.2. Ethical approval Ethics approval was gained from the Massey University Human Ethics Committee. All participants provided written consent. 3.3. Data collection Data were collected between February 2011 and March 2012 (inclusive). This study used both concurrent and retrospective think aloud. Concurrent think aloud reflects cognitive processes occurring at the current time and provides more accurate data when compared to retrospective think aloud which gathers verbalisations after the task (Bucknall and Aitken, 2010; Hoffman et al., 2009; Kuusela and Paul, 2000). Each participant completed the case scenario using think aloud protocol in a private office in their workplace with the researcher present. All participants completed the same case scenario and signed a participant confidentiality agreement. Prior to completing the case scenario, each participant received verbal and written directions and completed a short practice session on another case scenario to ensure unfamiliarity with the computer programme did not influence the data collected. No time limit was allocated to complete the scenario as concurrent thinking may slow down the diagnostic reasoning process (Kuusela and Paul, 2000). The case scenario data, including the patient’s health history, physical examination and diagnostic tests results,
Retrospective
were presented using the Scenario Based Learning Interactive (SBLi) programme. Diagnostic tests results included laboratory results, a lateral and posterior–anterior chest Xray and a chest X-ray report. The case scenario data were divided into 23 segments and presented one segment at a time; each segment provided multiple pieces of clinical information relevant to the case. Presenting one segment at a time provides more control of the stimuli and more information about participants’ cognitive processes (Joseph and Patel, 1990). Participants chose the order and rate in which each segment was presented and were able to access the information presented in prior segments. Both the computer programme and the researcher recorded the time taken to complete the case scenario. Participants were prompted to think aloud at regular intervals if required. After presentation of the entire case, participants provided a summary of their final diagnoses, problems and action plan. This used retrospective think aloud and provided the opportunity for participants to articulate diagnoses, problems and actions they may not have articulated during concurrent think aloud. At the end of the case scenario, participants were asked to comment on how the case presentation reflected the type of patients they would see regularly in their practice setting. A portable MP3 player audiotaped participants’ verbalisations. 3.4. Data analysis The audiotaped case scenario think aloud data were transcribed verbatim and analysed using coding and categorising described by Elstein et al. (1993). Elstein et al. developed and applied coding schemes to Joseph and Patel’s (1990) study assessing the diagnostic accuracy of endocrinologists and cardiologists when analysing an endocrine case presentation (Joseph and Patel, 1990). On applying the coding schemes, Elstein et al. found the same results as Joseph and Patel. The coding schemes described by Elstein et al. are outlined in Boxes 3 and 4. Although think aloud is a qualitative method providing qualitative data, the qualitative data obtained from the think aloud transcriptions were transformed into quantitative data; this process is referred to as quantitising (Bazeley, 2009; Cresswell and Plano Clarke, 2011; Sandelowski, 2000). As the research needed to contribute to future health workforce development, quantitising the data meant the analysis and results were more likely to be understood and valued by medical, nursing and governmental bodies responsible for workforce planning.
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Box 3. Rules for coding process constructs copied from Elstein et al. (1993, p. 28). Rules for Coding Process Constructs 1. Formulates working diagnostic hypothesis. Subject arrives at a diagnostic hypothesis involving a disease entity of medical condition. This construct is scored for every unique diagnostic hypothesis that is given. Do not score if a subject repeats a diagnostic hypothesis given in a previous section. 2. Confirms or strengthens diagnostic hypothesis using new information. Score when subject retains, confirms, or strengthens hypothesis using vignette information. Simply repeating a previously mentioned diagnostic hypothesis is not enough to score here. The subject must say why or clearly be using vignette material to retain or confirm the hypothesis. 3. Rejects or weakens diagnostic hypothesis using new information. Score if subject rejects or weakens a hypothesis using findings or rules out a diagnosis based on finding presents in a segment. A subject may reject a hypothesis without explicitly proposing it. When this happens, go back and score it as a diagnostic hypothesis and indicate that it was never explicitly hypothesised by writing ‘‘rejects’’ by it. A subject also may reject a proposed hypothesis in the same segment and may prose and reject virtually at the same time. 4. Multiple-cue inference. Score when the subject draws conclusions about diagnosis, prognosis, or treatment on the basis of multiple cues. These cues may be information given in the vignette or information generated by the subject. The cue must be fairly explicit. An exception to the explicitcues rule occurs when the subject says this is a ‘‘classic’’ case or example. When a subject gives multiple inferences embedded in one sentence, score each instance 5. Single cue inference. Score when the subject draws conclusion about diagnosis, prognosis, or treatment on the basis of a single cue. This cue may be information given in the vignette or information generated by the subject. Score also when the subject adds one more piece of information to a clinical picture and makes an inference. That is, the clinical picture may contain numerous cues, but the subject explicitly picks one piece of information and adds it, resulting in the subject’s final inference
Each diagnosis, problem and action was given a numerical code. All coding was completed by the first author; the coding process was reviewed by the second and third authors for peer checking. Coded data were entered into a Statistical Package for the Social Sciences (SPSS) version 19. Each diagnosis, problem and action identified by each participant was entered as a 1 under labels reflecting those diagnoses, problems and actions. Adding this data within SPSS provided the number of accurate diagnoses, problems and actions obtained from each participant. Following this analysis, the number of correct, diagnoses, problems and actions were added to provide a single score reflecting each participant’s
721
diagnostic reasoning ability. Case scenario think aloud data were assessed for how the case scenario reflected cases regularly seen in participants’ practice areas. The two-tailed independent t-test, 95% confidence interval (CI), chi square test (x2) and Fisher’s Exact test (FET) were used to compare the nurse practitioner data to the doctor data. Pearson’s product moment correlation coefficient (r) Spearman’s rank order (rho) coefficient (rs) and the Kruskal–Wallis test (x2) were used for within group analysis. The alpha level for statistical significance was set at 0.05. 4. Results Overall 46 participants took part in the study, 30 nurse practitioners and 16 doctors. The nurse practitioners and doctors worked in gerontology, emergency care, primary health care/general practice, cardiology, respiratory, and palliative care. In both the nurse practitioner and doctor group the largest number were from primary health care, the smallest number from palliative care. 4.1. Participant demographics The characteristics of each of the groups are outlined in Tables 2 and 3. The small number of nurse practitioners in NZ and the need for participant anonymity prevents the numbers from each specialty area being shared. Four doctors had previously worked as a doctor in another specialty training programme prior to commencing their current specialty training. One doctor was previously a medical specialist in another specialty. 4.2. Diagnostic reasoning abilities Nurse practitioners identified a mean of 5.47 (95% CI: 4.86–6.08) correct diagnoses (n = 10) compared to a mean of 6.19 (95% CI: 5.27–7.10) generated by doctors. Analysis revealed no difference between the two groups (95% CI: 1.76 to 0.32, p = 0.17). For the five correct definitive diagnoses, no differences were found between the nurse practitioner and doctor groups however this was not the case for the five differential diagnoses (Table 4). When compared to doctors, fewer nurse practitioners correctly identified a lower respiratory tract infection, pleural effusion and pulmonary embolus. Fewer doctors than nurse practitioners correctly identified chronic obstructive pulmonary disease. Relationships between nurse practitioners’ ability to identify the correct diagnoses and their demographic data found a positive correlation with the number of years prescribing as a nurse practitioner in NZ (rs = 0.37, p = 0.04). This implies nurse practitioners’ abilities to correctly diagnose are improved as they gain more experience as a NZ prescribing nurse practitioner. The expert panel agreed the problem, ‘poor adherence to medications’ should be identified. Sixteen nurse practitioners (53%) identified this problem compared to nine (56%) doctors. Analysis revealed no difference between the two groups (x2 = 0.00, p = 1.0).
722
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Box 4. Rules for coding knowledge utilisation copied from Elstein et al. (1993, p. 28). Rules for coding knowledge utilisation 1. Do not draw inference across sentences If a diagnosis is drawn on the basis of symptoms mentioned in an immediately preceding separate sentence, it should be coded ‘‘diagnosis’’ not ‘‘finding to diagnosis.’’ But if such sentences begin with connectives such as ‘‘therefore’’ they should get relational codings. 2. Draw inferences across segments within the same sentence. That is, if, for example, a diagnosis is drawn on the basis of a symptom mentioned in a preceding segment of the same sentence, it should be coded ‘‘symptom to diagnosis.’’ This holds even when there is no tell-tale word such as ‘‘so’’ or ‘‘therefore,’’ so long as the diagnosis seems clearly to have been elicited by the symptoms mentioned. 3. Use ‘‘Context’’ to code statements about nonmedical facts about the patient. E.g. the patient’s age, job history, living conditions, normal bodily statistics, daily activities and nonmedical events prior to admission. Normal bodily statistics of an every day sort are context; abnormal are findings. E.g. weight is context, overweight is a finding. 4. Use ‘‘Finding’’ to code statements about medical history and direct physical examination. Include testimony of the patient or the patient’s friends, family, etc. 5. Use ‘‘Test’’ to code statement based on laboratory tests or special diagnostic procedures. E.g. X-rays, blood cultures, sputum culture, CT scan. ‘‘Test’’ refers only to direct test results; code inference from test results as ‘‘diagnosis’’ (see rule 6). 6. Use ‘‘Diagnosis’’ to code any statement about the possible underlying pathophysiology or cause of the problem. Not just for statements about diseases. 7. Use ‘‘Treatment’’ to code statements about therapy and management. When treatment information is used to make diagnostic inference, code as ‘‘Treatment to Diagnosis’’ 8. Use ‘‘Other’’ to code ‘‘Same’’ and ‘‘I don’t know,’’ as well as the following: a. General programmatic or procedure statements. b. B. Nonmedical or metacognitive statements 9. Follow relational order given in the sentence syntax to determine code order unless the reasoning goes clearly the other way. 10. Procedural or goal statements that name categories with filling them should be codes as though the categories were filled. 11. Assign two codings to segments that seem to combine more than two basic categories in a way that cannot be resolved by further segmentation. 12. Use relational category of the for ‘‘C’’ to ‘‘C’’ in two ways: a. To code segments in which information in one basic category leads to other information from the same category, as when one symptom leads to consideration of another
b. To code segments in which information in one basic category leads to an elaboration of that information, as when a diagnosis leads to consideration of a complication that could result from that diagnosis.
Table 2 Demographic characteristics of nurse practitioner group. Characteristic
Number
Gender Female Male Mean years of nurse practitioner experience Prescribing authority Yes No Mean registered nurse experience
27 (90.00%) 3 (10.00%) 2.2 (95% CI: 1.60–2.80)
27 (90.00%) 3 (10.00%) 28.2 years (95% CI: 25.58–30.82) 17.03 years (95% CI: 14.11–19.96)
Mean registered nurse experience in specialty area Master’s degree obtained from a: New Zealand university New Zealand polytechnic Overseas
20 (66.70%) 7 (23.30%) 3 (10%)
Table 3 Demographic characteristics of doctor group. Characteristic Gender Female Male Mean number of years house officer experience Mean number of years experience in current specialist training programme Training specialty General practice Cardiology Respiratory Emergency care Gerontology General medicine Completed part one of specialty training Yes No Mean number of years experience prior to part one specialist exams Mean number of years experience post part one specialist exams
Number 9 (56.00%) 7 (44.00%) 2.88 (95% CI: 2.18–3.57). 3.42 (95% CI: 2.06–4.78)
5 3 1 2 1 4
(31.25%) (18.75%) (6.00%) (13.00%) (6.00%) (25.00%)
13 (81.25%) 3 (18.75%) 1.63 (95% CI: 0.93–2.32) 1.80 (95% CI: 0.98–2.61)
Of the 13 correct actions, nurse practitioners identified a mean of 4.30 (95% CI: 3.52–5.08) compared to a mean of 4.13 (95% CI: 2.76–5.49) proposed by doctors. Analysis revealed no difference between the two groups (95% CI: 1.23 to 1.58, p = 0.80). Analysis revealed a difference between the two groups in planning for computerised tomography (CT)/CT pulmonary angiogram (Table 5). When compared to doctors, fewer nurse practitioners planned for this investigation. As more nurse practitioners discussed the patient with a medical consultant colleague (n = 22, 73%) when compared
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
723
Table 4 Correct diagnoses identified by participants. Correct diagnoses as determined by expert panel
Nurse practitioner frequency (%) n = 30
Doctor frequency (%) n = 16
Significance
Hypertension Gastric bleeding Gastric ulcer Reasonably well controlled Type II diabetes mellitus Well controlled hyperlipidaemia ? chronic obstructive pulmonary disease ? lower respiratory tract infection ? lung cancer ? pleural effusion ? pulmonary embolus
30 25 21 18 17 26 11 8 4 4
15 10 10 6 10 7 12 9 10 10
FET p = 0.35 FET p = 0.15 x2 = 04, p = 0.86 x2 =1.31, p = 0.26 x2 = 06, p = 0.94 FET p = 0.005* x2 = 4.70, p = 0.04* x2 = 2.75, p = 0.10 FET p = 0.002* FET p = 0.002*
(100.0) (83.3) (70.0) (60.0) (56.7) (86.7) (36.7) (26.7) (13.3) (13.3)
(94) (62) (62) (38) (63) (44) (75) (56) (63) (63)
* Statistical significance. Table 5 Correct actions identified by participants. Action plan
Nurse practitioner frequency (%) n = 30
Doctor frequency (%) n = 16
Significance
Review need for increased anti-hypertensive therapy Refer hospital for specialist team review and/or hospital admission Diabetes referral/education Lung function tests Stop diclofenac Sputum culture Change antibiotic to include macrolide Stop aspirin Gastroscopy Proton pump inhibitor Test for Haemophilus pylori Computerised tomography (CT)/CT pulmonary angiogram
17 17 16 16 13 13 10 8 7 5 3 1
7 9 6 4 4 6 8 3 9 4 0 6
x2 = 28, p = 0.60 x2 = 00, p = 1.0 x2 = 51, p = 0.48 x2 = 2.68, p = 0.10 x2 = 2.35, p = 0.12 x2 = 01, p = 0.94 x2 = 62, p = 0.44
(56.7) (56.7) (53.3) (53.3) (43.3) (43.3) (33.3) (26.7) (23.3) (16.7) (10.0) (3.3)
(44) (56) (38) (25) (25) (38) (50) (19) (56) (25) (38)
FET p=.72 x2 = 3.64, p = 0.06 FET p = 0.70 FET p = 0.54 FET p = 0.01*
* Statistical significance.
to the doctor group (n = 1, 6%), x2 = 18.78, p =
Contents lists available at ScienceDirect
International Journal of Nursing Studies journal homepage: www.elsevier.com/ijns
Nurse practitioners versus doctors diagnostic reasoning in a complex case presentation to an acute tertiary hospital: A comparative study Alison M. Pirret a,b,*, Stephen J. Neville a, Steven J. La Grow c a b c
School of Nursing, College of Health, Massey University, New Zealand Critical Care Complex, Middlemore Hospital, Auckland, New Zealand College of Health, Massey University, New Zealand
A R T I C L E I N F O
A B S T R A C T
Article history: Received 20 December 2013 Received in revised form 20 August 2014 Accepted 25 August 2014
Background: Nurse practitioners perform a diagnostic role previously delivered by doctors. Multiple studies demonstrate nurse practitioners are as effective as doctors when managing chronic conditions and minor illnesses and injuries. No studies have focused on how nurse practitioners compare to doctors in their management of complex cases presenting for the first time. Objective: This study assessed how nurse practitioners’ diagnostic reasoning abilities when managing a complex case compared to those of doctors’? Design: A comparative research design. Participants: Purposeful sampling recruited 30 nurse practitioners and 16 doctors working in multiple specialties in New Zealand. All doctors were completing postgraduate specialist training programmes. Specialties included older adults, emergency care, primary health care/general practice, cardiology, respiratory and palliative care. Methods: A complex case scenario assessed by an expert panel and think aloud protocol was used to assess diagnostic reasoning abilities. The ability of 30 nurse practitioners to determine diagnoses, identify the problem, and propose actions was compared to that of 16 doctors. Correct responses were determined by an expert panel. Data gained from the case scenario using think aloud protocol were quantified for analysis. Results: 61.9% of doctors identified the correct diagnoses, 56.3% the problem and 34.4% the actions as determined by the expert panel. This compares to 54.7% of nurse practitioners identifying the correct diagnoses, 53.3% the problem and 35.8% the actions. Analysis revealed no difference between these groups (diagnoses 95% CI: 1.76 to 0.32, p = 0.17, problem x2 = 0.00, p = 1.0, or actions 95% CI: 1.23 to 1.58, p = 0.80). Conclusion: Nurse practitioners’ diagnostic reasoning abilities compared favourably to those of doctors in terms of diagnoses made, problems identified and action plans proposed from a complex case scenario. In times of global economic restraints this adds further support to alternative models of care. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Complex case Diagnostic accuracy Diagnostic reasoning Nurse practitioners Registrars
* Corresponding author at: School of Nursing, College of Health, Massey University, Private Bag 102904, North Shore, Auckland 0745, New Zealand. Tel.: +64 94140800; fax: +64 94418165. E-mail address: [email protected] (A.M. Pirret). http://dx.doi.org/10.1016/j.ijnurstu.2014.08.009 0020-7489/ß 2014 Elsevier Ltd. All rights reserved.
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
What is already known about the topic? Multiple studies demonstrate nurse practitioners achieve similar patient outcomes to doctors. Most studies comparing nurse practitioner patient outcomes to doctors compare their management of chronic conditions and minor illnesses and injuries Few studies compare nurse practitioner diagnostic reasoning abilities to those of doctors. Most studies comparing nurse practitioner diagnostic reasoning abilities compare them to house officers. What the paper adds Nurse practitioners analysis of a single complex case compare favourably to that of registrars in terms of diagnoses made, problems identified and action plans proposed from a complex case scenario. The findings of the study suggest nurse practitioners’ diagnostic reasoning developed from education and experience enables them to diagnose and manage complex patients presenting for the first time. 1. Introduction Nurse practitioners in New Zealand (NZ) were introduced to increase patients’ access to healthcare, improve patient outcomes (Ministry of Health, 2002) and provide a solution to doctor shortages (Forde, 2008; Ministry of Health, 2009). Combining advanced nursing practice with skills from medicine, nurse practitioners diagnose, assess and manage patients and can order diagnostic tests and prescribe; historically these roles were considered exclusive to medicine (Forde, 2008; Ministry of Health, 2002). In addition nurse practitioners promote health, encourage self-care and look beyond the diagnosis to consider nonmedical interventions (Ministry of Health, 2002). 1.1. International differences in the use of the nurse practitioner title There are international differences as to how the title nurse practitioner is used. Countries, such has NZ, Australia and the United States (US) have a rigorous assessment process and require a Master’s degree (Carryer et al., 2007; Kleinpell et al., 2008). New Zealand and Australia and some US states allow nurse practitioners to practice independently without supervision from a physician (Carryer et al., 2007; Kleinpell et al., 2008; Lowes, 2014). The US recommend nurse practitioner education programmes move to a Doctorate degree by 2015 (American Association of Colleges of Nursing, 2012), however laws and regulations pertaining to nurse practitioner scope of practice (including prescribing authority) remain inconsistent from state to state (Poghosyan et al., 2012). In Canada, legislation, regulations and standardisation of the nurse practitioner title are in place in most provinces and territories (Sangster-Gormley et al., 2011) unlike the United Kingdom (UK) where no legislation protects the title nurse practitioner. Many nurses working in advanced nursing practice roles in the UK have a Master’s degree but
717
nurses are still able to do a one-week course and use the title advanced nurse practitioner (Coombes, 2008). Over recent years, the Royal College of Nursing has lobbied for a registered trade title for nurse practitioner similar to NZ (Coombes, 2008) but this has not yet occurred (Santry, 2010). 1.2. Nurse practitioner and physician patient outcomes Multiple studies demonstrate nurse practitioners achieve similar patient outcomes to medical doctors (Dierick-van Daele et al., 2009; Horrocks et al., 2002; Laurant et al., 2008) however many of these studies focus on patients referred by the doctor to nurse practitioners for management of chronic conditions, patients presenting for the first time with minor illnesses or injuries, and nurse practitioners working alongside general practitioners (Horrocks et al., 2002; Laurant et al., 2008; Newhouse et al., 2011). Few studies compare nurse practitioner diagnostic reasoning abilities to those of doctors. Sakr et al. (1999) showed UK emergency care nurse practitioners and house officers had similar rates of diagnostic error (9.2% compared to 10.7%) when assessing and treating patients presenting with minor illnesses and injuries (Sakr et al., 1999). This finding was further supported in a study of Dutch emergency care nurse practitioners and senior house officers; this study also found no difference in the rate of missed injuries and inappropriate management of patients presenting with minor illnesses and injuries (van der Linden et al., 2010). In a study of UK primary care nurse practitioners and general practitioners, Offredy (2002) attributed the differences between the diagnostic accuracy of the two groups to general practitioners having more knowledge and experience than nurse practitioners. This was related to nurse practitioners’ lack of familiarity with the case presentations due to the restrictions general practitioners placed on the type of consultations they performed. Although all the nurse practitioners in the study had completed the Royal College of Nursing nurse practitioner degree programme, their limited scope of practice means these results may not reflect the international context. Whilst research demonstrates nurse practitioners compare favourably to doctors in their management of minor illnesses/injuries and chronic conditions, to the best of our knowledge no research compares nurse practitioners’ and doctors’ diagnostic reasoning abilities pertaining to a complex case. 1.3. Diagnostic reasoning theory Familiar case presentations automatically use intuitive processing (Croskerry, 2009; Djulbegovic et al., 2012) which is developed through experience. It allows rapid diagnosis however it is influenced by environmental information, pattern recognition and the use of mental short cuts known as heuristics (Croskerry, 2009; Stanovich, 2010), which increase the risk of diagnostic error. If the patient presentation is not initially recognised, time permits or the clinician is uncertain, analytic processing
718
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
is triggered (Croskerry, 2009; Pelaccia et al., 2011; Stolper et al., 2011). Analytic processing involves slower, step-by-step, conscious, logical and defensible processes (Coderre et al., 2010; Croskerry, 2009; Ely et al., 2011; Heiberg Engel, 2008; Szaflarski, 1997). This approach requires explicit knowledge of pathophysiology, diseases/conditions and clinical manifestations (Croskerry, 2009; Szaflarski, 1997) developed through learning that continue to develop as clinicians mature (Croskerry, 2009). 1.4. Using complex cases to trigger analytic processing Analytic processing is triggered by complex cases. Ilgen et al. (2011) assessed the diagnostic accuracy of medical students and residents using both simple and complex cases. They defined a simple case as a typical presentation of a single diagnosis and a complex case as one that introduces features of multiple diagnoses (Ilgen et al., 2011). These definitions resonate those used by Mamede et al. (2008) measuring the effect of reflective practice on the diagnostic accuracy of 42 internal medical residents in Brazil. Their simple cases included a single diagnosis that was frequently encountered by residents. Their complex cases comprised of either a combination of different acute clinical conditions, patients with co-morbidities, an atypical disease presentation or a case rarely seen by residents (Mamede et al., 2008). Diagnostic accuracy differs in simple and complex cases. Mamede et al. (2008) found reflective practice achieved a mean accuracy of 73% in simple cases and 42% in complex cases. Ilgen et al. (2011) in their study of 51 medical students (novices), 26 first or second year residents (intermediates) and 38 third year residents (experienced) found participants achieved high scores in simple cases but lower scores in complex cases. In simple cases novices scored a mean of 69.9%, intermediates 80.8% and experienced 89.5%. In the complex cases novices scored a mean of 31%, intermediates 47% and experienced 55% (Ilgen et al., 2011). In a study to test the relationship between speed and accuracy of Canadian emergency care residents, Sherbino et al. (2012) deliberately made the cases difficult so analytic processing could be triggered. Following a pilot study, they removed cases that were too easy (achieving a diagnostic accuracy approaching 100%) and those that were too difficult (achieved a diagnostic accuracy approaching 0%). Findings from their study showed residents achieved a mean accuracy score of 49% (Sherbino et al., 2012). As intuitive processing is faster than analytic processing, speed in generating a diagnosis has historically been associated with a higher rate of diagnostic error. This is thought to be related to premature closure or using intuitive processing when the case requires analytic processing (Elstein, 2009; Lucchiari and Pravettoni, 2012; Norman and Eva, 2010). Premature closure is the acceptance of a diagnosis before sufficient verification has occurred and failure to consider other plausible alternatives once it has been reached (Levy et al., 2007). Sherbino et al. (2012) measured residents’ diagnostic accuracy using both intuitive and analytic processing; fast
times indicated intuitive processing while slower times signified analytic processing. They found the response times varied but the most difficult case was associated with the longest time. The ability of nurse practitioners to diagnoses complex cases has been challenged. Gorman (2009) views the doctor in the future as a health professional who has largely a cognitive function, translating patients’ signs and symptoms into a diagnosis; this role, he argues, cannot be substituted by nurse practitioners. Gorman sees medicine as having a strong diagnostic role at the front door of healthcare facilities, referring to nurse practitioners and other health professional-led intervention clinics when required. This view suggests doctors are better suited to diagnosis and treatment whereas nurse practitioners are better suited to ongoing interventions once the diagnosis is made. 2. Research aim This study compared nurse practitioners’ and doctors’ diagnostic reasoning abilities related to a complex case. The study answered the research question, how do nurse practitioners’ diagnostic reasoning abilities of a complex case compare to those of doctors’? This question was based on the underlying assumption, that as nurse practitioners focus on health promotion and disease prevention, when compared to doctors, their diagnostic abilities when analysing a complex case may be inferior. The primary outcome for this study was the number of correct diagnoses, problem and actions identified. In this study, the term diagnostic reasoning indicates the cognitive process involving data collection, identification of diagnoses and problems and the formulation of an action plan (Baid, 2006; Carneiro, 2003; Stausberg and Person, 1999). The term diagnosis denotes labelling of the disease or illness and the term problem means abnormal findings or problems needing intervention (Brykczynski, 1989, 1999; Elstein et al., 1993; Frauman and Skelly, 1999; Hoffman et al., 2009; Muller-Staub et al., 2008). The term action plan refers to applying interventions, prescribing and referring in response to identified diagnoses and problems (Baid, 2006; Carneiro, 2003; Weiss, 2011). 3. Method Using a comparative research design, this study used a complex case scenario and think aloud protocol to compare nurse practitioners’ and doctors’ diagnostic reasoning abilities. Think aloud protocol is a qualitative method (Arocha and Patel, 2008; Bucknall and Aitken, 2010; Hoffman et al., 2009; Lundgren-Laine and Salantera, 2010) used to analyse how clinicians’ use their knowledge to generate diagnoses and the complex relationships between knowledge transition and generation of diagnoses (Joseph and Patel, 1990). In this study, the ability of 30 nurse practitioners to determine diagnoses, identify the problem, and propose actions related to a complex case was compared to that of 16 doctors. Correct responding was determined by an expert panel consisting of a Professor of General Practice,
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Box 1. Summary of case scenario. A 67-year-old man presents with worsening productive cough, purulent sputum and left sided chest pain. He has a 2-week history of flu-like illness, backache, posterior chest pain, productive cough and increased shortness of breath. He visited his GP 1 week ago and was prescribed amoxicillin 500 mg three times daily but has had no improvement. He has multi-morbidities including insulin dependent Type II diabetes, hypertension and hyperlipidaemia. He has stopped taking his insulin as he does not think he needs it. He is an ex-smoker with a 40-pack year history. He has a reduced appetite with a 5 kg weight loss over past few weeks. He is experiencing generalised abdominal pain and has black hard stools. He is hypertensive with a slightly raised respiratory rate. Left sided chest pain is present on inspiration and coughing, with crackles left base on auscultation and dull left base on percussion. He feels slightly clammy to touch but peripherally warm. Chest X-ray indicates opacity in the left lower lobe. Chest X-ray report indicates no consolidation and suggests changes in left lower lobe relate to old scarring. It also reports hyperinflation of the lungs consistent with a degree of COPD.
an Associate Professor of Rheumatology and a Diabetes nurse practitioner. The decision to use an expert panel was based on previous research showing the diagnostic accuracy of doctors was inferior when compared to medical specialists (Allen et al., 1998) hence it was crucial nurse practitioners’ and doctors’ diagnostic reasoning abilities were compared to what experts considered best practice. The expert panel was also used to gain consensus on the case scenario’s complexity and ability to trigger participants’ analytic processing. The medical experts on the expert panel were academically and clinically involved in training doctors and experienced at using simple and complex cases to assess doctors’ diagnostic reasoning abilities. They
considered the case scenario complex as it contained multiple diagnoses within a single case presentation. A summary of the case scenario is outlined in Box 1. The case scenario was based on a real case presenting to an acute tertiary hospital. It was obtained by the researcher (who is also a nurse practitioner) after the patient was identified by a ward nurse manager as being a complex case with diagnostic uncertainty. Data from the presenting patient’s health history and physical examination were obtained from the clinical notes. Data pertaining to laboratory and radiology results were downloaded from the hospital data management system. All patient identifying data were removed. The data were transcribed and inserted into a word document for review by the expert panel. The hospital provided written approval for this case scenario to be used in this study. As patient identifying data were removed, the hospital research office and Massey University Ethics Committee did not require patient consent for the case scenario to be used. The Delphi technique gained consensus from the expert panel. For the correct diagnoses, problems and actions, two rounds of Delphi were used to achieve a 96% consensus. The panel did not agree on the need for spirometry to confirm the diagnoses of chronic obstructive pulmonary disease. The diagnoses, problem and action plan the expert panel identified are outlined in Box 2. One round of Delphi achieved a 100% consensus on the complexity of the case scenario and its ability to trigger analytic reasoning. 3.1. Participant selection As the study incorporated quantitative analysis, prospective power analysis calculated the desired sample size. This calculation was based on the study’s underlying assumption that when compared to doctors, nurse practitioners may have inferior diagnostic abilities when analysing a complex case. Previous research identified a 15.7% difference between junior and senior residents when analysing a complex case (Ilgen et al., 2011); hence for this study the percentage difference between the two groups
Box 2. Diagnoses, problems and actions the expert panel expected of doctors.
Expected diagnoses/problems
Expected action plan
? lower respiratory tract infection ? pleural effusion ? pulmonary embolism ? lung cancer ? chronic obstructive pulmonary disease Gastric bleeding ? Gastric ulcer
Computerised tomography or with pulmonary angiogram Sputum culture Change antibiotics to include a macrolide Lung function tests – spirometry
Hypertension Reasonably well controlled Type II diabetes mellitus Well controlled hyperlipidaemia Poor adherence to medications
719
Gastroscopy Proton pump inhibitor Stop both aspirin Stop diclofenac Test Haemophilus-Pylori Refer hospital for specialist team review/? hospital admission Recheck and monitor blood pressure and if required review antihypertensive medications Diabetic referral/Diabetic education
720
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Table 1 Prospective and retrospective power analysis for statistical tests. Statistical test
Effect size
Prospective Power
Sample size
Power
Sample size
Two tailed between group independent t-test One tailed between group independent t-test Two tailed within group independent t-test Two tailed Pearson’s product moment correlation coefficient Chi-square (with one degree of freedom)
0.80 0.80 0.80 0.80 0.80
0.86 0.92 0.99 0.99
60 60 30 19
0.75 0.85 0.99 0.98 0.97
46 46 30 16 46
was expected to be greater with a mean difference (MD) >1.0 when determining diagnoses and >1.3 when planning actions. Purposive sampling was used to recruit 30 nurse practitioners and 30 doctors working in adult practice areas in NZ with a patient population reflected in the case scenario using think aloud. Only doctors completing postgraduate specialist training programmes were recruited. Nurse practitioners working in areas, such as child and mental health, and specialist areas, such as ophthalmology and organ transplant, were excluded from the study. Multiple professional networks failed to recruit the planned number of doctors, particularly in the general practice specialty; as a result only 16 doctors were recruited. The calculations for the prospective and retrospective power analyses are outlined in Table 1. As most of the nurse practitioners meeting the selection criteria were recruited into the study, no randomisation was performed. 3.2. Ethical approval Ethics approval was gained from the Massey University Human Ethics Committee. All participants provided written consent. 3.3. Data collection Data were collected between February 2011 and March 2012 (inclusive). This study used both concurrent and retrospective think aloud. Concurrent think aloud reflects cognitive processes occurring at the current time and provides more accurate data when compared to retrospective think aloud which gathers verbalisations after the task (Bucknall and Aitken, 2010; Hoffman et al., 2009; Kuusela and Paul, 2000). Each participant completed the case scenario using think aloud protocol in a private office in their workplace with the researcher present. All participants completed the same case scenario and signed a participant confidentiality agreement. Prior to completing the case scenario, each participant received verbal and written directions and completed a short practice session on another case scenario to ensure unfamiliarity with the computer programme did not influence the data collected. No time limit was allocated to complete the scenario as concurrent thinking may slow down the diagnostic reasoning process (Kuusela and Paul, 2000). The case scenario data, including the patient’s health history, physical examination and diagnostic tests results,
Retrospective
were presented using the Scenario Based Learning Interactive (SBLi) programme. Diagnostic tests results included laboratory results, a lateral and posterior–anterior chest Xray and a chest X-ray report. The case scenario data were divided into 23 segments and presented one segment at a time; each segment provided multiple pieces of clinical information relevant to the case. Presenting one segment at a time provides more control of the stimuli and more information about participants’ cognitive processes (Joseph and Patel, 1990). Participants chose the order and rate in which each segment was presented and were able to access the information presented in prior segments. Both the computer programme and the researcher recorded the time taken to complete the case scenario. Participants were prompted to think aloud at regular intervals if required. After presentation of the entire case, participants provided a summary of their final diagnoses, problems and action plan. This used retrospective think aloud and provided the opportunity for participants to articulate diagnoses, problems and actions they may not have articulated during concurrent think aloud. At the end of the case scenario, participants were asked to comment on how the case presentation reflected the type of patients they would see regularly in their practice setting. A portable MP3 player audiotaped participants’ verbalisations. 3.4. Data analysis The audiotaped case scenario think aloud data were transcribed verbatim and analysed using coding and categorising described by Elstein et al. (1993). Elstein et al. developed and applied coding schemes to Joseph and Patel’s (1990) study assessing the diagnostic accuracy of endocrinologists and cardiologists when analysing an endocrine case presentation (Joseph and Patel, 1990). On applying the coding schemes, Elstein et al. found the same results as Joseph and Patel. The coding schemes described by Elstein et al. are outlined in Boxes 3 and 4. Although think aloud is a qualitative method providing qualitative data, the qualitative data obtained from the think aloud transcriptions were transformed into quantitative data; this process is referred to as quantitising (Bazeley, 2009; Cresswell and Plano Clarke, 2011; Sandelowski, 2000). As the research needed to contribute to future health workforce development, quantitising the data meant the analysis and results were more likely to be understood and valued by medical, nursing and governmental bodies responsible for workforce planning.
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Box 3. Rules for coding process constructs copied from Elstein et al. (1993, p. 28). Rules for Coding Process Constructs 1. Formulates working diagnostic hypothesis. Subject arrives at a diagnostic hypothesis involving a disease entity of medical condition. This construct is scored for every unique diagnostic hypothesis that is given. Do not score if a subject repeats a diagnostic hypothesis given in a previous section. 2. Confirms or strengthens diagnostic hypothesis using new information. Score when subject retains, confirms, or strengthens hypothesis using vignette information. Simply repeating a previously mentioned diagnostic hypothesis is not enough to score here. The subject must say why or clearly be using vignette material to retain or confirm the hypothesis. 3. Rejects or weakens diagnostic hypothesis using new information. Score if subject rejects or weakens a hypothesis using findings or rules out a diagnosis based on finding presents in a segment. A subject may reject a hypothesis without explicitly proposing it. When this happens, go back and score it as a diagnostic hypothesis and indicate that it was never explicitly hypothesised by writing ‘‘rejects’’ by it. A subject also may reject a proposed hypothesis in the same segment and may prose and reject virtually at the same time. 4. Multiple-cue inference. Score when the subject draws conclusions about diagnosis, prognosis, or treatment on the basis of multiple cues. These cues may be information given in the vignette or information generated by the subject. The cue must be fairly explicit. An exception to the explicitcues rule occurs when the subject says this is a ‘‘classic’’ case or example. When a subject gives multiple inferences embedded in one sentence, score each instance 5. Single cue inference. Score when the subject draws conclusion about diagnosis, prognosis, or treatment on the basis of a single cue. This cue may be information given in the vignette or information generated by the subject. Score also when the subject adds one more piece of information to a clinical picture and makes an inference. That is, the clinical picture may contain numerous cues, but the subject explicitly picks one piece of information and adds it, resulting in the subject’s final inference
Each diagnosis, problem and action was given a numerical code. All coding was completed by the first author; the coding process was reviewed by the second and third authors for peer checking. Coded data were entered into a Statistical Package for the Social Sciences (SPSS) version 19. Each diagnosis, problem and action identified by each participant was entered as a 1 under labels reflecting those diagnoses, problems and actions. Adding this data within SPSS provided the number of accurate diagnoses, problems and actions obtained from each participant. Following this analysis, the number of correct, diagnoses, problems and actions were added to provide a single score reflecting each participant’s
721
diagnostic reasoning ability. Case scenario think aloud data were assessed for how the case scenario reflected cases regularly seen in participants’ practice areas. The two-tailed independent t-test, 95% confidence interval (CI), chi square test (x2) and Fisher’s Exact test (FET) were used to compare the nurse practitioner data to the doctor data. Pearson’s product moment correlation coefficient (r) Spearman’s rank order (rho) coefficient (rs) and the Kruskal–Wallis test (x2) were used for within group analysis. The alpha level for statistical significance was set at 0.05. 4. Results Overall 46 participants took part in the study, 30 nurse practitioners and 16 doctors. The nurse practitioners and doctors worked in gerontology, emergency care, primary health care/general practice, cardiology, respiratory, and palliative care. In both the nurse practitioner and doctor group the largest number were from primary health care, the smallest number from palliative care. 4.1. Participant demographics The characteristics of each of the groups are outlined in Tables 2 and 3. The small number of nurse practitioners in NZ and the need for participant anonymity prevents the numbers from each specialty area being shared. Four doctors had previously worked as a doctor in another specialty training programme prior to commencing their current specialty training. One doctor was previously a medical specialist in another specialty. 4.2. Diagnostic reasoning abilities Nurse practitioners identified a mean of 5.47 (95% CI: 4.86–6.08) correct diagnoses (n = 10) compared to a mean of 6.19 (95% CI: 5.27–7.10) generated by doctors. Analysis revealed no difference between the two groups (95% CI: 1.76 to 0.32, p = 0.17). For the five correct definitive diagnoses, no differences were found between the nurse practitioner and doctor groups however this was not the case for the five differential diagnoses (Table 4). When compared to doctors, fewer nurse practitioners correctly identified a lower respiratory tract infection, pleural effusion and pulmonary embolus. Fewer doctors than nurse practitioners correctly identified chronic obstructive pulmonary disease. Relationships between nurse practitioners’ ability to identify the correct diagnoses and their demographic data found a positive correlation with the number of years prescribing as a nurse practitioner in NZ (rs = 0.37, p = 0.04). This implies nurse practitioners’ abilities to correctly diagnose are improved as they gain more experience as a NZ prescribing nurse practitioner. The expert panel agreed the problem, ‘poor adherence to medications’ should be identified. Sixteen nurse practitioners (53%) identified this problem compared to nine (56%) doctors. Analysis revealed no difference between the two groups (x2 = 0.00, p = 1.0).
722
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
Box 4. Rules for coding knowledge utilisation copied from Elstein et al. (1993, p. 28). Rules for coding knowledge utilisation 1. Do not draw inference across sentences If a diagnosis is drawn on the basis of symptoms mentioned in an immediately preceding separate sentence, it should be coded ‘‘diagnosis’’ not ‘‘finding to diagnosis.’’ But if such sentences begin with connectives such as ‘‘therefore’’ they should get relational codings. 2. Draw inferences across segments within the same sentence. That is, if, for example, a diagnosis is drawn on the basis of a symptom mentioned in a preceding segment of the same sentence, it should be coded ‘‘symptom to diagnosis.’’ This holds even when there is no tell-tale word such as ‘‘so’’ or ‘‘therefore,’’ so long as the diagnosis seems clearly to have been elicited by the symptoms mentioned. 3. Use ‘‘Context’’ to code statements about nonmedical facts about the patient. E.g. the patient’s age, job history, living conditions, normal bodily statistics, daily activities and nonmedical events prior to admission. Normal bodily statistics of an every day sort are context; abnormal are findings. E.g. weight is context, overweight is a finding. 4. Use ‘‘Finding’’ to code statements about medical history and direct physical examination. Include testimony of the patient or the patient’s friends, family, etc. 5. Use ‘‘Test’’ to code statement based on laboratory tests or special diagnostic procedures. E.g. X-rays, blood cultures, sputum culture, CT scan. ‘‘Test’’ refers only to direct test results; code inference from test results as ‘‘diagnosis’’ (see rule 6). 6. Use ‘‘Diagnosis’’ to code any statement about the possible underlying pathophysiology or cause of the problem. Not just for statements about diseases. 7. Use ‘‘Treatment’’ to code statements about therapy and management. When treatment information is used to make diagnostic inference, code as ‘‘Treatment to Diagnosis’’ 8. Use ‘‘Other’’ to code ‘‘Same’’ and ‘‘I don’t know,’’ as well as the following: a. General programmatic or procedure statements. b. B. Nonmedical or metacognitive statements 9. Follow relational order given in the sentence syntax to determine code order unless the reasoning goes clearly the other way. 10. Procedural or goal statements that name categories with filling them should be codes as though the categories were filled. 11. Assign two codings to segments that seem to combine more than two basic categories in a way that cannot be resolved by further segmentation. 12. Use relational category of the for ‘‘C’’ to ‘‘C’’ in two ways: a. To code segments in which information in one basic category leads to other information from the same category, as when one symptom leads to consideration of another
b. To code segments in which information in one basic category leads to an elaboration of that information, as when a diagnosis leads to consideration of a complication that could result from that diagnosis.
Table 2 Demographic characteristics of nurse practitioner group. Characteristic
Number
Gender Female Male Mean years of nurse practitioner experience Prescribing authority Yes No Mean registered nurse experience
27 (90.00%) 3 (10.00%) 2.2 (95% CI: 1.60–2.80)
27 (90.00%) 3 (10.00%) 28.2 years (95% CI: 25.58–30.82) 17.03 years (95% CI: 14.11–19.96)
Mean registered nurse experience in specialty area Master’s degree obtained from a: New Zealand university New Zealand polytechnic Overseas
20 (66.70%) 7 (23.30%) 3 (10%)
Table 3 Demographic characteristics of doctor group. Characteristic Gender Female Male Mean number of years house officer experience Mean number of years experience in current specialist training programme Training specialty General practice Cardiology Respiratory Emergency care Gerontology General medicine Completed part one of specialty training Yes No Mean number of years experience prior to part one specialist exams Mean number of years experience post part one specialist exams
Number 9 (56.00%) 7 (44.00%) 2.88 (95% CI: 2.18–3.57). 3.42 (95% CI: 2.06–4.78)
5 3 1 2 1 4
(31.25%) (18.75%) (6.00%) (13.00%) (6.00%) (25.00%)
13 (81.25%) 3 (18.75%) 1.63 (95% CI: 0.93–2.32) 1.80 (95% CI: 0.98–2.61)
Of the 13 correct actions, nurse practitioners identified a mean of 4.30 (95% CI: 3.52–5.08) compared to a mean of 4.13 (95% CI: 2.76–5.49) proposed by doctors. Analysis revealed no difference between the two groups (95% CI: 1.23 to 1.58, p = 0.80). Analysis revealed a difference between the two groups in planning for computerised tomography (CT)/CT pulmonary angiogram (Table 5). When compared to doctors, fewer nurse practitioners planned for this investigation. As more nurse practitioners discussed the patient with a medical consultant colleague (n = 22, 73%) when compared
A.M. Pirret et al. / International Journal of Nursing Studies 52 (2015) 716–726
723
Table 4 Correct diagnoses identified by participants. Correct diagnoses as determined by expert panel
Nurse practitioner frequency (%) n = 30
Doctor frequency (%) n = 16
Significance
Hypertension Gastric bleeding Gastric ulcer Reasonably well controlled Type II diabetes mellitus Well controlled hyperlipidaemia ? chronic obstructive pulmonary disease ? lower respiratory tract infection ? lung cancer ? pleural effusion ? pulmonary embolus
30 25 21 18 17 26 11 8 4 4
15 10 10 6 10 7 12 9 10 10
FET p = 0.35 FET p = 0.15 x2 = 04, p = 0.86 x2 =1.31, p = 0.26 x2 = 06, p = 0.94 FET p = 0.005* x2 = 4.70, p = 0.04* x2 = 2.75, p = 0.10 FET p = 0.002* FET p = 0.002*
(100.0) (83.3) (70.0) (60.0) (56.7) (86.7) (36.7) (26.7) (13.3) (13.3)
(94) (62) (62) (38) (63) (44) (75) (56) (63) (63)
* Statistical significance. Table 5 Correct actions identified by participants. Action plan
Nurse practitioner frequency (%) n = 30
Doctor frequency (%) n = 16
Significance
Review need for increased anti-hypertensive therapy Refer hospital for specialist team review and/or hospital admission Diabetes referral/education Lung function tests Stop diclofenac Sputum culture Change antibiotic to include macrolide Stop aspirin Gastroscopy Proton pump inhibitor Test for Haemophilus pylori Computerised tomography (CT)/CT pulmonary angiogram
17 17 16 16 13 13 10 8 7 5 3 1
7 9 6 4 4 6 8 3 9 4 0 6
x2 = 28, p = 0.60 x2 = 00, p = 1.0 x2 = 51, p = 0.48 x2 = 2.68, p = 0.10 x2 = 2.35, p = 0.12 x2 = 01, p = 0.94 x2 = 62, p = 0.44
(56.7) (56.7) (53.3) (53.3) (43.3) (43.3) (33.3) (26.7) (23.3) (16.7) (10.0) (3.3)
(44) (56) (38) (25) (25) (38) (50) (19) (56) (25) (38)
FET p=.72 x2 = 3.64, p = 0.06 FET p = 0.70 FET p = 0.54 FET p = 0.01*
* Statistical significance.
to the doctor group (n = 1, 6%), x2 = 18.78, p =
