Research in Nursing & Health, 1991, 14, 59- 66
Effect of Teaching Decision Analysis on Student Nurses’ Clinical Intervention Decision Making Judith Shamian
The purpose of this study was to evaluate the effect of teaching decision analysis on nursing students’ ability to prioritize clinical interventions given the probabilities for the decision situation. Sixty-eight university nursing students were randomly assigned to one of two groups. The 37 nursing students in the experimental group received a 4-hour didactic and interactive teaching session on decision analysis. The 31 nursing students in the control group also received a 4-hour interactive session on a nondecision analysis topic. A posttest experimental design was selected to minimize testing bias. Three clinical case studies were developed, tested, and utilized for data collection. Subjects in the experimental group selected clinical decisions that were in accordance with clinical decisions made by experts more often and more consistently than did subjects in the control group (p < .0001).
The practice of nursing requires that nurses continually make clinical decisions (Aspinall & Tanner, 1981; College of Nurses of Ontario, 1985). Clinical judgment is a complex process that uses an enormous knowledge base in conjunction with certain decision making processes in order to arrive at desired decisions regarding patient care (Kassirer & Gorry, 1978). Clinical decision making derives from a sound command of the knowledge base related to the decision, in addition to the ability to select and combine facts appropriately from this knowledge base. Although decision making is central to the discipline of nursing, the present understanding of clinical decision making processes uses by nurses is limited. Even less is known about facilitating the attainment of optimal nursing decisions (Ozbolt, Schultz, Swain, Abraham, & Farchaus-Stein, 1984). Decision analysis is one method that has been identified by nurses as a potentially useful tool for improving clinical decision making (Aspinall,
1979; Grier, 1976, 1984; Grier & Schnitzler, 1979; Tanner, 1987). Decision analysis affords greater precision in decision making than is otherwise readily attainable (Corcoran, 1986; Kassirer, Moskowitz, Lau, & Pauker, 1987; Schwartz, Gony, Kassirer, & Essig, 1973) and as such has been employed by various health care professionals (Corcoran, 1986; Kassirer et al., 1987; Krischer, 1980; Pliskin, Shepard, & Weinstein, 1980). Decision analysis is based on normative decision making models that have been derived from principles of mathematics to generate probabilities of certain and uncertain events. Decision analysis models use a reductionistic strategy to decompose complex problems into a series of simpler steps, thereby reducing reliance on heuristics. Moreover, attention is focused on the values inherent in clinical situations of any consequence (Patel, 1980). A number of investigators have examined some elements of normative decision making in nursing. Specifically, three nursing research studies were identified in which investigators addressed elements
Judith Shamian, RN, PhD, is Vice President, Nursing, Mount Sinai Hospital, Toronto, and an assistant professor in the School of Nursing, University of Toronto. The author acknowledges Dr. P. F. Brennen, Dr. J. J. Fitzpatrick, and Ms. L. Nagle for helpful comments on the manuscript. This manuscript was received on July 14, 1989, was revised, and accepted for publication June 20, 1990. Requests for reprints can be addressed to Dr. Judith Shamian, Vice President, Nursing, Mount Sinai Hospital, 600 University Avenue, Toronto, Ontario, M5G 1X5 Canada.
0 1991 John Wiley & Sons. Inc. 0160-6891191101059-08 $04.00
60
RESEARCH IN NURSING & HEALTH
of decision analysis in conjunction with decision theory (Aspinall, 1979; Grier, 1976; Hammond, Kelly, Schneider, & Vancini, 1967). The study of Hammond et al. (1967) was designed to compare nurses’ revisions of clinical decisions with those expected or prescribed by a mathematical model. Bayesian Theory was used to mathematically replicate six nurses’ estimates of the probability of certain nursing problems. Findings regarding consistency demonstrated that nurses manipulated the probabilities in a rational manner that was congruent with the axiom of probability theory. On the other hand, findings concerning accuracy indicated that the subjects’ probability revisions were cognitively cautious and lagged behind the mathematical model. Grier (1976) investigated the correlation between 50 nurses’ intuitive decisions and decisions prescribed by the use of utility theory, a form of normative decision making. Nurses intuitively chose nursing actions that had the highest expected value in 109 out of 185 decisions. The findings supported the hypothesis that the expected value and nursing action were in agreement. The findings also indicated “that a systematic and objective process was used in making most of the decisions, resulting in a justifiable choice for achieving the desired goal” (p. 108). Grier (1976) suggested that decision theory “focuses on essential aspects of the nursing process” (p. 109) and hence nurses should be provided instruction in decision theory. Finally, Aspinall (1979) studied whether the availability of decision trees would improve the accuracy of nurses’ clinical decisions. The investigator found that nurses in the experimental group who utilized the decision trees selected significantly more clinical diagnoses that were considered accurate than the two control groups who did not use decision trees. In summary, the investigators in all three studies examined elements associated with decision analysis and decision theory. The researchers demonstrated initial scientific support for decision analysis strategies and recommended further examination of such strategies in order to assist nurses in clinical decision making. However, it it striking to note that further research in clinical decision analysis has not been pursued in nursing for the past 10 years, despite ongoing research in related health fields and the recommendations of the authors. Hence, this preliminary exploration proposed to examine thc effect of systematically teaching decision analysis on nursing students’ ability to prioritize clinical interventions, given the probabilities for the decision situation, as compared with clinical experts.
METHOD A posttest, experimental design was used in order to address the following research question: What are the differences, if any, in the ranking of optimal clinical nursing intervention decisions between those subjects receiving decision analysis teaching and those not receiving decision analysis teaching?
Sample The study was conducted at a Canadian university faculty of nursing in a large metropolitan area. Sample size was determined by power analysis (Cohen, 1977). Given an alpha level of .05 and a power level of .80 a sample size of 72 was determined to be sufficient. The sample consisted of a convenience group of 68 third and fourthyear university nursing students. All subjects had completed an adult medical-surgical nursing course and had participated in a supervised clinical medical-surgical experience. Participants were randomly assigned either to an experimental or a control group. The experimental and the control groups were equivalent with respect to the demographic variables of age, previous education, and previous health care work experience. The mean age of both groups was 24. Previous research has shown that these demographic variables may be associated with the quality of clinical decision making (Davis, 1972; Grier, 1976; Grier & Schnitzler, 1979; Verhonick, Nichols, Glor, & McCarthy, 1968). In order to ascertain whether the groups were equivalent concerning previous education and health care work experience, Chisquare statistics were calculated (Table 1). The participants ranged in age from 21 to 42, with a mean age of 24. Seven participants had previous nursing education. Furthermore, 40 of the 68 subjects had previous work experience in the health care field. This experience varied in length from 2 months to 17 years, the mean was 2 years and 8 months.
Measures The Nursing Clinical Case Studies instrument was developed for the purposes of the study in order to elicit data pertaining to the prioritization of clinical interventions. The Clinical Case Studies Instrument consists of three clinical case studies each addressing general, medical, and surgical clinical problems (see Appendix). The case studies were developed by the investigator and six clinical experts who were second-year masers students
61
EFFECT OF TEACHING DECISION ANALYSIS / SHAMIAN
Table 1. Chi-square for Previous Education and Previous Health Care Work Experience by Group Control
Experimental Education
n
(W
X2
df
P
17 13
(57) (43)
.21
1
.65
College courses University courses Nursing diploma Baccalaureate science None Health care work experience Yes No Note: Experimental Group n = 37 Control Grouo n = 31.
in medical surgical nursing and had previous clinical experience in medical surgical nursing. It should be noted that decision analysis is most appropriate for complex decisions. Therefore, case studies were developed with considerable complexity as indicated by multiple nursing diagnoses and interventions. The investigator and expert panel attempted to generate clinical interventions that were, for the purposes of this study, considered to be mutually exclusive. Each of the case studies described in the clinical case studies instrument was followed by two to four nursing diagnoses. The nursing diagnoses for each clinical case study had four or five interventions together with two short-term goals and one outcome goal. The information generated by the clinical case studies was organized according to nursing diagnosis and outcome goals in order to provide a meaningful structure in which to analyze the case studies and generate the decision trees and probabilities. The expert panel used the process of decision analysis to identify decision trees and probability values for each intervention given the likelihood of attaining each short-term value (goal) (see Fig. 1). The data collection form that was attached to the case study was presented in a table format similar to a nursing care plan, so that study participants would be familiar with the clinical nursing interventions format (see Fig. 2). The subjects were asked to read each situation and prioritize the four or five interventions that pertained to each nursing diagnosis from the most preferred to least preferred, taking into account the desired short term goals and the outcome values (goals). A score of 1 was assigned to those interventions that were ranked in accordance with the experts’ rankings. Whereas, a score of 0 was assigned to those interventions that were not ranked in accordance with the experts’ rankings. The scores were
summed across all the interventions for all nursing diagnoses pertaining to each clinical case. The maximum score possible for Case Study Number 1 was 17; for Case Study Number 2, 8; Case Study Number 3, 20. Content validity of the clinical case studies was evaluated by an expert review panel that was different than the panel who developed the instrument. The expert panel consisted of four first-year masters nursing students who were enrolled in the medical surgical clinical program and were experienced in medical surgical nursing. The panel indicated that the clinical case studies were relevant and comprehensive. The panel also agreed that the case studies presented information that was consistent with similar case presentations in the clinical setting. Furthermore, they agreed that the identified nursing diagnoses, clinical interventions, and the respective probability values were appropriate for the clinical case studies. Following data collection, reliability of the nursing clinical case studies instrument was tested and Cronbach alpha coefficients were generated for each case. Reliability of the instrument was assessed by treating each of the three clinical situations as an independent instrument. Each intervention that was identified was considered an item. Case Study Number 1 had 17 items with an alpha value of .92. Case Study Number 2 had 8 items with an alpha value of .78, and Case Study Number 3 had 20 items with an alpha value of .90.
Procedure Third and fourth-year nursing students were invited to take part in the study. Those who volunteered were randomly assigned to either the experimental
62
RESEARCH IN NURSING & HEALTH
Incrmase In Rmst Pmrlod
Outcome: Increased A b l l l t y t o Perform ADL
Increase In Enmrgy
1
Good
5
Fair
25
1 Poor
\
0
V e r y Good
1
Good
5
Falr
25
Poor V e r y Good
0 1
Toach P r o g o s s l v e
Good
5
Muscio R e l a x a t i o n
Fair
25
Glve Back Rub
HS
Poor
0
Vary Good
1
Good
5
Falr
25
Poor
0
FIGURE 1. Example of a decision tree.
or the control group. Written consent was obtained from all participants. Both the experimental and the control group sessions were held on the same day at different locations in order to eliminate any information exchange among subjects in the two groups. The intervention was 4 hours in length and was organized in three blocks of time with breaks between the sessions. During the third session the clinical case studies instrument was administered. Following these sessions, the data collection instruments were administered to the study subjects. Experimental intervention. The intervention in the experimental group included videotaped instruction on decision analysis as well as an interactive session that focused on the application of decision analysis. The videotaped sessions addressed four aspects of decision making and decision analysis including: (a) the importance of decision making in nursing; (b) principles of decision analysis and its limitations; (c) clinical problems and the use of decision analysis; and (d) nursing clinical problems and decision analysis. The interactive session included a step-by-step application of decision analysis to a clinical nursing problem. The teaching program was based on methods outlined by Weinstein, Fineberg, Elstein, Frazier, Neuhauser, Neutra, and McNeil (1980) for teaching clinical decision analysis.
Control intervention. The intervention in the control group consisted of presenting four videotaped health-related ethical case situations including: (a) a chronic problem; (b) the courage of one’s conviction; (c) the old person’s friend; and (d) critical choice. Ethical case studies were selected for the control intervention because ethical cases were nonclinical, yet of concern and interest to nurses. It was thought that providing nonclinical cases for the control intervention might strengthen the effect of the experimental intervention. The subjects in the control group were required to view the videotapes and participate in an unstructured discussion following each videotape. The experimental and control groups spent comparable amounts of time viewing videotaped material and engaging in interactive sessions with the respective group leaders. R ESULTS The difference in the number of responses that were in accordance with the experts between the experimental and control group were compared utilizing the ?-test. Statistically significant differences were observed between the experimental and control group at p < .0001 in all three case
63
EFFECT OF TEACHING DECISION ANALYSIS / SHAMIAN
agreement within the experimental group, whereas the Kappa value of .729 with 30 degrees of freedom and a larger confidence interval of .699-.759 indicates that there was a fair to good level of agreement amongst the control group (Fleis, 1971). There was a high consistency of agreement among subjects’ clinical decision making in the experimental group, a consistency not due to chance, whereas, in the control group, the level of agreement could be due to chance alone. In summary, the above findings suggest that the experimental group demonstrated prioritizations of clinical interventions that were significantly more in accordance with the experts’ process of decision analysis than the control group. Further, the experimental group was more consistent in their clinical decision choices than the control group.
studies. The mean score of the experimental group was significantly higher than that of the control group (Table 2). In all case studies, the experimental group selected clinical decisions made by experts more often than the control group.
lnterrater Agreement Upon initial examination of the data, it appeared that there was an increased level of agreement regarding the choice of clinical interventions among the respondents in the experimental group than in the control, perhaps indicating that a more consistent clinical approach was being identified in the experimental group. Therefore, it was decided that further analysis of the level of interrater agreement would enhance the understanding of the results. Cohen’s weighted kappa statistics were generated to determine the rate of agreement between subjects in the experimental and control groups. The Kappa value of .999 with 36 degrees of freedom s ? d a small confidence interval of .997- 1 .OOO, indicates that there was a very strong level of interrater
DISCUSSION This study provides preliminary evidence that teaching decision analysis to nursing students in
Nursing Diagnosis:
Sleep pttern disturbance related to anxiety
Short Term Coals:
1. 2.
Increasr in rest periods Facilitate an increase in energy level
Time Frame for Goal Attainment: Chitcome Value (Coal):
Within next 24 hours
Increased rest and increased energy to perform ADL
In the s p e provided ( ) , prioritize each intervention from the most preferred intervent,iori ( 1 ) to t.he least prrferreci intervent~ion(4)
.
. .-
~ ~~
!
I Short Tern Goals I Nursing
:
Diagnosis
: : :
Sleep Pattern Disturb.
: :
Possible Intervention
:
Administer Sedative
I
:Increase in I Increase : Increased Ability :Rest,P e r i o d : in Energy I to Perform ADL I I
Yes ( . 5 ) Yes ( . 5 ) No ( . 5 ) No ( . 5 )
: : : :
Yes Yes
(.a) (.a)
I
No No
(.2) (.2)
: :
Yes No Yes No
Teach : Yes ( . 7 ) progressi ve : Yes (.7) muscle relaxation: No ( . 3 ) I No ( . 3 )
I
Yes ( . 8 ) I
I I
(.2) I Yes ( . 1 ) I No ( . 9 ) :
:
I
: I I
;
: I
:
I
: :
Organize : Care (i.e. m e d s , I V/Sl I
:
~-
-4-
Give back rub h.s.
Cutcome Value((;oal)
I--
; 8-
: : I I
Yes i . 3 ) Yes ( . 3 ) No ( . 7 ) No ( . 7 )
:
I
: :
Yes ( . 7 )
(.3) Yes ( . 2 ) No (.8) No
: j ,
: :
(.8) (.2) I (.1) : (.9)
:
No
Yes ( . 7 ) I No ( . 3 ) Yes ( . 1 ) I No
(.9) I
Very Good Fair Poor
Good ( 1 )
Very Good Fair Poor
Cood ( 1 )
Very Good Fair Poor
Good ( 1 )
Very Good Fair Poor
Good ( 1 )
a . -
-t___ppI
FIGURE 2.
~~~
~~
Example of a data collection form.
(.5) (.25) (0) (.5) (.25) (0) (.5) (.25)
(0) (.5) (.25) (0)
I
:
64
RESEARCH IN NURSING 8 HEALTH
Table 2. T-test for Difference between the Experimental and Control Groups Case Study
n
M
SD
Range
1.E. C. 2.E.
37 31 37 31 37 31
15.78 8.35 7.56 5.00 18.20 11.40
1.827 4.401 ,987 1.966 2.451 4.850
11-17 2-15 4-8 1-8 10-20 5-20
C. 3.E. C.
-
df
t
P
38.6
8.78
.0001
42.4
6.60
,0001
42.6
7.04
,0001
Note: E = Experimental Group C = Control (Sham) Group
addition to providing the probabilities for the decision situation enables nursing students to prioritize clinical interventions in accordance with clinical experts. Furthermore, the nursing students in this study readily used normative decision rules, once these rules were made available to them. The findings also suggest that nursing students benefited from the experimental intervention that included both the didactic videotaped teaching section which outlined the principles of probability and decision analysis, as well as the interactive session where those general principles were applied to a clinical case study. Because of the study design, it is impossible to determine whether the individual aspects of the teaching program specifically, the decision analysis principles, or the interactive problem solving, would have resulted in the same outcome as the combined elements. However, the teaching program developed for this study could serve as the foundation for a more in-depth and intensive teaching program. In the teaching arena, nurse educators should explore what type of decision analysis program would have the greater impact on clinical decision making. It is also striking that nursing students in the control group arrived at diverse prioritizations of the clinical interventions in the clinical case studies instrument, while the experimental group arrived at consistent prioritizations of the clinical interventions in the clinical case studies instrument. The lack of consistency in prioritization of clinical decisions among the control group suggests that the decision approaches that were used by the control group did not lead to accurate and consistent clinical decisions. Further research is needed in order to generate decision making approaches that will result in consistent and accurate clinical decisions. The study has a number of limitations due largely to its exploratory nature and the use of a convenience sample. Specifically, the sample of university nursing students is not representative of
the population of nurses and, in addition, the sample of students were self-selected from one institution. Further, the study design did not allow the investigator to study the effectiveness over time of teaching decision analysis. Finally, some difficulties exist concerning the instrumentation used in the study. Importantly, the validity of paper and pencil tests for assessing clinical decision making ability has not been firmly established. Further it was difficult to meet the underlying assumptions of decision analysis for the purposes of instrument development. In particular, the criteria of mutually exclusive alternatives remains the most problematic, given the nature of nursing practice. Additionally, normative decisions have not traditionally been used to prioritize decisions; however, these methods do generate numerical expected values. In this study, these numerical values have been used to indicate a priority ranking of interventions that will best achieve the desired outcomes. Although this approach has not been empirically tested, the application of normative decision methods may be well suited to complex clinical situations in order to determine more accurate, consistent decisions. In summary, a number of steps must be undertaken in order to take this study out of the realm of the exploratory and into that of clinical usefulness. The study needs to be replicated with various nursing students from different settings, such as other universities and community colleges in addition to practicing nurses, in order to determine whether the findings are consistent between these groups. Further studies will be required in order to: (a) generate clinical probabilities for many of the nursing interventions; and (b) further understand the impact of practitioner identified probabilities on the desired outcome. In the teaching arena, nurse educators should explore what type of decision program would result in the greatest impact on clinical decision making.
EFFECT OF TEACHING DECISION ANALYSIS / SHAMIAN
APPENDIX: CASE STUDY NO. 2 Section 1 Please read the following clinical situation. After you have completed reading the situation you will find some of the nursing diagnoses identified and possible interventions offered. The purpose of this exercise is that you examine the possible interventions offered for each diagnosis and on the basis of this clinical situation prioritize each from the most preferred one ( 1 ) to the least preferred.
Clinical Case Study No. 2.0 You are working the day shift on a 32 bed cardiac stepdown unit. Your patient assignment includes Keven Derand, a 45 year old, admitted two days ago to CCU for possible MI. The diagnosis of Myocardial Infarction was ruled out today and he is being transferred to your unit with a diagnosis of “Unstable Angina.” During his stay in CCU, he had several instances of sinus tachycardia, but no arrhythmias have been detected. His cardiac status continues to be monitored with telemetry. He currently has a Heparin lock in place for 1V access and has only required O2 on one occasion since his admission. In reading the nursing history, you determine that Mr. Derand is self-employed and runs a franchise of a convenience store chain. He is married and has two children living away from home: a son, 25 years old, and a daughter, 22 years old. His wife helps out in the store with him, but is physically disabled as a result of a car accident several years ago. He and his wife live in a large apartment above the store. In reading the medical history, you determine that Mr. Derand has never been hospitalized before. His father died at the age of 56 from a massive myocardial infarct. His mother had a CVA last year. at the age of 72 and now resides in a nursing home. He has no siblings. Mr. Derand is a non-smoker, not overweight, but was told by his family physician during a routine checkup last year, that he should limit his sugar intake. On admission he complained of substernal chest pain, was diaphoretic, dyspneic and complained of extreme weakness. Initially, he required Morphine 2 mg IV push, times three doses, before his pain was relieved. His EKC demonstrated peaked T waves and ST segment depression. During the first 48 hours following admission, his bloodwork revealed minimal elevation of his cardiac enzymes. Chest x-ray demonstrated normal heart border and clear lung fields. A fasting blood sugar drawn yesterday morning was reported at 16.8 mmol/l. An endocrinology consult has been requested. When you received Mr. Derand in transfer, his medical orders include: Nitropaste 1” topically QID Ativan 1 mg s/l qhs prn Diltiazem 30 mg QID
65
Nitroglycerin 0.6 mg s/l prn Betaloc 50 mg p.0. BID Bathroom privileges and up in chair X 30 min QID Diet: As tolerated O2 3 1% by mask prn The nurse transferring Mr. Derand describes him as a talkative, nervous man. Since his admission, his wife has been only able to visit once, but calls at least three times a day. His comments indicate that he is anxious to retum home and is concerned about his wife managing on her own. Although he sleeps frequently throughout the day, Mr. Derand has been having difficulty getting to sleep at night. The nurse reports that he has been up in the chair twice today and on the last occasion complained of a “funny” feeling in his chest within five minutes of being up. The nurse describes this incident to you with some skepticism, reporting that Mr. Derand became very panicky, wanting to return to bed and have oxygen and a “heart pill” right away. Within five minutes of providing these measures, Mr. Derand settled. At the time, his cardiac arrhythmia strip showed no evidence of ischemia or anythmia and changes in his vital signs were not significant. When you first visit Mr. Derand, he asks if you have come to take away his heart monitor and assures you that it is not bothersome. He then asks if you are aware of his needs for oxygen and “heart pills.” He tells you that he had a very bad “spell” earlier today and does not think he should get up this evening.
REFERENCES Aspinall, M.J. (1979). Use of a decision tree to improve accuracy of diagnosis. Nursing Research, 28, 182-185. Aspinall, M. J . , and Tanner, C . A. ( I98 I ). Decision making f o r patient care: Applying the nursing process. New York: Appleton-Century Crofts. Cohen, J. (1 977). Statistical power analysis f o r the behavioral sciences. (Rev. Ed.) New York: Academic Press. College of Nurses of Ontario (1985). The standards and levels of nursing practice including the assumptive base. Discussion paper. Corcoran, S.A. (1986). Task complexity and nursing expertise as factors in decision making. Nursing Research. 35, 107-1 12. Davis, B.G. (1972). Clinical expertise as a function of education preparation. Nursing Research, 2 1 , 530-534. Fleis, J. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 375-382. Crier, M . R . (1976). Decision making about patient care. Nursing Research, 2 5 , 105- 110. Crier, M . R . (1984). Information processing in nursing practice. In H.H. Werley & J.J. Fitzpatrick
66
RESEARCH IN NURSING & HEALTH
(Eds.), Annual Review ofNursing Research (pp. 265-287). New York: Springer. Crier, M.R., & Schnitzler, C.P. (1979). Nurses’ propensity to risk. Nursing Research, 28, 186191. Hammond, K.R., Kelly, K.J., Schneider, R.J., & Vancini, M. (1967). Clinical inference in nursing: Revising judgements. Nursing Research, 16, 38-45. Kassirer, J.P., & Gorry, G.A. (1978). Clinical problem solving: A behavior analysis. Annals of Internal Medicine, 89, 245-255. Kassirer, J.P., Moskowitz, A.J., Lau, J., & Pauker, S.G. (1987). Decision analysis: A progress report. Annals of Internal Medicine, 106, 275-291. Krischer, J.P. (1980). An annotated bibliography of decision analytic applications to health care. Operations Research, 28, 97- 113. Ozbolt, J.G., Schultz, 11, S . , Swain, M.A.P., Abraham, I.L., & Farchaus-Stein, M.S. (1984). Developing an expert system for nursing practice. Proceedings of the Eighth Symposium on Compurer Applications in Medical Care, 654-657.
Patel, V. (1980). Clinical reasoning in medicine: A review of cognitive-information processing approach. Unpublished manuscript, Centre for Medical Education, Montreal. Pliskin, J . S . , Shepard, D.S., & Weinstein, M.C. (1980). Utility functions for life years and health status. Operations Research, 28, 206-224. Schwartz. W.B., Gorry, G . A . , Kassirer. J.P., & Essig, A. (1973). Decision analysis and clinical judgement. American Journal of Medicine, 5 5 , 459-472. Tanner, C.A. (1987). Teaching clinical judgement. In J.J. Fitzpatrick & R.L. Taunton (Eds.), Annual Review of Nursing Research (pp. 153- 174). New York: Springer. Verhonick, P.J., Nichols, G.A., Glor, B.A.K., & McCarthy, R.T. (1968). I came, I saw, 1 responded: Nursing observation and action survey. Nursing Research, 1 7 , 38-44. Weinstein, M.C., Fineberg, H.V., Elstein, A.S., Frazier, H . S . , Neuhauser, D., Neutra, R.R., & McNeil, B .J . (1980). Clinical decision analysis. Philadelphia: Saunders.
Effect of Teaching Decision Analysis on Student Nurses’ Clinical Intervention Decision Making Judith Shamian
The purpose of this study was to evaluate the effect of teaching decision analysis on nursing students’ ability to prioritize clinical interventions given the probabilities for the decision situation. Sixty-eight university nursing students were randomly assigned to one of two groups. The 37 nursing students in the experimental group received a 4-hour didactic and interactive teaching session on decision analysis. The 31 nursing students in the control group also received a 4-hour interactive session on a nondecision analysis topic. A posttest experimental design was selected to minimize testing bias. Three clinical case studies were developed, tested, and utilized for data collection. Subjects in the experimental group selected clinical decisions that were in accordance with clinical decisions made by experts more often and more consistently than did subjects in the control group (p < .0001).
The practice of nursing requires that nurses continually make clinical decisions (Aspinall & Tanner, 1981; College of Nurses of Ontario, 1985). Clinical judgment is a complex process that uses an enormous knowledge base in conjunction with certain decision making processes in order to arrive at desired decisions regarding patient care (Kassirer & Gorry, 1978). Clinical decision making derives from a sound command of the knowledge base related to the decision, in addition to the ability to select and combine facts appropriately from this knowledge base. Although decision making is central to the discipline of nursing, the present understanding of clinical decision making processes uses by nurses is limited. Even less is known about facilitating the attainment of optimal nursing decisions (Ozbolt, Schultz, Swain, Abraham, & Farchaus-Stein, 1984). Decision analysis is one method that has been identified by nurses as a potentially useful tool for improving clinical decision making (Aspinall,
1979; Grier, 1976, 1984; Grier & Schnitzler, 1979; Tanner, 1987). Decision analysis affords greater precision in decision making than is otherwise readily attainable (Corcoran, 1986; Kassirer, Moskowitz, Lau, & Pauker, 1987; Schwartz, Gony, Kassirer, & Essig, 1973) and as such has been employed by various health care professionals (Corcoran, 1986; Kassirer et al., 1987; Krischer, 1980; Pliskin, Shepard, & Weinstein, 1980). Decision analysis is based on normative decision making models that have been derived from principles of mathematics to generate probabilities of certain and uncertain events. Decision analysis models use a reductionistic strategy to decompose complex problems into a series of simpler steps, thereby reducing reliance on heuristics. Moreover, attention is focused on the values inherent in clinical situations of any consequence (Patel, 1980). A number of investigators have examined some elements of normative decision making in nursing. Specifically, three nursing research studies were identified in which investigators addressed elements
Judith Shamian, RN, PhD, is Vice President, Nursing, Mount Sinai Hospital, Toronto, and an assistant professor in the School of Nursing, University of Toronto. The author acknowledges Dr. P. F. Brennen, Dr. J. J. Fitzpatrick, and Ms. L. Nagle for helpful comments on the manuscript. This manuscript was received on July 14, 1989, was revised, and accepted for publication June 20, 1990. Requests for reprints can be addressed to Dr. Judith Shamian, Vice President, Nursing, Mount Sinai Hospital, 600 University Avenue, Toronto, Ontario, M5G 1X5 Canada.
0 1991 John Wiley & Sons. Inc. 0160-6891191101059-08 $04.00
60
RESEARCH IN NURSING & HEALTH
of decision analysis in conjunction with decision theory (Aspinall, 1979; Grier, 1976; Hammond, Kelly, Schneider, & Vancini, 1967). The study of Hammond et al. (1967) was designed to compare nurses’ revisions of clinical decisions with those expected or prescribed by a mathematical model. Bayesian Theory was used to mathematically replicate six nurses’ estimates of the probability of certain nursing problems. Findings regarding consistency demonstrated that nurses manipulated the probabilities in a rational manner that was congruent with the axiom of probability theory. On the other hand, findings concerning accuracy indicated that the subjects’ probability revisions were cognitively cautious and lagged behind the mathematical model. Grier (1976) investigated the correlation between 50 nurses’ intuitive decisions and decisions prescribed by the use of utility theory, a form of normative decision making. Nurses intuitively chose nursing actions that had the highest expected value in 109 out of 185 decisions. The findings supported the hypothesis that the expected value and nursing action were in agreement. The findings also indicated “that a systematic and objective process was used in making most of the decisions, resulting in a justifiable choice for achieving the desired goal” (p. 108). Grier (1976) suggested that decision theory “focuses on essential aspects of the nursing process” (p. 109) and hence nurses should be provided instruction in decision theory. Finally, Aspinall (1979) studied whether the availability of decision trees would improve the accuracy of nurses’ clinical decisions. The investigator found that nurses in the experimental group who utilized the decision trees selected significantly more clinical diagnoses that were considered accurate than the two control groups who did not use decision trees. In summary, the investigators in all three studies examined elements associated with decision analysis and decision theory. The researchers demonstrated initial scientific support for decision analysis strategies and recommended further examination of such strategies in order to assist nurses in clinical decision making. However, it it striking to note that further research in clinical decision analysis has not been pursued in nursing for the past 10 years, despite ongoing research in related health fields and the recommendations of the authors. Hence, this preliminary exploration proposed to examine thc effect of systematically teaching decision analysis on nursing students’ ability to prioritize clinical interventions, given the probabilities for the decision situation, as compared with clinical experts.
METHOD A posttest, experimental design was used in order to address the following research question: What are the differences, if any, in the ranking of optimal clinical nursing intervention decisions between those subjects receiving decision analysis teaching and those not receiving decision analysis teaching?
Sample The study was conducted at a Canadian university faculty of nursing in a large metropolitan area. Sample size was determined by power analysis (Cohen, 1977). Given an alpha level of .05 and a power level of .80 a sample size of 72 was determined to be sufficient. The sample consisted of a convenience group of 68 third and fourthyear university nursing students. All subjects had completed an adult medical-surgical nursing course and had participated in a supervised clinical medical-surgical experience. Participants were randomly assigned either to an experimental or a control group. The experimental and the control groups were equivalent with respect to the demographic variables of age, previous education, and previous health care work experience. The mean age of both groups was 24. Previous research has shown that these demographic variables may be associated with the quality of clinical decision making (Davis, 1972; Grier, 1976; Grier & Schnitzler, 1979; Verhonick, Nichols, Glor, & McCarthy, 1968). In order to ascertain whether the groups were equivalent concerning previous education and health care work experience, Chisquare statistics were calculated (Table 1). The participants ranged in age from 21 to 42, with a mean age of 24. Seven participants had previous nursing education. Furthermore, 40 of the 68 subjects had previous work experience in the health care field. This experience varied in length from 2 months to 17 years, the mean was 2 years and 8 months.
Measures The Nursing Clinical Case Studies instrument was developed for the purposes of the study in order to elicit data pertaining to the prioritization of clinical interventions. The Clinical Case Studies Instrument consists of three clinical case studies each addressing general, medical, and surgical clinical problems (see Appendix). The case studies were developed by the investigator and six clinical experts who were second-year masers students
61
EFFECT OF TEACHING DECISION ANALYSIS / SHAMIAN
Table 1. Chi-square for Previous Education and Previous Health Care Work Experience by Group Control
Experimental Education
n
(W
X2
df
P
17 13
(57) (43)
.21
1
.65
College courses University courses Nursing diploma Baccalaureate science None Health care work experience Yes No Note: Experimental Group n = 37 Control Grouo n = 31.
in medical surgical nursing and had previous clinical experience in medical surgical nursing. It should be noted that decision analysis is most appropriate for complex decisions. Therefore, case studies were developed with considerable complexity as indicated by multiple nursing diagnoses and interventions. The investigator and expert panel attempted to generate clinical interventions that were, for the purposes of this study, considered to be mutually exclusive. Each of the case studies described in the clinical case studies instrument was followed by two to four nursing diagnoses. The nursing diagnoses for each clinical case study had four or five interventions together with two short-term goals and one outcome goal. The information generated by the clinical case studies was organized according to nursing diagnosis and outcome goals in order to provide a meaningful structure in which to analyze the case studies and generate the decision trees and probabilities. The expert panel used the process of decision analysis to identify decision trees and probability values for each intervention given the likelihood of attaining each short-term value (goal) (see Fig. 1). The data collection form that was attached to the case study was presented in a table format similar to a nursing care plan, so that study participants would be familiar with the clinical nursing interventions format (see Fig. 2). The subjects were asked to read each situation and prioritize the four or five interventions that pertained to each nursing diagnosis from the most preferred to least preferred, taking into account the desired short term goals and the outcome values (goals). A score of 1 was assigned to those interventions that were ranked in accordance with the experts’ rankings. Whereas, a score of 0 was assigned to those interventions that were not ranked in accordance with the experts’ rankings. The scores were
summed across all the interventions for all nursing diagnoses pertaining to each clinical case. The maximum score possible for Case Study Number 1 was 17; for Case Study Number 2, 8; Case Study Number 3, 20. Content validity of the clinical case studies was evaluated by an expert review panel that was different than the panel who developed the instrument. The expert panel consisted of four first-year masters nursing students who were enrolled in the medical surgical clinical program and were experienced in medical surgical nursing. The panel indicated that the clinical case studies were relevant and comprehensive. The panel also agreed that the case studies presented information that was consistent with similar case presentations in the clinical setting. Furthermore, they agreed that the identified nursing diagnoses, clinical interventions, and the respective probability values were appropriate for the clinical case studies. Following data collection, reliability of the nursing clinical case studies instrument was tested and Cronbach alpha coefficients were generated for each case. Reliability of the instrument was assessed by treating each of the three clinical situations as an independent instrument. Each intervention that was identified was considered an item. Case Study Number 1 had 17 items with an alpha value of .92. Case Study Number 2 had 8 items with an alpha value of .78, and Case Study Number 3 had 20 items with an alpha value of .90.
Procedure Third and fourth-year nursing students were invited to take part in the study. Those who volunteered were randomly assigned to either the experimental
62
RESEARCH IN NURSING & HEALTH
Incrmase In Rmst Pmrlod
Outcome: Increased A b l l l t y t o Perform ADL
Increase In Enmrgy
1
Good
5
Fair
25
1 Poor
\
0
V e r y Good
1
Good
5
Falr
25
Poor V e r y Good
0 1
Toach P r o g o s s l v e
Good
5
Muscio R e l a x a t i o n
Fair
25
Glve Back Rub
HS
Poor
0
Vary Good
1
Good
5
Falr
25
Poor
0
FIGURE 1. Example of a decision tree.
or the control group. Written consent was obtained from all participants. Both the experimental and the control group sessions were held on the same day at different locations in order to eliminate any information exchange among subjects in the two groups. The intervention was 4 hours in length and was organized in three blocks of time with breaks between the sessions. During the third session the clinical case studies instrument was administered. Following these sessions, the data collection instruments were administered to the study subjects. Experimental intervention. The intervention in the experimental group included videotaped instruction on decision analysis as well as an interactive session that focused on the application of decision analysis. The videotaped sessions addressed four aspects of decision making and decision analysis including: (a) the importance of decision making in nursing; (b) principles of decision analysis and its limitations; (c) clinical problems and the use of decision analysis; and (d) nursing clinical problems and decision analysis. The interactive session included a step-by-step application of decision analysis to a clinical nursing problem. The teaching program was based on methods outlined by Weinstein, Fineberg, Elstein, Frazier, Neuhauser, Neutra, and McNeil (1980) for teaching clinical decision analysis.
Control intervention. The intervention in the control group consisted of presenting four videotaped health-related ethical case situations including: (a) a chronic problem; (b) the courage of one’s conviction; (c) the old person’s friend; and (d) critical choice. Ethical case studies were selected for the control intervention because ethical cases were nonclinical, yet of concern and interest to nurses. It was thought that providing nonclinical cases for the control intervention might strengthen the effect of the experimental intervention. The subjects in the control group were required to view the videotapes and participate in an unstructured discussion following each videotape. The experimental and control groups spent comparable amounts of time viewing videotaped material and engaging in interactive sessions with the respective group leaders. R ESULTS The difference in the number of responses that were in accordance with the experts between the experimental and control group were compared utilizing the ?-test. Statistically significant differences were observed between the experimental and control group at p < .0001 in all three case
63
EFFECT OF TEACHING DECISION ANALYSIS / SHAMIAN
agreement within the experimental group, whereas the Kappa value of .729 with 30 degrees of freedom and a larger confidence interval of .699-.759 indicates that there was a fair to good level of agreement amongst the control group (Fleis, 1971). There was a high consistency of agreement among subjects’ clinical decision making in the experimental group, a consistency not due to chance, whereas, in the control group, the level of agreement could be due to chance alone. In summary, the above findings suggest that the experimental group demonstrated prioritizations of clinical interventions that were significantly more in accordance with the experts’ process of decision analysis than the control group. Further, the experimental group was more consistent in their clinical decision choices than the control group.
studies. The mean score of the experimental group was significantly higher than that of the control group (Table 2). In all case studies, the experimental group selected clinical decisions made by experts more often than the control group.
lnterrater Agreement Upon initial examination of the data, it appeared that there was an increased level of agreement regarding the choice of clinical interventions among the respondents in the experimental group than in the control, perhaps indicating that a more consistent clinical approach was being identified in the experimental group. Therefore, it was decided that further analysis of the level of interrater agreement would enhance the understanding of the results. Cohen’s weighted kappa statistics were generated to determine the rate of agreement between subjects in the experimental and control groups. The Kappa value of .999 with 36 degrees of freedom s ? d a small confidence interval of .997- 1 .OOO, indicates that there was a very strong level of interrater
DISCUSSION This study provides preliminary evidence that teaching decision analysis to nursing students in
Nursing Diagnosis:
Sleep pttern disturbance related to anxiety
Short Term Coals:
1. 2.
Increasr in rest periods Facilitate an increase in energy level
Time Frame for Goal Attainment: Chitcome Value (Coal):
Within next 24 hours
Increased rest and increased energy to perform ADL
In the s p e provided ( ) , prioritize each intervention from the most preferred intervent,iori ( 1 ) to t.he least prrferreci intervent~ion(4)
.
. .-
~ ~~
!
I Short Tern Goals I Nursing
:
Diagnosis
: : :
Sleep Pattern Disturb.
: :
Possible Intervention
:
Administer Sedative
I
:Increase in I Increase : Increased Ability :Rest,P e r i o d : in Energy I to Perform ADL I I
Yes ( . 5 ) Yes ( . 5 ) No ( . 5 ) No ( . 5 )
: : : :
Yes Yes
(.a) (.a)
I
No No
(.2) (.2)
: :
Yes No Yes No
Teach : Yes ( . 7 ) progressi ve : Yes (.7) muscle relaxation: No ( . 3 ) I No ( . 3 )
I
Yes ( . 8 ) I
I I
(.2) I Yes ( . 1 ) I No ( . 9 ) :
:
I
: I I
;
: I
:
I
: :
Organize : Care (i.e. m e d s , I V/Sl I
:
~-
-4-
Give back rub h.s.
Cutcome Value((;oal)
I--
; 8-
: : I I
Yes i . 3 ) Yes ( . 3 ) No ( . 7 ) No ( . 7 )
:
I
: :
Yes ( . 7 )
(.3) Yes ( . 2 ) No (.8) No
: j ,
: :
(.8) (.2) I (.1) : (.9)
:
No
Yes ( . 7 ) I No ( . 3 ) Yes ( . 1 ) I No
(.9) I
Very Good Fair Poor
Good ( 1 )
Very Good Fair Poor
Cood ( 1 )
Very Good Fair Poor
Good ( 1 )
Very Good Fair Poor
Good ( 1 )
a . -
-t___ppI
FIGURE 2.
~~~
~~
Example of a data collection form.
(.5) (.25) (0) (.5) (.25) (0) (.5) (.25)
(0) (.5) (.25) (0)
I
:
64
RESEARCH IN NURSING 8 HEALTH
Table 2. T-test for Difference between the Experimental and Control Groups Case Study
n
M
SD
Range
1.E. C. 2.E.
37 31 37 31 37 31
15.78 8.35 7.56 5.00 18.20 11.40
1.827 4.401 ,987 1.966 2.451 4.850
11-17 2-15 4-8 1-8 10-20 5-20
C. 3.E. C.
-
df
t
P
38.6
8.78
.0001
42.4
6.60
,0001
42.6
7.04
,0001
Note: E = Experimental Group C = Control (Sham) Group
addition to providing the probabilities for the decision situation enables nursing students to prioritize clinical interventions in accordance with clinical experts. Furthermore, the nursing students in this study readily used normative decision rules, once these rules were made available to them. The findings also suggest that nursing students benefited from the experimental intervention that included both the didactic videotaped teaching section which outlined the principles of probability and decision analysis, as well as the interactive session where those general principles were applied to a clinical case study. Because of the study design, it is impossible to determine whether the individual aspects of the teaching program specifically, the decision analysis principles, or the interactive problem solving, would have resulted in the same outcome as the combined elements. However, the teaching program developed for this study could serve as the foundation for a more in-depth and intensive teaching program. In the teaching arena, nurse educators should explore what type of decision analysis program would have the greater impact on clinical decision making. It is also striking that nursing students in the control group arrived at diverse prioritizations of the clinical interventions in the clinical case studies instrument, while the experimental group arrived at consistent prioritizations of the clinical interventions in the clinical case studies instrument. The lack of consistency in prioritization of clinical decisions among the control group suggests that the decision approaches that were used by the control group did not lead to accurate and consistent clinical decisions. Further research is needed in order to generate decision making approaches that will result in consistent and accurate clinical decisions. The study has a number of limitations due largely to its exploratory nature and the use of a convenience sample. Specifically, the sample of university nursing students is not representative of
the population of nurses and, in addition, the sample of students were self-selected from one institution. Further, the study design did not allow the investigator to study the effectiveness over time of teaching decision analysis. Finally, some difficulties exist concerning the instrumentation used in the study. Importantly, the validity of paper and pencil tests for assessing clinical decision making ability has not been firmly established. Further it was difficult to meet the underlying assumptions of decision analysis for the purposes of instrument development. In particular, the criteria of mutually exclusive alternatives remains the most problematic, given the nature of nursing practice. Additionally, normative decisions have not traditionally been used to prioritize decisions; however, these methods do generate numerical expected values. In this study, these numerical values have been used to indicate a priority ranking of interventions that will best achieve the desired outcomes. Although this approach has not been empirically tested, the application of normative decision methods may be well suited to complex clinical situations in order to determine more accurate, consistent decisions. In summary, a number of steps must be undertaken in order to take this study out of the realm of the exploratory and into that of clinical usefulness. The study needs to be replicated with various nursing students from different settings, such as other universities and community colleges in addition to practicing nurses, in order to determine whether the findings are consistent between these groups. Further studies will be required in order to: (a) generate clinical probabilities for many of the nursing interventions; and (b) further understand the impact of practitioner identified probabilities on the desired outcome. In the teaching arena, nurse educators should explore what type of decision program would result in the greatest impact on clinical decision making.
EFFECT OF TEACHING DECISION ANALYSIS / SHAMIAN
APPENDIX: CASE STUDY NO. 2 Section 1 Please read the following clinical situation. After you have completed reading the situation you will find some of the nursing diagnoses identified and possible interventions offered. The purpose of this exercise is that you examine the possible interventions offered for each diagnosis and on the basis of this clinical situation prioritize each from the most preferred one ( 1 ) to the least preferred.
Clinical Case Study No. 2.0 You are working the day shift on a 32 bed cardiac stepdown unit. Your patient assignment includes Keven Derand, a 45 year old, admitted two days ago to CCU for possible MI. The diagnosis of Myocardial Infarction was ruled out today and he is being transferred to your unit with a diagnosis of “Unstable Angina.” During his stay in CCU, he had several instances of sinus tachycardia, but no arrhythmias have been detected. His cardiac status continues to be monitored with telemetry. He currently has a Heparin lock in place for 1V access and has only required O2 on one occasion since his admission. In reading the nursing history, you determine that Mr. Derand is self-employed and runs a franchise of a convenience store chain. He is married and has two children living away from home: a son, 25 years old, and a daughter, 22 years old. His wife helps out in the store with him, but is physically disabled as a result of a car accident several years ago. He and his wife live in a large apartment above the store. In reading the medical history, you determine that Mr. Derand has never been hospitalized before. His father died at the age of 56 from a massive myocardial infarct. His mother had a CVA last year. at the age of 72 and now resides in a nursing home. He has no siblings. Mr. Derand is a non-smoker, not overweight, but was told by his family physician during a routine checkup last year, that he should limit his sugar intake. On admission he complained of substernal chest pain, was diaphoretic, dyspneic and complained of extreme weakness. Initially, he required Morphine 2 mg IV push, times three doses, before his pain was relieved. His EKC demonstrated peaked T waves and ST segment depression. During the first 48 hours following admission, his bloodwork revealed minimal elevation of his cardiac enzymes. Chest x-ray demonstrated normal heart border and clear lung fields. A fasting blood sugar drawn yesterday morning was reported at 16.8 mmol/l. An endocrinology consult has been requested. When you received Mr. Derand in transfer, his medical orders include: Nitropaste 1” topically QID Ativan 1 mg s/l qhs prn Diltiazem 30 mg QID
65
Nitroglycerin 0.6 mg s/l prn Betaloc 50 mg p.0. BID Bathroom privileges and up in chair X 30 min QID Diet: As tolerated O2 3 1% by mask prn The nurse transferring Mr. Derand describes him as a talkative, nervous man. Since his admission, his wife has been only able to visit once, but calls at least three times a day. His comments indicate that he is anxious to retum home and is concerned about his wife managing on her own. Although he sleeps frequently throughout the day, Mr. Derand has been having difficulty getting to sleep at night. The nurse reports that he has been up in the chair twice today and on the last occasion complained of a “funny” feeling in his chest within five minutes of being up. The nurse describes this incident to you with some skepticism, reporting that Mr. Derand became very panicky, wanting to return to bed and have oxygen and a “heart pill” right away. Within five minutes of providing these measures, Mr. Derand settled. At the time, his cardiac arrhythmia strip showed no evidence of ischemia or anythmia and changes in his vital signs were not significant. When you first visit Mr. Derand, he asks if you have come to take away his heart monitor and assures you that it is not bothersome. He then asks if you are aware of his needs for oxygen and “heart pills.” He tells you that he had a very bad “spell” earlier today and does not think he should get up this evening.
REFERENCES Aspinall, M.J. (1979). Use of a decision tree to improve accuracy of diagnosis. Nursing Research, 28, 182-185. Aspinall, M. J . , and Tanner, C . A. ( I98 I ). Decision making f o r patient care: Applying the nursing process. New York: Appleton-Century Crofts. Cohen, J. (1 977). Statistical power analysis f o r the behavioral sciences. (Rev. Ed.) New York: Academic Press. College of Nurses of Ontario (1985). The standards and levels of nursing practice including the assumptive base. Discussion paper. Corcoran, S.A. (1986). Task complexity and nursing expertise as factors in decision making. Nursing Research. 35, 107-1 12. Davis, B.G. (1972). Clinical expertise as a function of education preparation. Nursing Research, 2 1 , 530-534. Fleis, J. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 375-382. Crier, M . R . (1976). Decision making about patient care. Nursing Research, 2 5 , 105- 110. Crier, M . R . (1984). Information processing in nursing practice. In H.H. Werley & J.J. Fitzpatrick
66
RESEARCH IN NURSING & HEALTH
(Eds.), Annual Review ofNursing Research (pp. 265-287). New York: Springer. Crier, M.R., & Schnitzler, C.P. (1979). Nurses’ propensity to risk. Nursing Research, 28, 186191. Hammond, K.R., Kelly, K.J., Schneider, R.J., & Vancini, M. (1967). Clinical inference in nursing: Revising judgements. Nursing Research, 16, 38-45. Kassirer, J.P., & Gorry, G.A. (1978). Clinical problem solving: A behavior analysis. Annals of Internal Medicine, 89, 245-255. Kassirer, J.P., Moskowitz, A.J., Lau, J., & Pauker, S.G. (1987). Decision analysis: A progress report. Annals of Internal Medicine, 106, 275-291. Krischer, J.P. (1980). An annotated bibliography of decision analytic applications to health care. Operations Research, 28, 97- 113. Ozbolt, J.G., Schultz, 11, S . , Swain, M.A.P., Abraham, I.L., & Farchaus-Stein, M.S. (1984). Developing an expert system for nursing practice. Proceedings of the Eighth Symposium on Compurer Applications in Medical Care, 654-657.
Patel, V. (1980). Clinical reasoning in medicine: A review of cognitive-information processing approach. Unpublished manuscript, Centre for Medical Education, Montreal. Pliskin, J . S . , Shepard, D.S., & Weinstein, M.C. (1980). Utility functions for life years and health status. Operations Research, 28, 206-224. Schwartz. W.B., Gorry, G . A . , Kassirer. J.P., & Essig, A. (1973). Decision analysis and clinical judgement. American Journal of Medicine, 5 5 , 459-472. Tanner, C.A. (1987). Teaching clinical judgement. In J.J. Fitzpatrick & R.L. Taunton (Eds.), Annual Review of Nursing Research (pp. 153- 174). New York: Springer. Verhonick, P.J., Nichols, G.A., Glor, B.A.K., & McCarthy, R.T. (1968). I came, I saw, 1 responded: Nursing observation and action survey. Nursing Research, 1 7 , 38-44. Weinstein, M.C., Fineberg, H.V., Elstein, A.S., Frazier, H . S . , Neuhauser, D., Neutra, R.R., & McNeil, B .J . (1980). Clinical decision analysis. Philadelphia: Saunders.
