Hormonal Replacement Therapy: Analysis of Clinical Strategies Used by Residents ARTHUR S. ELSTEIN, PhD, GERALD B. HOLZMAN, MD, LAURIE J. BELZER, RUTH D. ELLIS, MD The authors investigated strategies employed by resident physicians to decide whether to prescribe hormonal replacement therapy (HRT) for menopausal women, a matter of continuing clinical controversy. Verbal protocols were obtained from 21 residents in three specialties as they responded to 12 brief case descriptions. The cases incorporated three levels of cancer risk and two levels of osteoporosis risk in a 3 x 2 factorial design with two replications in each cell. Substantial variation in willingness to prescribe HRT was observed. By clustering subjects with relatively similar approaches to the problem, three treatment strategies were formulated that accounted for the decisions of 20 subjects. Each strategy is a simplified representation of the conflicting considerations in this clinical dilemma that facilitates rapid decision making. The differences between these representations and formal decision-analytic models help to explain why observed clinical decisions were inconsistent with expected utility maximization. Key words: hormonal replacement therapy; decision making; subjective probability estimates; regret; intuitive clinical reasoning. (Med Decis Making 1992;12:265-273)
In the 1970s the continued
use of replacement eswith linked an increased rate of endotrogen metrial cancer, and clinical opinions about the appropriateness of and indications for hormonal replacement therapy (HRT) for menopausal and postmenopausal women have varied widely since.’-’ Academic opinion has shifted toward recommending estrogen with progestin both as short-term therapy for hot flashes and as long-term therapy to reduce the incidence of osteoporosis.’ Estrogen appears to decrease low-density lipoprotein and increase high-density lipoprotein. This is the explanation given when it is suggested that postmenopausal estrogen may reduce the incidence of myocardial infarction.6 The possibility that sex hormones may lead to an increased incidence of breast cancer has always been a theoretical consideration.’-’ Several very recent papers suggest an increased, albeit small, risk/J.1O while several letters take issue with this suggestion.’ 1-17 The risk for patients receiving HRT of developing breast cancer has not appeared to be of major concern to clinicians.
case for HRT in preventing heart disease does not yet appear to be a significant factor in the considerations of practicing physicians. This may change as more data concerning this risk are reported. Individual clinicians appear to have clear preferences, but there is little consensus among them. Since decisions concerning hormonal replacement for post-
Likewise, the
was
menopausal women necessarily arise frequently, an understanding of the psychological processes involved in making them should be of considerable interest to health-care researchers, physicians, and patients.
Previous research on experienced clinicians’ decision making about HRT for menopausal women showed that endometrial cancer risk figured much more prominently in intuitive treatment decisions than in
decision-analytic representation.18,19Further, these were poorly accounted for even by a decision-analytic model that incorporated each physia
decisions
cian’s estimates of the risks and benefits, cast in the form of subjective probabilities, utilities, and importance
weights.
argued that the general reluctance of experienced physicians to prescribe HRT reflected attitudes prevailing during their years of training and that their practice patterns had not changed despite a different approach advocated by the academic comIt could be
Received October 23, 1991, from the Department of Medical Education, University of Illinois College of Medicine, Chicago, Illinois IASE, LJB, RDE), and the Department of Obstetrics and Gynecology, Medical College of Georgia, Augusta, Georgia (GBH). Revision accepted for publication March 26,1992. Presented at the ninth annual meeting of the Society for Medical Decision Making, Philadelphia, Pennsylvania, October 14, 1987. Supported in part by grants to the senior author from the Medical Cognitive Science Program of the Josiah Macy Jr., Foundation, #B8520004, and from the National Library of Medicine, R01-LM-4583. Address correspondence and reprint requests to Dr. Elstein, Department of Medical Education Im/c 591), University of Illinois at
Chicago,
Box 6998,
Chicago,
munity. Therefore, amine the decision
our
research
was
extended to
ex-
making of third-year residents, a cohort of clinicians currently in training and potentially &dquo;up to date.&dquo; In the first step of this investigation,2° intuitive clinical decisions were systematically compared with treatment decisions reached by two decision-analytic models that incorporated probabil-
IL 60680.
265
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
266
ities and utilities provided by the subjects themselves. That study asked whether changes in the decisionanalytic model could bring about a better fit between the model and observed clinical decisions. The recommendations of the two formal models were in agreement over 80% of the time, but concurred with the observed decisions of the residents in only about 15% of instances. It was reasonable to conclude that a decision-analytic model did not describe the decisions made in this clinical situation. The next step of our study is reported here. It builds on previous work by using analysis of thinking aloud to extract the rules guiding intuitive clinical decisions. These rules are not adequately mapped by decision analysis, as shown by our previous research. This study identifies some of the strategies used in everyday clinical decision making for HRT for menopausal and postmenopausal women. It presents new data obtained from qualitative protocol analysis that complements results obtained by our quantitative analysis2° and discusses some of the reasons for the discrepancies we have found between observed clinical decisions and the recommendations of decision-analytic models using probabilities and utilities provided by the same
physicians.
of physical mobility, amount of exercise, and menstrual history, including postmenopausal bleeding. There were six cells in the design, each containing two case vignettes for each combination of the three cancer-risk and two fracture-risk levels. Both patients in the high-cancer-risk, high-fracture-risk cell of the design had had hysterectomies for previous Stage I or II endometrial cancers; every other patient had an intact uterus. Vasomotor symptom severity was excluded from these cases, since previous research had shown that physicians did not think that it should be given much weight in this decision, as compared with the two factors identified. Breast cancer risk was also excluded, since it was not felt to be a significant risk when the study was planned and implemented. SUBJECTS
Twenty-eight third-year residents in training programs at four Chicago-area teaching hospitals were approached to participate in this study. Twenty-two agreed to participate, and 21 completed the entire task. The final sample included seven residents in each of three specialties: internal medicine, family practice, and obstetrics and gynecology. Their ages ranged from of 29.4 years. There were in the group. A bookstore voucher (up to $75) valid for a textbook of the resident’s choice was his or her remuneration for participation. 25 to 38 years, with ten women and 11
Methods
a mean men
MATERIALS PROCEDURES
To examine the influences of two risk factors, the risk of endometrial cancer and the risk of fractures because of osteoporosis, on prescribing HRT, we designed 12 case vignettes. They were brief summaries of relevant medical facts, not abstracts of patients’ charts. A sample case is presented in appendix A. Each case
described a postmenopausal woman appearing for a scheduled checkup with mild menopausal symptoms that did not interfere with normal activities. None was using or had ever used any HRT regimen. The instructions made clear that none of the patients had a preference either for or against HRT; all would comply with the physician’s decision. The intent of this instruction was to place the responsibility for the decisions on the clinician’s shoulders. In a few instances, the clinicians indicated that data about patient preferences would have been helpful, but as the data show, there were few toss-ups and most were unrelated to this issue. The cases systematically varied the levels of the two risk factors. Three levels of cancer risk (standard, moderate, and high) and two levels of risk of fracture because of osteoporosis (standard and high) were used. Risk levels for each case were not explicitly identified. They were implicit in relevant features of each patient’s medical record, including body size, gravidity, parity, a history of diabetes or hypertension, degree
Two practice cases were presented at the beginning of the experimental session to familiarize the subjects with the procedures. Each session had two parts: 1) reading, thinking aloud, and decision making on 12 case vignettes; and 2) a structured interview to assess each participant’s probabilities and utilities. A brief questionnaire was also completed at this session. Several weeks after this initial research session, the subjects were debriefed individually. Each subject read each case aloud, highlighted the important features of the vignette text with a marking pen, and indicated how likely he or she was &dquo;to prescribe a steroid&dquo; by marking a 10-cm line anchored at &dquo;Virtually certain I WOULD NOT prescribe&dquo; and &dquo;Virtually certain I WOULD prescribe.&dquo; The word &dquo;estrogen&dquo; was not used at this point in the experiment to help ensure that the subjects understood they were to decide about employing whatever regimen they would ordinarily use. The subjects were asked to identify and discuss the important features of the case and to comment on its difficulty. If there were frequent pauses, the subject was reminded to think aloud. The responses were classified into three decision categories : responses below 0.40 were classified as &dquo;Do not prescribe&dquo; (don’t treat), those between 0.40 and 0.60 inclusive as &dquo;Undecided&dquo; (toss-up), and those above
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
267
&dquo;Prescribe&dquo; (treat). This threshold setting was but shifting the cutting scores would make little difference in the overall pattern of the data because the clinicians were rarely undecided about these cases. Changing the toss-up zone to say, 0.30 and 0.70 or even 0.25 and 0.75 does not markedly add to the number of toss-ups. Knowledge of and beliefs about the efficacies of estrogen in reducing fracture risk and of combining progestin with estrogen to reduce cancer risk were assessed in the structured interview by asking the resident about his or her usual regimen and by comparing estimated probabilities of fracture and cancer conditional on various treatment regimens. For example, the difference between the estimated rates of fractures with and without HRT was taken a measure of its perceived efficacy in reducing this risk. The probability estimates were elicited by referring to hypothetical cohorts of 1,000 patients with a set of clinical features representing a particular level of cancer or fracture risk. The same features were embedded in the case vignettes but the probabilities were not explicitly elicited. A questionnaire was used to obtain information about each resident’s usual dosage when prescribing estrogen, possible use of progestin or other hormones in a combination regimen, and some biographical in0.60
as
Table 1 .
Probabilities of Prescribing Hormonal Replacement Therapy, by Risk Level, of 21 Physicians Responding to 12 Case Vignettes
arbitrary,
formation, including any personal
or professional experience with either endometrial cancer or fracture due to osteoporosis. Protocol analysis was used to determine the problem representations underlying the HRT decisions. The thinking-aloud discussions, when typed, ranged from one sentence to three double-spaced typevoitten pages
per case, with an average length of approximately 20 lines per case. (See examples in appendix B.) After each case discussion was abstracted, a decision rule was formulated for the case incorporating the information from the subject’s verbalizations that led to the treatment decision. A rule was written for each case (12) for each subject (21), a total of 252. These rules, in the form of &dquo;If ...,then ...&dquo; statements, indicate that a certain action should be taken provided certain conditions are met. To check the reliability of this procedure, a 10% random sample (25 case-subject pairs) was reread by the coder (RDE). New rules were written and compared with those composed about three months earlier. Agreement was obtained in 24 of 25 comparisons, which was considered to establish satisfactory consistency in protocol reading and rule construction. To identify general strategies or approaches to the problem, we wrote out each subject’s rules on a single sheet of paper, and formulated some general statements characterizing that subject’s approach. Rules common to groups of subjects were noted. By an iterative approach, clusters of subjects with similar approaches were identified. Ultimately, the subjects were assigned to one of three clusters representing strate-
gies that accounted reasonably well for the decision making of 20 of the 21 subjects. The strategies were identified
reading
empirically and were
and
not
reviewing the coded
formulated before
rules.
Resoks Table 1 summarizes data on the probability of prescribing HRT for each risk combination. The entries are in ascending order of cancer risk and descending
order of fracture risk. The residents’
comments about the cases show that they generally interpreted the risk levels in the cases as intended and that the observed decisions were not a result of systematic misunderstanding of the cases.
The standard-cancer-risk-high-fracture-risk cases the only pair of cases in which the majority of subjects would prescribe HRT, but this was not a unanimous decision, as the range shows. The variability of decision making for each case is demonstrated by the range. Regardless of the risk level for either cancer or fracture, there was at least one physician who would definitely prescribe HRT and one who would not. As stated earlier, both patients classified as having a high risk for cancer and a high risk for fracture had had hysterectomies for early-stage endometrial cancers. Table 1 shows that the probability of prescribing HRT was the same for these two patients as it was for the pair who were at high risk for cancer and at standard risk for fracture. This is initial evidence suggesting that most subjects did not think that patients who had had hysterectomies for endometrial cancer could be assigned to the category of low cancer risk. were
CASE-RELATED DECISION RULES
A set of rules that describe the thought processes of all subjects was developed by summarizing the individual rules for each pair of cases. These rules are presented in table 2. Treatment was most often prescribed for cases with high fracture risk and standard cancer risk (category A). Even in this situation, no treatment was selected almost a third of the time. In general, as cancer risk increased, the preference to pre-
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
268
T1ll0 2 o
*DVT
Making for Hormonal Replacement Therapy, Aggregated by Levels of Cancer and Fracture Risks, of 21 Physicians Responding to 12 Case Vignettes
Rules for Decision
deep venous thrombosis. tcontraindications cancer risk factors, diabetes, hypertension, obesity. *Indications for treatment symptoms, osteoporosis risk factors. =
=
=
scribe HRT declined.
declaring with close
a
toss-up
follow-up,
Cognitive conflict,
or
as
by recommending
was
shown
by
treatment
strongest for the moderate-
cancer-risk-high-fracture-risk cases (category C). Withinsubject inconsistency was also highest for this pair of cases.
Categories E and F show that the management of patients at high risk for cancer was not much affected by a previous medical history of hysterectomy for endometrial cancer. We had thought that some subjects would consider the risk of future endometrial cancer to have been eliminated in these cases (category E),
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
269
thus justifving treatment, while others might consider these patients to be at risk for recurrence or at a higher general risk for future estrogen-dependent cancer of another organ. The similarity of the decisions in categories E and F shows that patients with previous histories of endometrial cancer were almost uniformly viewed as being at continued high cancer risk. All three groups of residents treated these patients as essentially high-risk for recurrence or another malignancy. In their spontaneous thinking aloud, the residents rarely used a numerical probability estimate to talk about the risks in a case. Global expressions of perceived risk (e.g., &dquo;the risk of cancer for this woman is vel-v high&dquo;) were the general rule. They did not naturallv encode their estimates on a probability scale, although they could do so if asked, as in the structured interview.
attention to intra-individual consistency. To what extent did the individual physicians display consistent philosophies or strategies of treatment that cut across cases? ’Three groups characterized by their treatment strategies are summarized in table 3. The most common global strategy, used by eight physicians, resulted in the greatest proportion of HRT prescriptions. It involved scanning the data for indicators of cancer risk and treating only if they were absent. Fracture-risk indicators were secondaiy. A second strategy, used by seven subjects, can be summarized as &dquo;prescribe only if fracture risk is high and no other risk is apparent.&dquo; Since prescription of HRT was limited to cases at high risk for fracture, some weight was evidently given to cancer risk. This strategy led to more indecision than the other two strategies, as represented in the relatively greater number of toss-ups. The third strategy, &dquo;do not prescribe,&dquo; accounted for the decisions of five physicians who repeatedly stated they were reluctant to prescribe HR’I’. Only one prescribed HRT and then for only one case.
TREATMENT STRATEGIES
Having explored changes
in treatment rules
function of the features of the cases,
Table 3 o Higher-order Strategies Response
*FP
=
to 12 Case
family practice residents;
we
Derived from 21
turned
as a
our
Physicians’
Decisions Whether to Prescribe Hormonal Replacement
Vignettes
OB - obstetrics and
gynecology residents;
IM
=
internal medicine residents.
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
Therapy
in
270
Table 4 o Comparison Therapy
’Strategy
in
of Strategies 1 Response to 12
1: if cancer is not
apparent; otherwise, toss-up
a
or
and 2* Derived from 21 Case Vignettes
threat, prescribe; otherwise toss-up
or
Physicians’ Decisions Whether to Prescribe Hormonal Replacement
do not
prescribe; strategy
2: Prescribe
only
if fracture risk is
high
and
no
other risk is
do not prescribe.
The decision making of one subject was so idiosyncratic that it could not be characterized by any of the three global strategies. A review of that physician’s transcript suggests that unfamiliarity with the HRT problem in general resulted in inteipretation of the vignette features. This subject was inconsistent in three pairs of cases, a number equalled by five other subjects and exceeded by one (table 3). However, the other subjects’ decisions could be summarized into strategies because their verbal explanations clarified what they were tiying to do. The unclassifiable subject’s explanations were not as consistent or
unreliable
helpful.
egy egy
2 resembled those of
subjects
13 and 11 in strat-
1.
To validate the classification of subjects into strategies and to determine whether the strategies were related to differences in prescribing, we reviewed the treatment decisions resulting from applying to each
pair the rules corresponding to the different strategies (table 4). This analysis showed that strategies 1 and 2 proceeded along very similar lines for three case pairs (6 and 12 ; 1 and 7; 2 and 8). Both strategies favored not prescribing HRT for high-cancer-risk cases and prescribing it for standard-cancer-risk-high-fracturerisk cases. They differed about what to do with modcase
erate-cancer-risk-high-fracture-risk CASE-RELATED VARIATION BETWEEN STRATEGIES
toss-up. With As table 3 shows, individual clinicians were frequently inconsistent in their judgments within a pair
of cases. Only six subjects were completely consistent, and 13 had two or more inconsistencies. Some inconsistencies were due to misinteupretation of a clinical feature. For example, a feature of the patient’s histoiy intended as a cue for high fracture risk was occasionally interpreted as suggesting standard risk. In other cases, some elements were introduced into the story that were not even in the case vignette. And, on occasion, minor differences in the cases that we thought were immaterial became the basis for different decisions. However, much of the variation was due to cognitive conflicts embedded in the cases, as suggested by table 2. Although the strategies sketched here are not perfect predictors of individual clinicians’ behaviors, the clinicians did seem to be striving to implement them, even if they did not always succeed. Because of the inconsistencies, there were two potential difficulties with the classification of three strategies: 1) prescribing decisions were not uniform within each strategy and 2) the decisions of some subjects might be classified equally well into two different strategies. For example, the decisions of subject 21 in strat-
(4 and 10).
cases
Strategy 1 favored prescribing HRT and following closely; strategy 2 favored not treating or calling the case a a
moderate
cancer
fracture risk (cases 3 and 11),
risk and
most
a
strategy
standard
1
subjects
prescribe HRT and follow closely, while strategy 2 subjects strongly favored not treating. When
preferred
to
both cancer and fracture risks were standard (cases 5 and 9), the differences were most dramatic: all strategy 1 subjects used the rule, &dquo;prescribe if there is no contraindication&dquo; ;all strategy 2 subjects employed a &dquo;do not treat if there is no indication for treatment&dquo; rule. Since patients who are at standard risk for both problems make up the vast majority of the population, deep underlying differences in approach are clearly seen in these strategies. These differences help to explain why hormonal replacement therapy has been such a controversial matter. REGIMEN AND TREATMENT CHOICE
The
policy concerning progestin use expressed by compared with his or her subjective estimate of its effectiveness in reducing probability cancer risk. Eleven of 15 progestin prescribers knew each resident was
use decreases the risk of endometrial four did not. Of the six non-prescribers, four believed progestin use would reduce the risk of
that
progestin
cancer, but
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
271
endometrial cancer but would not use a progestin anyway. Thus, of a group of 21 residents, knowledge base and practice were consistent for only 13. Four did not know why progestin should be used but would prescribe it, and four knew the arguments for using it but did not. The chi-square statistic indicated that the difference in prescribing habits between prescribers and non-prescribers was not statistically significant.
Discussion The data in this study demonstrate an overall patof preferring not to prescribe HRT that is remarkably similar to the collective approach of a more senior group of clinicians assessed earlier?8 Although the views of the academic medical community have changed substantially since 1975,3these results suggest that a reluctance to prescribe HRT persists and that it is related neither to ignorance of the benefits of estrogen in reducing the risk of osteoporosis nor to a lack of awareness of the possibility of combining progestin with estrogen to reduce the risk of endometrial cancer. The physicians’ efforts to balance identified risks and benefits were summarized in three general strategies that characterized 20 of the 21 subjects. The rules formulated from analysis of their verbalizations and the strategies derived from clustering and grouping these rules converge on some central features of intuitive clinical decision making. These features indicate how everyday clinical reasoning differs from a formal, quantitative decision-analytic model and why the conclusions of these two modes of reasoning may differ. First, in everyday clinical reasoning, levels of risk tend to be treated categorically rather than on a continuous probability scale. The formal model requires that probabilities be specified for a detailed set of conditions, events, diseases, etc. Second, a clinical decision rule is naturally and easily encoded as a production rule stating that a certain action should be taken provided certain conditions are present. The task for the decision maker then becomes determining whether those enabling conditions are present. Third, intuitive clinical decision making is rarely undecidedthere were far fewer toss-ups in the observed decisions than in the recommendations of the decisionanalytic models we have used to study this clinical dilemma.l9~2o Fourth, based on our reading of the subjects’ justifications for their decisions, issues of anticipated regret and responsibility for bad outcomes loomed much larger in the observed decisions than they did in the quantitative models. Aggregating across cases and strategies, the residents in this study, like the experienced physicians studied previously, preferred not to prescribe hormonal replacement. This reluctance appears to have been related to the weight they assigned to the perceived risk of cancer and to tern
their wish not to be implicated in increasing that risk. Our data do not permit us to judge whether the physicians were irrationally overweighting a small risk or whether the formal model is insufficiently sensitive to it. Fifth, intuitive decision making generally processes risk factors sequentially rather than simultaneously. A case is scanned first for evidence concerning one risk and, if that does not yield a decision, the second is evaluated. Decisions may vary depending on which risk is considered first. The HRT decision is becoming more complex. The effects of postmenopausal estrogen on the breast and the cardiovascular system are still controversial. Their impact on clinical decision making should be evaluated once they are better understood. METHODOLOGIC CONSIDERATIONS
repeated-measures ANOVA or regression analysis might have been used to &dquo;capture&dquo; the policies of individual judges.21.22 Although there were enough cases in the study to derive reasonably stable estimates of regression weights, we did not use this approach for two reasons. First, we had already done this type of analysis for HRT decisions.&dquo; Moreover, as noted previously, many subjects have a standard approach to HRT decisions that varies little with changes in the independent variables. When subjects have a standard approach to a problem, their algorithms are more likely to be uncovered by verbal protocol-analysis techniques. The results of this study suggest that there are at least three standard algorithms, not just one. While two of these might be represented by different weights given to cancer-risk and fracture-risk terms in a regression equation, the &dquo;do not treat&dquo; strategy could not be adequately characterized this way.23 Protocol analysis showed that an essential part of the processing of clinical information concerned the A
sequence in which these cases were scanned for relevant risk factors. The problem was too complex for simultaneous processing of information. Regression equations, however, assume that a simultaneous scan-
ning procedure is used. A regression equation might be a good decision aid for this clinical problem, but it does not satisfactorily account for how the clinicians actually processed the information. Using case vignettes to study clinical decision making raises the question of external validity: clinicians may respond quite differently to actual cases. Although they allowed us to systematically vary two relevant factors in a clinically realistic format, the case vignettes did not include every factor that might be considered by physicians. On the other hand, the subjects indicated that the cases were engaging and challenging. Although the vignettes were brief, the subjects were rarely undecided about what to do and rarely postponed a decision on the grounds that additional information was needed.
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
272
Even with very brief case vignettes that are much ambiguous and complex than a complete medical history of a real patient, there is substantial variability in the features singled out by physicians as the enabling conditions of a production rule that concludes with a &dquo;treat&dquo; or &dquo;do not treat&dquo; action. Even more variability may be anticipated in actual practice, since
Drs. Emmett Lamb, Fred Wolf, and Peter Politser provided thoughtful and constructive criticism that helped clarify the presentation. Paul Feltovich and Margaret Holmes gave useful advice regarding protocol analysis. The authors are grateful to Robin M. Hogarth for very useful suggestions relating the results to research on causal judgments. They thank James Dod for his assistance with data tabulation.
real cases are even more complex. There was no significant burden on the physicians’ capacity to recall facts since the cases were relatively brief and the respondents made their decisions while the texts of the cases were before them. These features imply that differences in decisions cannot be reasonably attributed to differences in recall. The observed differences should, therefore, be attributed either to differences in the physicians’ knowledge bases or to differences in their structuring of the problems and in the weighing of risk and benefits. The evidence presented shows that differences in knowledge about the outcomes of different regimens cannot account for the reluctance to prescribe HRT. The sample size in this study was small compared with the numbers of clinical encounters commonly analyzed in retrospective health services research. Since
References
less
protocol analysis
is
inherently labor-intensive,
it
was
necessary to confine our efforts to a small set of cases and a small sample of physicians. Any study with only 21 subjects runs a substantial risk of lacking enough statistical power to detect differences between groups (type II error). This study was not well suited to detecting differences between groups of residents in their readiness to prescribe HRT. Protocol analysis is also not the method of choice for identifying factors affecting the decision process that are so implicit as to be unverbalized or that are outside the subject’s awareness, such as organizational variables. Retrospective health services research, including review of the prescribing patterns of physicians, is better suited to documenting the effects of social and organizational variables upon clinical decisions. 21 On the other hand, studies using chart review cannot control for differences in the patient mixes seen by individual groups, and observed differences may be due to these confounding factors. Observational and retrospective studies rarely explain the mechanisms mediating the effects documented 25 Approaching a problem from several points of view and using a combination of methods is one way to overcome the weaknesses of any single research approach. 21 One final comment: we assumed that the probability estimates obtained in the structured interviews were reasonably accurate representations of the beliefs used in responding to the case vignettes, although the interviews elicited subjective probabilities in the context of groups of patients. Subsequent research27,28 has shown that probability estimates may be affected by the elicitation context. This methodologic weakness should be corrected in future research.
CB, Maxson WS. Current status of estrogen therapy for the menopause. Fertil Steril. 1982;37:5-25. 2. NIH Consensus Conference. Osteoporosis. JAMA. 1984;252:7991. Hammond
802.
MacDonald P. Estrogen
plus progestin in postmenopausal women—Act II. N Engl J Med. 1986;315:959-61. 4. Weinstein MC, Schiff I. Cost-effectiveness of hormone replacement therapy in the menopause. Obstet Gynecol Survey. 3.
1983;38:445-55. Hammond CB, Nachtigall LE. Is estrogen replacement therapy necessary? J Reprod Med. 1985;30:797-801. 6. Stampfer MJ, Colditz GA, Willett WC, et al. Postmenopausal estrogen therapy and cardiovascular disease: ten-year follow-up from the Nurses’ Health Study. N Engl J Med. 1991;325:756-62. 7. Lufkin EG, Carpenter PC, Ory SJ, Malkasian GD, Edmonson JH. Estrogen replacement therapy: current recommendations. Mayo Clin Proc. 1988;63:453-60. 8. Mishell DR, Jr. Menopause. In: Droegemueller W, Herbst AL, Mishell DR, Jr, Stenchever MA, eds. Comprehensive gynecology. 5.
St. Louis: C. V.
Mosby,
1987.
L, Adami HO, Persson I, Hoover R, Schairer C. The risk of breast cancer after estrogen and estrogen-progestin replacement. N Engl J Med. 1989;321:293-7. 10. Hulka BS. Hormone-replacement therapy and the risk of breast cancer. Ca. 1990;40:289-96. 11. Stevenson JC, Whitehead MI. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:201-2. 12. Jacobs SL, Luciano AA, Peterson, MGE, Kegeles SS. Breast cancer and estrogen replacement (letter). N Engl Med J. 1990;322:202. 13. Cutler WB, Genovese E. Breast cancer and estrogen replacement 9.
Bergkvist
(letter). N Engl J Med. 1990;322:202-3. Epstein SE. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:203. 15. Mauvais-Jarvis P, Kuttenn F. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:203. 16. Simon JA. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:203-4. 17. Bergkvist L, Adami HO, Persson I, Schairer C, Hoover RN. Breast cancer and estrogen replacement (letter). N Engl J Med.
14.
1990;322:204. AS, Holzman GB, Ravitch MM, et al. Comparison of physicians’ decisions regarding estrogen replacement therapy for menopausal women and decisions derived from a decision an-
18. Elstein
model. Am J Med. 1986;80:246-58. Holzman GB, Ravitch MM, Metheny WP, Rothert ML, Holmes MM, Hoppe RB. Physicians’ judgments about estrogen replacement therapy for menopausal women. Obstet Gynecol. 1984;63:303-11. 20. Elstein AS, Dod JM, Holzman GB. Estrogen replacement decisions of third-vear residents: clinical intuition and decision analysis. In: Evans DA, Patel VL, eds. Cognitive science in medicine. Cambridge, MA: MIT Press, 1989. 21. Hammond KR, McClelland GH, Mumpower J. Human judgment and decision making: theories, methods, and procedures. New
alytic
19.
York, Praeger, 1980. 22.
Wigton RS. Use of linear models to analyze physicians’ decisions. Med Decis Making. 1988;8:241-52.
23. Holmes MM, Rovner
longo
DR, Rothert ML, Schmitt N, Given CW, Ia-
NS. Methods of
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
analyzing physician practice patterns
in
273
24.
25. 26. 27.
hypertension. Med Care. 1989;27:59-68. Epstein AM, McNeil BJ. Physician characteristics and organizational factors influencing use of ambulatory tests. Med Decis Making. 1985 ;5:401-15. Schwartz RK, Soumerai SB, Avorn J. Physician motivations for nonscientific drug prescribing. Soc Sci Med. 1989;28:577-82. Elstein AS, Shulman LS, Sprafka SA. Medical problem solving: a ten-year retrospective. Eval Health Prof. 1990;13:5-36. Ellis RD, Alderman BW, Elstein AS. Subjective probabilities and estrogen replacement therapy (abstr). Med Decis Making. 1988;8:336.
28.
Redelmeier DA, Tversky A. Discrepancy between medical decisions for individuals and for groups. N Engl J Med.1990;322:11624.
0
APPENDIX B For illustrative purposes, we present the complete texts of the responses of two subjects to a single case. The case is standard risk for cancer and standard risk for fracture. One subject is clearly willing to prescribe, the other is not.
Subject
2
Again this lady, the symptoms of postmenopausal, you know, hot flashes and flushes occurring for two or three years, five years. It’s very unusual of them to occur for longer than that. She’s very active. Only need to tell her to take a lot of dairy products, calcium, calcium replacement [unintelligible phrase] twice a day. You give her estrogen and what happens is that she’s spotting. Any woman with postmenopausal spotting you have to suspect cancer, and by giving her the estrogen, if she develops any vaginal spotting then she has to go through the whole thing, D&C, that kind of junk. The symptoms don’t bother her.
Marked
APPENDIX A
of
Rule: If estrogen will
Alison Zimmer is 50 years old, 5’4&dquo; tall, 135 pounds, married, gravida 3, para 2, Ab 1. She is the sales manager of a discount department store and plays racketball three times a week for recreation. She has experienced hot flashes and flushes 2-4 times per day for the last 14 months, but says they are a minor annoyance. Her last menstrual period was 12 months ago, and she has had no vaginal bleeding since then. Mark an &dquo;X&dquo; along the prescribe estrogen.
probability
line to indicate how
likely
you
are
to
Subject
prescribing: cause
0.12.
spotting,
do not
prescribe.
111
No postmenopausal bleeding’s important. LMP a year and half ago, they do not disturb her. She has occasional hot flashes. I’d probably give this lady combination therapy, mostly because of her risk for osteoporosis. I’m not sure that her hot flashes are probably that important, but she really has no contraindications and with close follow-up hopefully we’ll keep these ladies from breaking their hips. I think this lady should get it. a
Marked
probability
Rule: If there is
no
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
of
prescribing:
1.0.
contraindication, prescribe.
In the 1970s the continued
use of replacement eswith linked an increased rate of endotrogen metrial cancer, and clinical opinions about the appropriateness of and indications for hormonal replacement therapy (HRT) for menopausal and postmenopausal women have varied widely since.’-’ Academic opinion has shifted toward recommending estrogen with progestin both as short-term therapy for hot flashes and as long-term therapy to reduce the incidence of osteoporosis.’ Estrogen appears to decrease low-density lipoprotein and increase high-density lipoprotein. This is the explanation given when it is suggested that postmenopausal estrogen may reduce the incidence of myocardial infarction.6 The possibility that sex hormones may lead to an increased incidence of breast cancer has always been a theoretical consideration.’-’ Several very recent papers suggest an increased, albeit small, risk/J.1O while several letters take issue with this suggestion.’ 1-17 The risk for patients receiving HRT of developing breast cancer has not appeared to be of major concern to clinicians.
case for HRT in preventing heart disease does not yet appear to be a significant factor in the considerations of practicing physicians. This may change as more data concerning this risk are reported. Individual clinicians appear to have clear preferences, but there is little consensus among them. Since decisions concerning hormonal replacement for post-
Likewise, the
was
menopausal women necessarily arise frequently, an understanding of the psychological processes involved in making them should be of considerable interest to health-care researchers, physicians, and patients.
Previous research on experienced clinicians’ decision making about HRT for menopausal women showed that endometrial cancer risk figured much more prominently in intuitive treatment decisions than in
decision-analytic representation.18,19Further, these were poorly accounted for even by a decision-analytic model that incorporated each physia
decisions
cian’s estimates of the risks and benefits, cast in the form of subjective probabilities, utilities, and importance
weights.
argued that the general reluctance of experienced physicians to prescribe HRT reflected attitudes prevailing during their years of training and that their practice patterns had not changed despite a different approach advocated by the academic comIt could be
Received October 23, 1991, from the Department of Medical Education, University of Illinois College of Medicine, Chicago, Illinois IASE, LJB, RDE), and the Department of Obstetrics and Gynecology, Medical College of Georgia, Augusta, Georgia (GBH). Revision accepted for publication March 26,1992. Presented at the ninth annual meeting of the Society for Medical Decision Making, Philadelphia, Pennsylvania, October 14, 1987. Supported in part by grants to the senior author from the Medical Cognitive Science Program of the Josiah Macy Jr., Foundation, #B8520004, and from the National Library of Medicine, R01-LM-4583. Address correspondence and reprint requests to Dr. Elstein, Department of Medical Education Im/c 591), University of Illinois at
Chicago,
Box 6998,
Chicago,
munity. Therefore, amine the decision
our
research
was
extended to
ex-
making of third-year residents, a cohort of clinicians currently in training and potentially &dquo;up to date.&dquo; In the first step of this investigation,2° intuitive clinical decisions were systematically compared with treatment decisions reached by two decision-analytic models that incorporated probabil-
IL 60680.
265
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
266
ities and utilities provided by the subjects themselves. That study asked whether changes in the decisionanalytic model could bring about a better fit between the model and observed clinical decisions. The recommendations of the two formal models were in agreement over 80% of the time, but concurred with the observed decisions of the residents in only about 15% of instances. It was reasonable to conclude that a decision-analytic model did not describe the decisions made in this clinical situation. The next step of our study is reported here. It builds on previous work by using analysis of thinking aloud to extract the rules guiding intuitive clinical decisions. These rules are not adequately mapped by decision analysis, as shown by our previous research. This study identifies some of the strategies used in everyday clinical decision making for HRT for menopausal and postmenopausal women. It presents new data obtained from qualitative protocol analysis that complements results obtained by our quantitative analysis2° and discusses some of the reasons for the discrepancies we have found between observed clinical decisions and the recommendations of decision-analytic models using probabilities and utilities provided by the same
physicians.
of physical mobility, amount of exercise, and menstrual history, including postmenopausal bleeding. There were six cells in the design, each containing two case vignettes for each combination of the three cancer-risk and two fracture-risk levels. Both patients in the high-cancer-risk, high-fracture-risk cell of the design had had hysterectomies for previous Stage I or II endometrial cancers; every other patient had an intact uterus. Vasomotor symptom severity was excluded from these cases, since previous research had shown that physicians did not think that it should be given much weight in this decision, as compared with the two factors identified. Breast cancer risk was also excluded, since it was not felt to be a significant risk when the study was planned and implemented. SUBJECTS
Twenty-eight third-year residents in training programs at four Chicago-area teaching hospitals were approached to participate in this study. Twenty-two agreed to participate, and 21 completed the entire task. The final sample included seven residents in each of three specialties: internal medicine, family practice, and obstetrics and gynecology. Their ages ranged from of 29.4 years. There were in the group. A bookstore voucher (up to $75) valid for a textbook of the resident’s choice was his or her remuneration for participation. 25 to 38 years, with ten women and 11
Methods
a mean men
MATERIALS PROCEDURES
To examine the influences of two risk factors, the risk of endometrial cancer and the risk of fractures because of osteoporosis, on prescribing HRT, we designed 12 case vignettes. They were brief summaries of relevant medical facts, not abstracts of patients’ charts. A sample case is presented in appendix A. Each case
described a postmenopausal woman appearing for a scheduled checkup with mild menopausal symptoms that did not interfere with normal activities. None was using or had ever used any HRT regimen. The instructions made clear that none of the patients had a preference either for or against HRT; all would comply with the physician’s decision. The intent of this instruction was to place the responsibility for the decisions on the clinician’s shoulders. In a few instances, the clinicians indicated that data about patient preferences would have been helpful, but as the data show, there were few toss-ups and most were unrelated to this issue. The cases systematically varied the levels of the two risk factors. Three levels of cancer risk (standard, moderate, and high) and two levels of risk of fracture because of osteoporosis (standard and high) were used. Risk levels for each case were not explicitly identified. They were implicit in relevant features of each patient’s medical record, including body size, gravidity, parity, a history of diabetes or hypertension, degree
Two practice cases were presented at the beginning of the experimental session to familiarize the subjects with the procedures. Each session had two parts: 1) reading, thinking aloud, and decision making on 12 case vignettes; and 2) a structured interview to assess each participant’s probabilities and utilities. A brief questionnaire was also completed at this session. Several weeks after this initial research session, the subjects were debriefed individually. Each subject read each case aloud, highlighted the important features of the vignette text with a marking pen, and indicated how likely he or she was &dquo;to prescribe a steroid&dquo; by marking a 10-cm line anchored at &dquo;Virtually certain I WOULD NOT prescribe&dquo; and &dquo;Virtually certain I WOULD prescribe.&dquo; The word &dquo;estrogen&dquo; was not used at this point in the experiment to help ensure that the subjects understood they were to decide about employing whatever regimen they would ordinarily use. The subjects were asked to identify and discuss the important features of the case and to comment on its difficulty. If there were frequent pauses, the subject was reminded to think aloud. The responses were classified into three decision categories : responses below 0.40 were classified as &dquo;Do not prescribe&dquo; (don’t treat), those between 0.40 and 0.60 inclusive as &dquo;Undecided&dquo; (toss-up), and those above
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
267
&dquo;Prescribe&dquo; (treat). This threshold setting was but shifting the cutting scores would make little difference in the overall pattern of the data because the clinicians were rarely undecided about these cases. Changing the toss-up zone to say, 0.30 and 0.70 or even 0.25 and 0.75 does not markedly add to the number of toss-ups. Knowledge of and beliefs about the efficacies of estrogen in reducing fracture risk and of combining progestin with estrogen to reduce cancer risk were assessed in the structured interview by asking the resident about his or her usual regimen and by comparing estimated probabilities of fracture and cancer conditional on various treatment regimens. For example, the difference between the estimated rates of fractures with and without HRT was taken a measure of its perceived efficacy in reducing this risk. The probability estimates were elicited by referring to hypothetical cohorts of 1,000 patients with a set of clinical features representing a particular level of cancer or fracture risk. The same features were embedded in the case vignettes but the probabilities were not explicitly elicited. A questionnaire was used to obtain information about each resident’s usual dosage when prescribing estrogen, possible use of progestin or other hormones in a combination regimen, and some biographical in0.60
as
Table 1 .
Probabilities of Prescribing Hormonal Replacement Therapy, by Risk Level, of 21 Physicians Responding to 12 Case Vignettes
arbitrary,
formation, including any personal
or professional experience with either endometrial cancer or fracture due to osteoporosis. Protocol analysis was used to determine the problem representations underlying the HRT decisions. The thinking-aloud discussions, when typed, ranged from one sentence to three double-spaced typevoitten pages
per case, with an average length of approximately 20 lines per case. (See examples in appendix B.) After each case discussion was abstracted, a decision rule was formulated for the case incorporating the information from the subject’s verbalizations that led to the treatment decision. A rule was written for each case (12) for each subject (21), a total of 252. These rules, in the form of &dquo;If ...,then ...&dquo; statements, indicate that a certain action should be taken provided certain conditions are met. To check the reliability of this procedure, a 10% random sample (25 case-subject pairs) was reread by the coder (RDE). New rules were written and compared with those composed about three months earlier. Agreement was obtained in 24 of 25 comparisons, which was considered to establish satisfactory consistency in protocol reading and rule construction. To identify general strategies or approaches to the problem, we wrote out each subject’s rules on a single sheet of paper, and formulated some general statements characterizing that subject’s approach. Rules common to groups of subjects were noted. By an iterative approach, clusters of subjects with similar approaches were identified. Ultimately, the subjects were assigned to one of three clusters representing strate-
gies that accounted reasonably well for the decision making of 20 of the 21 subjects. The strategies were identified
reading
empirically and were
and
not
reviewing the coded
formulated before
rules.
Resoks Table 1 summarizes data on the probability of prescribing HRT for each risk combination. The entries are in ascending order of cancer risk and descending
order of fracture risk. The residents’
comments about the cases show that they generally interpreted the risk levels in the cases as intended and that the observed decisions were not a result of systematic misunderstanding of the cases.
The standard-cancer-risk-high-fracture-risk cases the only pair of cases in which the majority of subjects would prescribe HRT, but this was not a unanimous decision, as the range shows. The variability of decision making for each case is demonstrated by the range. Regardless of the risk level for either cancer or fracture, there was at least one physician who would definitely prescribe HRT and one who would not. As stated earlier, both patients classified as having a high risk for cancer and a high risk for fracture had had hysterectomies for early-stage endometrial cancers. Table 1 shows that the probability of prescribing HRT was the same for these two patients as it was for the pair who were at high risk for cancer and at standard risk for fracture. This is initial evidence suggesting that most subjects did not think that patients who had had hysterectomies for endometrial cancer could be assigned to the category of low cancer risk. were
CASE-RELATED DECISION RULES
A set of rules that describe the thought processes of all subjects was developed by summarizing the individual rules for each pair of cases. These rules are presented in table 2. Treatment was most often prescribed for cases with high fracture risk and standard cancer risk (category A). Even in this situation, no treatment was selected almost a third of the time. In general, as cancer risk increased, the preference to pre-
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
268
T1ll0 2 o
*DVT
Making for Hormonal Replacement Therapy, Aggregated by Levels of Cancer and Fracture Risks, of 21 Physicians Responding to 12 Case Vignettes
Rules for Decision
deep venous thrombosis. tcontraindications cancer risk factors, diabetes, hypertension, obesity. *Indications for treatment symptoms, osteoporosis risk factors. =
=
=
scribe HRT declined.
declaring with close
a
toss-up
follow-up,
Cognitive conflict,
or
as
by recommending
was
shown
by
treatment
strongest for the moderate-
cancer-risk-high-fracture-risk cases (category C). Withinsubject inconsistency was also highest for this pair of cases.
Categories E and F show that the management of patients at high risk for cancer was not much affected by a previous medical history of hysterectomy for endometrial cancer. We had thought that some subjects would consider the risk of future endometrial cancer to have been eliminated in these cases (category E),
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
269
thus justifving treatment, while others might consider these patients to be at risk for recurrence or at a higher general risk for future estrogen-dependent cancer of another organ. The similarity of the decisions in categories E and F shows that patients with previous histories of endometrial cancer were almost uniformly viewed as being at continued high cancer risk. All three groups of residents treated these patients as essentially high-risk for recurrence or another malignancy. In their spontaneous thinking aloud, the residents rarely used a numerical probability estimate to talk about the risks in a case. Global expressions of perceived risk (e.g., &dquo;the risk of cancer for this woman is vel-v high&dquo;) were the general rule. They did not naturallv encode their estimates on a probability scale, although they could do so if asked, as in the structured interview.
attention to intra-individual consistency. To what extent did the individual physicians display consistent philosophies or strategies of treatment that cut across cases? ’Three groups characterized by their treatment strategies are summarized in table 3. The most common global strategy, used by eight physicians, resulted in the greatest proportion of HRT prescriptions. It involved scanning the data for indicators of cancer risk and treating only if they were absent. Fracture-risk indicators were secondaiy. A second strategy, used by seven subjects, can be summarized as &dquo;prescribe only if fracture risk is high and no other risk is apparent.&dquo; Since prescription of HRT was limited to cases at high risk for fracture, some weight was evidently given to cancer risk. This strategy led to more indecision than the other two strategies, as represented in the relatively greater number of toss-ups. The third strategy, &dquo;do not prescribe,&dquo; accounted for the decisions of five physicians who repeatedly stated they were reluctant to prescribe HR’I’. Only one prescribed HRT and then for only one case.
TREATMENT STRATEGIES
Having explored changes
in treatment rules
function of the features of the cases,
Table 3 o Higher-order Strategies Response
*FP
=
to 12 Case
family practice residents;
we
Derived from 21
turned
as a
our
Physicians’
Decisions Whether to Prescribe Hormonal Replacement
Vignettes
OB - obstetrics and
gynecology residents;
IM
=
internal medicine residents.
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
Therapy
in
270
Table 4 o Comparison Therapy
’Strategy
in
of Strategies 1 Response to 12
1: if cancer is not
apparent; otherwise, toss-up
a
or
and 2* Derived from 21 Case Vignettes
threat, prescribe; otherwise toss-up
or
Physicians’ Decisions Whether to Prescribe Hormonal Replacement
do not
prescribe; strategy
2: Prescribe
only
if fracture risk is
high
and
no
other risk is
do not prescribe.
The decision making of one subject was so idiosyncratic that it could not be characterized by any of the three global strategies. A review of that physician’s transcript suggests that unfamiliarity with the HRT problem in general resulted in inteipretation of the vignette features. This subject was inconsistent in three pairs of cases, a number equalled by five other subjects and exceeded by one (table 3). However, the other subjects’ decisions could be summarized into strategies because their verbal explanations clarified what they were tiying to do. The unclassifiable subject’s explanations were not as consistent or
unreliable
helpful.
egy egy
2 resembled those of
subjects
13 and 11 in strat-
1.
To validate the classification of subjects into strategies and to determine whether the strategies were related to differences in prescribing, we reviewed the treatment decisions resulting from applying to each
pair the rules corresponding to the different strategies (table 4). This analysis showed that strategies 1 and 2 proceeded along very similar lines for three case pairs (6 and 12 ; 1 and 7; 2 and 8). Both strategies favored not prescribing HRT for high-cancer-risk cases and prescribing it for standard-cancer-risk-high-fracturerisk cases. They differed about what to do with modcase
erate-cancer-risk-high-fracture-risk CASE-RELATED VARIATION BETWEEN STRATEGIES
toss-up. With As table 3 shows, individual clinicians were frequently inconsistent in their judgments within a pair
of cases. Only six subjects were completely consistent, and 13 had two or more inconsistencies. Some inconsistencies were due to misinteupretation of a clinical feature. For example, a feature of the patient’s histoiy intended as a cue for high fracture risk was occasionally interpreted as suggesting standard risk. In other cases, some elements were introduced into the story that were not even in the case vignette. And, on occasion, minor differences in the cases that we thought were immaterial became the basis for different decisions. However, much of the variation was due to cognitive conflicts embedded in the cases, as suggested by table 2. Although the strategies sketched here are not perfect predictors of individual clinicians’ behaviors, the clinicians did seem to be striving to implement them, even if they did not always succeed. Because of the inconsistencies, there were two potential difficulties with the classification of three strategies: 1) prescribing decisions were not uniform within each strategy and 2) the decisions of some subjects might be classified equally well into two different strategies. For example, the decisions of subject 21 in strat-
(4 and 10).
cases
Strategy 1 favored prescribing HRT and following closely; strategy 2 favored not treating or calling the case a a
moderate
cancer
fracture risk (cases 3 and 11),
risk and
most
a
strategy
standard
1
subjects
prescribe HRT and follow closely, while strategy 2 subjects strongly favored not treating. When
preferred
to
both cancer and fracture risks were standard (cases 5 and 9), the differences were most dramatic: all strategy 1 subjects used the rule, &dquo;prescribe if there is no contraindication&dquo; ;all strategy 2 subjects employed a &dquo;do not treat if there is no indication for treatment&dquo; rule. Since patients who are at standard risk for both problems make up the vast majority of the population, deep underlying differences in approach are clearly seen in these strategies. These differences help to explain why hormonal replacement therapy has been such a controversial matter. REGIMEN AND TREATMENT CHOICE
The
policy concerning progestin use expressed by compared with his or her subjective estimate of its effectiveness in reducing probability cancer risk. Eleven of 15 progestin prescribers knew each resident was
use decreases the risk of endometrial four did not. Of the six non-prescribers, four believed progestin use would reduce the risk of
that
progestin
cancer, but
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
271
endometrial cancer but would not use a progestin anyway. Thus, of a group of 21 residents, knowledge base and practice were consistent for only 13. Four did not know why progestin should be used but would prescribe it, and four knew the arguments for using it but did not. The chi-square statistic indicated that the difference in prescribing habits between prescribers and non-prescribers was not statistically significant.
Discussion The data in this study demonstrate an overall patof preferring not to prescribe HRT that is remarkably similar to the collective approach of a more senior group of clinicians assessed earlier?8 Although the views of the academic medical community have changed substantially since 1975,3these results suggest that a reluctance to prescribe HRT persists and that it is related neither to ignorance of the benefits of estrogen in reducing the risk of osteoporosis nor to a lack of awareness of the possibility of combining progestin with estrogen to reduce the risk of endometrial cancer. The physicians’ efforts to balance identified risks and benefits were summarized in three general strategies that characterized 20 of the 21 subjects. The rules formulated from analysis of their verbalizations and the strategies derived from clustering and grouping these rules converge on some central features of intuitive clinical decision making. These features indicate how everyday clinical reasoning differs from a formal, quantitative decision-analytic model and why the conclusions of these two modes of reasoning may differ. First, in everyday clinical reasoning, levels of risk tend to be treated categorically rather than on a continuous probability scale. The formal model requires that probabilities be specified for a detailed set of conditions, events, diseases, etc. Second, a clinical decision rule is naturally and easily encoded as a production rule stating that a certain action should be taken provided certain conditions are present. The task for the decision maker then becomes determining whether those enabling conditions are present. Third, intuitive clinical decision making is rarely undecidedthere were far fewer toss-ups in the observed decisions than in the recommendations of the decisionanalytic models we have used to study this clinical dilemma.l9~2o Fourth, based on our reading of the subjects’ justifications for their decisions, issues of anticipated regret and responsibility for bad outcomes loomed much larger in the observed decisions than they did in the quantitative models. Aggregating across cases and strategies, the residents in this study, like the experienced physicians studied previously, preferred not to prescribe hormonal replacement. This reluctance appears to have been related to the weight they assigned to the perceived risk of cancer and to tern
their wish not to be implicated in increasing that risk. Our data do not permit us to judge whether the physicians were irrationally overweighting a small risk or whether the formal model is insufficiently sensitive to it. Fifth, intuitive decision making generally processes risk factors sequentially rather than simultaneously. A case is scanned first for evidence concerning one risk and, if that does not yield a decision, the second is evaluated. Decisions may vary depending on which risk is considered first. The HRT decision is becoming more complex. The effects of postmenopausal estrogen on the breast and the cardiovascular system are still controversial. Their impact on clinical decision making should be evaluated once they are better understood. METHODOLOGIC CONSIDERATIONS
repeated-measures ANOVA or regression analysis might have been used to &dquo;capture&dquo; the policies of individual judges.21.22 Although there were enough cases in the study to derive reasonably stable estimates of regression weights, we did not use this approach for two reasons. First, we had already done this type of analysis for HRT decisions.&dquo; Moreover, as noted previously, many subjects have a standard approach to HRT decisions that varies little with changes in the independent variables. When subjects have a standard approach to a problem, their algorithms are more likely to be uncovered by verbal protocol-analysis techniques. The results of this study suggest that there are at least three standard algorithms, not just one. While two of these might be represented by different weights given to cancer-risk and fracture-risk terms in a regression equation, the &dquo;do not treat&dquo; strategy could not be adequately characterized this way.23 Protocol analysis showed that an essential part of the processing of clinical information concerned the A
sequence in which these cases were scanned for relevant risk factors. The problem was too complex for simultaneous processing of information. Regression equations, however, assume that a simultaneous scan-
ning procedure is used. A regression equation might be a good decision aid for this clinical problem, but it does not satisfactorily account for how the clinicians actually processed the information. Using case vignettes to study clinical decision making raises the question of external validity: clinicians may respond quite differently to actual cases. Although they allowed us to systematically vary two relevant factors in a clinically realistic format, the case vignettes did not include every factor that might be considered by physicians. On the other hand, the subjects indicated that the cases were engaging and challenging. Although the vignettes were brief, the subjects were rarely undecided about what to do and rarely postponed a decision on the grounds that additional information was needed.
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
272
Even with very brief case vignettes that are much ambiguous and complex than a complete medical history of a real patient, there is substantial variability in the features singled out by physicians as the enabling conditions of a production rule that concludes with a &dquo;treat&dquo; or &dquo;do not treat&dquo; action. Even more variability may be anticipated in actual practice, since
Drs. Emmett Lamb, Fred Wolf, and Peter Politser provided thoughtful and constructive criticism that helped clarify the presentation. Paul Feltovich and Margaret Holmes gave useful advice regarding protocol analysis. The authors are grateful to Robin M. Hogarth for very useful suggestions relating the results to research on causal judgments. They thank James Dod for his assistance with data tabulation.
real cases are even more complex. There was no significant burden on the physicians’ capacity to recall facts since the cases were relatively brief and the respondents made their decisions while the texts of the cases were before them. These features imply that differences in decisions cannot be reasonably attributed to differences in recall. The observed differences should, therefore, be attributed either to differences in the physicians’ knowledge bases or to differences in their structuring of the problems and in the weighing of risk and benefits. The evidence presented shows that differences in knowledge about the outcomes of different regimens cannot account for the reluctance to prescribe HRT. The sample size in this study was small compared with the numbers of clinical encounters commonly analyzed in retrospective health services research. Since
References
less
protocol analysis
is
inherently labor-intensive,
it
was
necessary to confine our efforts to a small set of cases and a small sample of physicians. Any study with only 21 subjects runs a substantial risk of lacking enough statistical power to detect differences between groups (type II error). This study was not well suited to detecting differences between groups of residents in their readiness to prescribe HRT. Protocol analysis is also not the method of choice for identifying factors affecting the decision process that are so implicit as to be unverbalized or that are outside the subject’s awareness, such as organizational variables. Retrospective health services research, including review of the prescribing patterns of physicians, is better suited to documenting the effects of social and organizational variables upon clinical decisions. 21 On the other hand, studies using chart review cannot control for differences in the patient mixes seen by individual groups, and observed differences may be due to these confounding factors. Observational and retrospective studies rarely explain the mechanisms mediating the effects documented 25 Approaching a problem from several points of view and using a combination of methods is one way to overcome the weaknesses of any single research approach. 21 One final comment: we assumed that the probability estimates obtained in the structured interviews were reasonably accurate representations of the beliefs used in responding to the case vignettes, although the interviews elicited subjective probabilities in the context of groups of patients. Subsequent research27,28 has shown that probability estimates may be affected by the elicitation context. This methodologic weakness should be corrected in future research.
CB, Maxson WS. Current status of estrogen therapy for the menopause. Fertil Steril. 1982;37:5-25. 2. NIH Consensus Conference. Osteoporosis. JAMA. 1984;252:7991. Hammond
802.
MacDonald P. Estrogen
plus progestin in postmenopausal women—Act II. N Engl J Med. 1986;315:959-61. 4. Weinstein MC, Schiff I. Cost-effectiveness of hormone replacement therapy in the menopause. Obstet Gynecol Survey. 3.
1983;38:445-55. Hammond CB, Nachtigall LE. Is estrogen replacement therapy necessary? J Reprod Med. 1985;30:797-801. 6. Stampfer MJ, Colditz GA, Willett WC, et al. Postmenopausal estrogen therapy and cardiovascular disease: ten-year follow-up from the Nurses’ Health Study. N Engl J Med. 1991;325:756-62. 7. Lufkin EG, Carpenter PC, Ory SJ, Malkasian GD, Edmonson JH. Estrogen replacement therapy: current recommendations. Mayo Clin Proc. 1988;63:453-60. 8. Mishell DR, Jr. Menopause. In: Droegemueller W, Herbst AL, Mishell DR, Jr, Stenchever MA, eds. Comprehensive gynecology. 5.
St. Louis: C. V.
Mosby,
1987.
L, Adami HO, Persson I, Hoover R, Schairer C. The risk of breast cancer after estrogen and estrogen-progestin replacement. N Engl J Med. 1989;321:293-7. 10. Hulka BS. Hormone-replacement therapy and the risk of breast cancer. Ca. 1990;40:289-96. 11. Stevenson JC, Whitehead MI. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:201-2. 12. Jacobs SL, Luciano AA, Peterson, MGE, Kegeles SS. Breast cancer and estrogen replacement (letter). N Engl Med J. 1990;322:202. 13. Cutler WB, Genovese E. Breast cancer and estrogen replacement 9.
Bergkvist
(letter). N Engl J Med. 1990;322:202-3. Epstein SE. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:203. 15. Mauvais-Jarvis P, Kuttenn F. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:203. 16. Simon JA. Breast cancer and estrogen replacement (letter). N Engl J Med. 1990;322:203-4. 17. Bergkvist L, Adami HO, Persson I, Schairer C, Hoover RN. Breast cancer and estrogen replacement (letter). N Engl J Med.
14.
1990;322:204. AS, Holzman GB, Ravitch MM, et al. Comparison of physicians’ decisions regarding estrogen replacement therapy for menopausal women and decisions derived from a decision an-
18. Elstein
model. Am J Med. 1986;80:246-58. Holzman GB, Ravitch MM, Metheny WP, Rothert ML, Holmes MM, Hoppe RB. Physicians’ judgments about estrogen replacement therapy for menopausal women. Obstet Gynecol. 1984;63:303-11. 20. Elstein AS, Dod JM, Holzman GB. Estrogen replacement decisions of third-vear residents: clinical intuition and decision analysis. In: Evans DA, Patel VL, eds. Cognitive science in medicine. Cambridge, MA: MIT Press, 1989. 21. Hammond KR, McClelland GH, Mumpower J. Human judgment and decision making: theories, methods, and procedures. New
alytic
19.
York, Praeger, 1980. 22.
Wigton RS. Use of linear models to analyze physicians’ decisions. Med Decis Making. 1988;8:241-52.
23. Holmes MM, Rovner
longo
DR, Rothert ML, Schmitt N, Given CW, Ia-
NS. Methods of
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
analyzing physician practice patterns
in
273
24.
25. 26. 27.
hypertension. Med Care. 1989;27:59-68. Epstein AM, McNeil BJ. Physician characteristics and organizational factors influencing use of ambulatory tests. Med Decis Making. 1985 ;5:401-15. Schwartz RK, Soumerai SB, Avorn J. Physician motivations for nonscientific drug prescribing. Soc Sci Med. 1989;28:577-82. Elstein AS, Shulman LS, Sprafka SA. Medical problem solving: a ten-year retrospective. Eval Health Prof. 1990;13:5-36. Ellis RD, Alderman BW, Elstein AS. Subjective probabilities and estrogen replacement therapy (abstr). Med Decis Making. 1988;8:336.
28.
Redelmeier DA, Tversky A. Discrepancy between medical decisions for individuals and for groups. N Engl J Med.1990;322:11624.
0
APPENDIX B For illustrative purposes, we present the complete texts of the responses of two subjects to a single case. The case is standard risk for cancer and standard risk for fracture. One subject is clearly willing to prescribe, the other is not.
Subject
2
Again this lady, the symptoms of postmenopausal, you know, hot flashes and flushes occurring for two or three years, five years. It’s very unusual of them to occur for longer than that. She’s very active. Only need to tell her to take a lot of dairy products, calcium, calcium replacement [unintelligible phrase] twice a day. You give her estrogen and what happens is that she’s spotting. Any woman with postmenopausal spotting you have to suspect cancer, and by giving her the estrogen, if she develops any vaginal spotting then she has to go through the whole thing, D&C, that kind of junk. The symptoms don’t bother her.
Marked
APPENDIX A
of
Rule: If estrogen will
Alison Zimmer is 50 years old, 5’4&dquo; tall, 135 pounds, married, gravida 3, para 2, Ab 1. She is the sales manager of a discount department store and plays racketball three times a week for recreation. She has experienced hot flashes and flushes 2-4 times per day for the last 14 months, but says they are a minor annoyance. Her last menstrual period was 12 months ago, and she has had no vaginal bleeding since then. Mark an &dquo;X&dquo; along the prescribe estrogen.
probability
line to indicate how
likely
you
are
to
Subject
prescribing: cause
0.12.
spotting,
do not
prescribe.
111
No postmenopausal bleeding’s important. LMP a year and half ago, they do not disturb her. She has occasional hot flashes. I’d probably give this lady combination therapy, mostly because of her risk for osteoporosis. I’m not sure that her hot flashes are probably that important, but she really has no contraindications and with close follow-up hopefully we’ll keep these ladies from breaking their hips. I think this lady should get it. a
Marked
probability
Rule: If there is
no
Downloaded from mdm.sagepub.com at Monash University on April 12, 2015
of
prescribing:
1.0.
contraindication, prescribe.
