CORRESPONDENCE
against potential benefit. Prehospital 12-lead ECGs d e m o n s t r a t e d their unique value in several cases by capturing prehospital ischemic events that resolved following paramedic treatment prior to hospital arrival. The potential risks and benefits of this technology are worthy of continued and careful investigation. The Milwaukee Prehospital Chest Pain Project is conducting a series of prospective studies to determine if, how, and in what patient subset beneficial changes may be implemented safely. Tom P Aufderheide, MD Gaff E Hendley, MD Emergency Medicine Medical College of Wisconsin Milwaukee 1. Hedges JR, Kobernick MS: Detection of myocardial ischemia/infarction in the emergency department patient with chest discomfort. Emerg Med ClJn North Am 1988;6:317-340. 2. Behar S, Scher S, Karix I, et al: Evaluation of the electrocardiogram in the emergency room as a decision making tool. Chest 1977;7h486-49i. 3. McQueen MJ, Holder D, E1-Maraghi RH: Assessment of the accuracy of serial electrocardiograms in the diagnosis of myocardial infarction. A m Heart J 1983; t05:258-261. 4. Lee TH, Cook EF, Weisberg M, et aI: Acute chest pain in the emergency room: Identification and examination of low risk patient. Arch Intern Med 1985; 145:65-69. 5. Wrenn KD: Protocols in the emergency room evaluation of chest pain: Do they fail to diagnose lateral wall myocardial infarction? J Gen Intern Med 1987;2:66-67. 6. Selker HP, D'Agostino RB, Laks MM: A predictive instrument for acute ischemic heart disease to improve coronary care unit admission practices: A potential online tool in a computerized electrocardiograph. J Electrocardiography 1988;2hS1 l-S17. 7. Selker HP: Coronary care unit triage decision aids: How do we know when they work? A m J Med t989;87:491-493.
CRP as a Screening Test for L u m b a r P u n c t u r e ? To the Editor: T h e a r t i c l e by L e m b o a n d Marchant, "Acute Phase Reactants and Risk of Bacterial M e n i n g i t i s Among Febrile Infants and Children" [January 1991;20:36-44] concluded that "measurement of CRP in serum is useful as an adjunct to history and physical examination for the detection of acute bacterial meningitis in 168/943
the acutely febrile child." In practical terms this implies that C-reactive protein (CRP) must either identify children needing lumbar puncture (LP) in whom LP would not usually be performed, or identify children with suspicious clinical findings in whom LP can be safely deferred. We believe that the data presented cannot a d e q u a t e l y address either of these important issues. The study population necessary to answer the question at hand must be representative of the universal pool of "acutely febrile children," ideally consecutive febrile children. The authors made the decision to study only children in whom the treating physicians had already determined that an LP was clinically necessary. Thus, a more appropriate conclusion from their data is that CRP is more likely to be elevated in children who need an LP anyway. The fundamental basis of the authors' results is that an elevated CRP is in some way associated with meningitis. They reported that eight of 75 had m e n i n g i t i s in the high CRP group and two of 85 in the low CRP group, an association barely achieving statistical significance (Fisher's exact test, P = .047).1 If one more patient with meningitis had possessed a low CRP the comparison would lack significance (P - .189), as it would if one more high CRP patient had not had meningitis (P = .084). We believe that research findings in which statistical significance pivots on a single patient lack appropriate s t r e n g t h to be used for c l i n i c a l decision making, especially when the decision is as vitally important as determining w h i c h children should receive LP. The authors describe their research simply as a "cross-sectional survey" and include c o m m e n t s ("patients were enrolled") to suggest that their data were collected prospectively. Specific wording in the results section ("meningeal signs were not recorded for three children"), however, implies that the study format was a chart review. If truly retrospective, the reliability of the study's multiple discretionary assignments (eg, "irritability," "lethargy," "poorly conAnnals of Emergency Medicine
soled crying") is dependent on such subjective factors as chart documentation and the judgment of a reviewer who did not see the patient and was not blinded to LP results. The fact that more than half the subjects enrolled (58%)had "insufficient sera" for CRP analysis also argues against a prospective study, as one would have expected the enrolling physicians to insist on redraw. Other critical facts were not presented. Although a median age of 6 months is described, the mean, standard deviation, and range were not, leaving the reader to wonder how many 10- to-15-year-olds were being compared to 1-month-olds in this analysis. To alleviate worries of selection bias suggested by excluding 58% due to "insufficient sera," a c o m p l e t e c o m p a r a t i v e l i s t i n g of signs, symptoms, diagnoses, and LP results is necessary. The data given regarding the relation of WBC count to meningitis were derived from only 154 of the 160 study patients; no explanation was given for the excluded six. Review of Tables 4 and 5 of their report suggests "data dredging. ''2 The authors identified in the methods section the intent to predict meningitis based on four features: signs, symptoms, WBC count, and CRP level. These features can be taken alone or combined in 15 possible permutations 3 using "and" arguments (eg, signs and high CRP); if one also uses "or" arguments there are hundreds of available combinations (eg, high WBC and signs or symptoms). The authors neither stated prospectively which of these numerous feature combination, s they intended to a n a l y z e nor r e t r o s p e c t i v e l y how many they actually did analyze. The fact that they identified one of these groupings (signs and high CRP) that was sensitive in identifying meningitis is hardly surprising. Because only ten patients had meningitis, the a u t h o r s appear to have a n a l y z e d m a n y more feature c o m b i n a t i o n s than they did actual diseased patients. We conclude that this report lacks appropriate methodology to afford useful conclusions regarding the util20:8 August 1991
CORRESPONDENCE
ity of CRP in s c r e e n i n g for m e n ingitis.
Steven M Green, MD Steven G Rothrock, MD Riverside General Hospital Loma Linda University Medical Center Riverside and Loma Linda, California 1. 8ystat @ 5.0 statistical software, Evanston, Illinois. 2. Berry DA: Multiple comparisons, multiple tests, and data dredging: A Bayesian perspective, in: Bernardo JM, DeGroot MH, Lindley DV, et aI (eds]: Bayesian Statistics 3. Oxford, Oxford University Press, 1988, p 79-94. 3. Woolson RF: Statistical Methods .for the Analysis of Biomedical Data. New York, John Wiley and Sons, 1987, p 82-85.
'Economic Malpractice': Inappropriate Use of Cost Analysis To the Editor." T h e article, " C o s t - E f f e c t i v e n e s s A n a l y s i s of P a r a m e d i c E m e r g e n c y Medical Services in the Treatment of P r e h o s p i t a l C a r d i o p u l m o n a r y Arrest" by Valenzuela et al [December 1990;19:1407-1411] has a n u m b e r of p r o b l e m s . T h e q u e s t i o n p o s e d is w r o n g , a n d t h e c o s t a n a l y s i s is wrong. About 1.5 million Americans have heart attacks each year; 300,000 die before reaching the hospital. An excellent study by Eisenberg et al, "Survival Rates F r o m O u t - o f - H o s p i t a l Cardiac Arrest: R e c o m m e n d a t i o n s for Uniform Definitions and Data to R e p o r t " [ N o v e m b e r 1990;19:12491257], found the highest average survival rates (defined as survival to hospital discharge) from cardiac arrest from ventricular fibrillation in e m e r g e n c y m e d i c a l services (EMS) systems employing the highest trained individuals, ie, EMT-Ds plus paramedics. Yet it is invalid to define the cost of treating sudden cardiac death in the prehospital environment as "the additional cost of operating a paramedic level EMS system versus an Intermediate EMT level system" with vastly different response times. The Valenzuela cost analysis is inappropriate. Costs have two basic categories: fixed and variable (marginal). 1 The authors cannot count the fixed cost items in their analysis that 20:8 August 1991
m u s t exist if a basic EMS system is to exist. Is the only reason to have an advanced paramedic s y s t e m just to treat one condition - cardiac arrest? No. The appropriate cost analysis is a marginal cost analysis 2 of the variable costs for each ambulance run in the care of a cardiac arrest victim. This m a r g i n a l cost should be estimated by taking the total overhead operating costs of the paramedic run system (including education training costs3), multiplied by the fraction of runs made for cardiac arrest, added to the special equipment and medication costs for each cardiac arrest run (intubation, IV therapy, and monitor/ defibrillators). A l t h o u g h Eisenberg et al defined survivors of out-of-hospital cardiac arrest as those surviving to hospital discharge (including those with neurologic deficits and patients disc h a r g e d to n u r s i n g homes), m o r e r e f i n e m e n t is n e e d e d in o u t c o m e measures to be able to define "effectiveness." Comparing costs per year of life saved for such disparate disease processes as cardiac arrest, acute leukemia, and diseases treated with heart, liver, and bone marrow transplantation is like comparing apples to artichokes to argula - they're not in the same category even if they are all life-threatening (or edible). We believe that the cardiac arrest p a t i e n t p o p u l a t i o n in Valenzuela's study should exclude pediatric arrest, poisonings, and k n o w n terminal illness as well as traumatic cardiac arrest. It would be more valid and more intriguing to c o m p a r e age-adjusted s u r v i v a l rates f r o m v a r i o u s treatments as measured by years of life in which patients are discharged able to resume their previous functional level of activity.
Janet A Eastaugh, MD, FACEP Steven R Eastaugh, ScD George Washington University Washington, DC 1. Eastaugh SR: Medical Economics and Health Fi nance. Westport, Connecticut, Greenwood Press, 1981, p 340. 2. Eastaugh SR: Financing Health Care: Economic Efficiency and Equit]z Westport, Connecticut, Greenwood Press, 1987, p 720. 3. Eastaugh SR: Financing the rate of growth of medical technology. Quarterly Review of Economics and Busi ~ hess 1990~30:92-98.
Annals of Emergency Medicine
In Reply: I am grateful for the careful attention paid by the Drs Eastaugh to our article on cost-effectiveness analysis of p a r a m e d i c EMS. U n f o r t u n a t e l y , their observations are in error, likely because of an unfamiliarity with prehospital care in general and the treatm e n t of out-of-hospital cardiopulmonary arrest in particular. It is entirely valid to compare costs in a paramedic EMS system with rigorous response time standards (four minutes basic life support response; eight minutes advanced life support response) to an I n t e r m e d i a t e EMT system with response times that do not meet that standard. The former is capable of consistent successful resuscitation of out-of-hospital arrest; the latter is not.t Therefore, the additional resources required are legitimately the marginal costs of a serious effort to address this problem. Ask any EMS medical director. Interestingly, Drs Eastaugh seem unaware that modern paramedic EMS s y s t e m s were, in fact, established primarily to treat the problem of prehospital cardiac arrest. I am well aware that paramedic EMS syst e m s treat other conditions; however, quantifying the benefits (lives saved) in conditions other than cardiac arrest is, at present, not possible. Our analysis, therefore, determined a m i n i m u m cost-effectiveness for paramedic EMS. To the extent that these benefits exist, as clearly stated in the article, paramedic cost-effectiveness was underestimated. Drs E a s t a u g h prefer t h e i r o w n method of calculating marginal cost; this is entirely understandable. Our methods arc not, therefore, rendered "inappropriate." I direct Annals readers to Dr Eisenberg's book, Sudden Cardiac Death in the Community, z w h i c h c o n t a i n s a cost-effectiveness analysis of alternative EMS system configurations using methods very similar to ours. Readers will decide for themselves whether the suggestions of Drs Eastaugh are more appropriate or useful. I, too, am interested in the course of o u t - o f - h o s p i t a l arrest survivors and am currently studying it. However, the end points (lives saved and 944/169
against potential benefit. Prehospital 12-lead ECGs d e m o n s t r a t e d their unique value in several cases by capturing prehospital ischemic events that resolved following paramedic treatment prior to hospital arrival. The potential risks and benefits of this technology are worthy of continued and careful investigation. The Milwaukee Prehospital Chest Pain Project is conducting a series of prospective studies to determine if, how, and in what patient subset beneficial changes may be implemented safely. Tom P Aufderheide, MD Gaff E Hendley, MD Emergency Medicine Medical College of Wisconsin Milwaukee 1. Hedges JR, Kobernick MS: Detection of myocardial ischemia/infarction in the emergency department patient with chest discomfort. Emerg Med ClJn North Am 1988;6:317-340. 2. Behar S, Scher S, Karix I, et al: Evaluation of the electrocardiogram in the emergency room as a decision making tool. Chest 1977;7h486-49i. 3. McQueen MJ, Holder D, E1-Maraghi RH: Assessment of the accuracy of serial electrocardiograms in the diagnosis of myocardial infarction. A m Heart J 1983; t05:258-261. 4. Lee TH, Cook EF, Weisberg M, et aI: Acute chest pain in the emergency room: Identification and examination of low risk patient. Arch Intern Med 1985; 145:65-69. 5. Wrenn KD: Protocols in the emergency room evaluation of chest pain: Do they fail to diagnose lateral wall myocardial infarction? J Gen Intern Med 1987;2:66-67. 6. Selker HP, D'Agostino RB, Laks MM: A predictive instrument for acute ischemic heart disease to improve coronary care unit admission practices: A potential online tool in a computerized electrocardiograph. J Electrocardiography 1988;2hS1 l-S17. 7. Selker HP: Coronary care unit triage decision aids: How do we know when they work? A m J Med t989;87:491-493.
CRP as a Screening Test for L u m b a r P u n c t u r e ? To the Editor: T h e a r t i c l e by L e m b o a n d Marchant, "Acute Phase Reactants and Risk of Bacterial M e n i n g i t i s Among Febrile Infants and Children" [January 1991;20:36-44] concluded that "measurement of CRP in serum is useful as an adjunct to history and physical examination for the detection of acute bacterial meningitis in 168/943
the acutely febrile child." In practical terms this implies that C-reactive protein (CRP) must either identify children needing lumbar puncture (LP) in whom LP would not usually be performed, or identify children with suspicious clinical findings in whom LP can be safely deferred. We believe that the data presented cannot a d e q u a t e l y address either of these important issues. The study population necessary to answer the question at hand must be representative of the universal pool of "acutely febrile children," ideally consecutive febrile children. The authors made the decision to study only children in whom the treating physicians had already determined that an LP was clinically necessary. Thus, a more appropriate conclusion from their data is that CRP is more likely to be elevated in children who need an LP anyway. The fundamental basis of the authors' results is that an elevated CRP is in some way associated with meningitis. They reported that eight of 75 had m e n i n g i t i s in the high CRP group and two of 85 in the low CRP group, an association barely achieving statistical significance (Fisher's exact test, P = .047).1 If one more patient with meningitis had possessed a low CRP the comparison would lack significance (P - .189), as it would if one more high CRP patient had not had meningitis (P = .084). We believe that research findings in which statistical significance pivots on a single patient lack appropriate s t r e n g t h to be used for c l i n i c a l decision making, especially when the decision is as vitally important as determining w h i c h children should receive LP. The authors describe their research simply as a "cross-sectional survey" and include c o m m e n t s ("patients were enrolled") to suggest that their data were collected prospectively. Specific wording in the results section ("meningeal signs were not recorded for three children"), however, implies that the study format was a chart review. If truly retrospective, the reliability of the study's multiple discretionary assignments (eg, "irritability," "lethargy," "poorly conAnnals of Emergency Medicine
soled crying") is dependent on such subjective factors as chart documentation and the judgment of a reviewer who did not see the patient and was not blinded to LP results. The fact that more than half the subjects enrolled (58%)had "insufficient sera" for CRP analysis also argues against a prospective study, as one would have expected the enrolling physicians to insist on redraw. Other critical facts were not presented. Although a median age of 6 months is described, the mean, standard deviation, and range were not, leaving the reader to wonder how many 10- to-15-year-olds were being compared to 1-month-olds in this analysis. To alleviate worries of selection bias suggested by excluding 58% due to "insufficient sera," a c o m p l e t e c o m p a r a t i v e l i s t i n g of signs, symptoms, diagnoses, and LP results is necessary. The data given regarding the relation of WBC count to meningitis were derived from only 154 of the 160 study patients; no explanation was given for the excluded six. Review of Tables 4 and 5 of their report suggests "data dredging. ''2 The authors identified in the methods section the intent to predict meningitis based on four features: signs, symptoms, WBC count, and CRP level. These features can be taken alone or combined in 15 possible permutations 3 using "and" arguments (eg, signs and high CRP); if one also uses "or" arguments there are hundreds of available combinations (eg, high WBC and signs or symptoms). The authors neither stated prospectively which of these numerous feature combination, s they intended to a n a l y z e nor r e t r o s p e c t i v e l y how many they actually did analyze. The fact that they identified one of these groupings (signs and high CRP) that was sensitive in identifying meningitis is hardly surprising. Because only ten patients had meningitis, the a u t h o r s appear to have a n a l y z e d m a n y more feature c o m b i n a t i o n s than they did actual diseased patients. We conclude that this report lacks appropriate methodology to afford useful conclusions regarding the util20:8 August 1991
CORRESPONDENCE
ity of CRP in s c r e e n i n g for m e n ingitis.
Steven M Green, MD Steven G Rothrock, MD Riverside General Hospital Loma Linda University Medical Center Riverside and Loma Linda, California 1. 8ystat @ 5.0 statistical software, Evanston, Illinois. 2. Berry DA: Multiple comparisons, multiple tests, and data dredging: A Bayesian perspective, in: Bernardo JM, DeGroot MH, Lindley DV, et aI (eds]: Bayesian Statistics 3. Oxford, Oxford University Press, 1988, p 79-94. 3. Woolson RF: Statistical Methods .for the Analysis of Biomedical Data. New York, John Wiley and Sons, 1987, p 82-85.
'Economic Malpractice': Inappropriate Use of Cost Analysis To the Editor." T h e article, " C o s t - E f f e c t i v e n e s s A n a l y s i s of P a r a m e d i c E m e r g e n c y Medical Services in the Treatment of P r e h o s p i t a l C a r d i o p u l m o n a r y Arrest" by Valenzuela et al [December 1990;19:1407-1411] has a n u m b e r of p r o b l e m s . T h e q u e s t i o n p o s e d is w r o n g , a n d t h e c o s t a n a l y s i s is wrong. About 1.5 million Americans have heart attacks each year; 300,000 die before reaching the hospital. An excellent study by Eisenberg et al, "Survival Rates F r o m O u t - o f - H o s p i t a l Cardiac Arrest: R e c o m m e n d a t i o n s for Uniform Definitions and Data to R e p o r t " [ N o v e m b e r 1990;19:12491257], found the highest average survival rates (defined as survival to hospital discharge) from cardiac arrest from ventricular fibrillation in e m e r g e n c y m e d i c a l services (EMS) systems employing the highest trained individuals, ie, EMT-Ds plus paramedics. Yet it is invalid to define the cost of treating sudden cardiac death in the prehospital environment as "the additional cost of operating a paramedic level EMS system versus an Intermediate EMT level system" with vastly different response times. The Valenzuela cost analysis is inappropriate. Costs have two basic categories: fixed and variable (marginal). 1 The authors cannot count the fixed cost items in their analysis that 20:8 August 1991
m u s t exist if a basic EMS system is to exist. Is the only reason to have an advanced paramedic s y s t e m just to treat one condition - cardiac arrest? No. The appropriate cost analysis is a marginal cost analysis 2 of the variable costs for each ambulance run in the care of a cardiac arrest victim. This m a r g i n a l cost should be estimated by taking the total overhead operating costs of the paramedic run system (including education training costs3), multiplied by the fraction of runs made for cardiac arrest, added to the special equipment and medication costs for each cardiac arrest run (intubation, IV therapy, and monitor/ defibrillators). A l t h o u g h Eisenberg et al defined survivors of out-of-hospital cardiac arrest as those surviving to hospital discharge (including those with neurologic deficits and patients disc h a r g e d to n u r s i n g homes), m o r e r e f i n e m e n t is n e e d e d in o u t c o m e measures to be able to define "effectiveness." Comparing costs per year of life saved for such disparate disease processes as cardiac arrest, acute leukemia, and diseases treated with heart, liver, and bone marrow transplantation is like comparing apples to artichokes to argula - they're not in the same category even if they are all life-threatening (or edible). We believe that the cardiac arrest p a t i e n t p o p u l a t i o n in Valenzuela's study should exclude pediatric arrest, poisonings, and k n o w n terminal illness as well as traumatic cardiac arrest. It would be more valid and more intriguing to c o m p a r e age-adjusted s u r v i v a l rates f r o m v a r i o u s treatments as measured by years of life in which patients are discharged able to resume their previous functional level of activity.
Janet A Eastaugh, MD, FACEP Steven R Eastaugh, ScD George Washington University Washington, DC 1. Eastaugh SR: Medical Economics and Health Fi nance. Westport, Connecticut, Greenwood Press, 1981, p 340. 2. Eastaugh SR: Financing Health Care: Economic Efficiency and Equit]z Westport, Connecticut, Greenwood Press, 1987, p 720. 3. Eastaugh SR: Financing the rate of growth of medical technology. Quarterly Review of Economics and Busi ~ hess 1990~30:92-98.
Annals of Emergency Medicine
In Reply: I am grateful for the careful attention paid by the Drs Eastaugh to our article on cost-effectiveness analysis of p a r a m e d i c EMS. U n f o r t u n a t e l y , their observations are in error, likely because of an unfamiliarity with prehospital care in general and the treatm e n t of out-of-hospital cardiopulmonary arrest in particular. It is entirely valid to compare costs in a paramedic EMS system with rigorous response time standards (four minutes basic life support response; eight minutes advanced life support response) to an I n t e r m e d i a t e EMT system with response times that do not meet that standard. The former is capable of consistent successful resuscitation of out-of-hospital arrest; the latter is not.t Therefore, the additional resources required are legitimately the marginal costs of a serious effort to address this problem. Ask any EMS medical director. Interestingly, Drs Eastaugh seem unaware that modern paramedic EMS s y s t e m s were, in fact, established primarily to treat the problem of prehospital cardiac arrest. I am well aware that paramedic EMS syst e m s treat other conditions; however, quantifying the benefits (lives saved) in conditions other than cardiac arrest is, at present, not possible. Our analysis, therefore, determined a m i n i m u m cost-effectiveness for paramedic EMS. To the extent that these benefits exist, as clearly stated in the article, paramedic cost-effectiveness was underestimated. Drs E a s t a u g h prefer t h e i r o w n method of calculating marginal cost; this is entirely understandable. Our methods arc not, therefore, rendered "inappropriate." I direct Annals readers to Dr Eisenberg's book, Sudden Cardiac Death in the Community, z w h i c h c o n t a i n s a cost-effectiveness analysis of alternative EMS system configurations using methods very similar to ours. Readers will decide for themselves whether the suggestions of Drs Eastaugh are more appropriate or useful. I, too, am interested in the course of o u t - o f - h o s p i t a l arrest survivors and am currently studying it. However, the end points (lives saved and 944/169
