Severity of Illness in Pregnancy' Stephen E. Lapinsky, MB BCh, MSc, FRCPC Intensive Care Unit Mount Sinai Hospital; and Interdepartmental Division of Critical Care Medicine University of Toronto Toronto, ON, Canada
S
everity-of-illness scores have been used for over 30 years in critical care for a variety of roles, including case-mix adjustment in research, benchmarking, performance improvement, resource utilization, and clinical decision support (1). A variety of approaches have been used in the development of these scores, but all have made use of large databases of clinical data for the derivation and validation of the score. The performance of the scoring system may be affected by the population of patients used in its derivation, with variability occurring in the geographic region, the type of ICU, and the diagnostic case-mix of patients. Obstetric patients account for only about 2.5% of ICU patients (2), and therefore, the validity of these scoring systems to the pregnant or postpartum patient is in question. Several studies from different geographic regions have evaluated Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score (SAPS) II in the obstetric patient, demonstrating good discrimination between survivors and nonsurvivors but overestimating mortality, with significant variation between studies (3-6). Many of these scoring systems use diagnosis-specific weighting systems for the calculation of predicted mortality— unfortunately such weighting is not available for common obstetric diagnoses. The physiological changes in the pregnant woman may result in alterations in the normal range of values for several physiological and laboratory variables, skewing the points allocated to a score. However, a study that corrected for this altered physiology did not demonstrate an improvement in the operating characteristics of APACHE II or SAPS II (7). The Mortality Prediction Model (MPM) scoring system calculates a score at ICU admission (MPM^), unlike APACHE and SAPS which use data from the first 24 hours in the ICU. The MPM-II and the recalibrated MPM-III scores are simple and easy to apply, using a relatively small number of variables which are easily obtainable at ICU admission (8,9). Evaluation of the MPMij-III score in six diagnostic subgroups (not including obstetric patients, but evaluating cardiovascular, trauma, medical, elective surgery, neurosurgery, and emergency surgery)
'See also p. 1047. Key Words: oritioal care; pregnancy complications; severity of illness index; triage The author has disclosed that he does not have any potential conflicts of interest. Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins
identified excellent performance in each of these subgroups (10). Few studies have evaluated the MPM scoring system in the obstetric patient. Using a small database, el-Solh and Grant (3) found the MPM-II score to accurately predict mortality in critically ill obstetric patients. MPM has potential advantages as an obstetric scoring system, in that its performance appears to be less dependent on the diagnostic subgroup, and it relies less on physiological and laboratory values which may be altered by pregnancy. In this issue of Critical Care Medicine, Rojas-Suarez et al ( 11 ) contribute significantly to this field with a study evaluating several scoring systems in a relatively large database of obstetric patients admitted to their ICU in Cartagena, Colombia. Their single-center study identified 726 obstetric patients admitted to the ICU over a 5-year period, with 22 obstetric-related deaths and nine indirect deaths (overall mortality 4.27%). SAPS-2, SAPS-3, MPM^-II, and MPMj-III were evaluated. Each showed good discrimination, with an area under the receiver-operating characteristic (ROC) curve of greater than 0.86. Calibration, that is, the relationship between the actual and predicted mortality was good only for the MPMjj-II score, with a mortality ratio of 0.88 and supportive Hosmer-Lemeshow statistic. The SAPS scores significantly overestimated mortality and MPM^-III slightly underestimated mortality. However, the results generated by this cohort may not be generalizable to all ICUs that admit obstetric patients— their ICU admission rate of 14.2 per 1,000 deliveries is higher than the median 2.7 per 1,000 deliveries reported in a systematic review of obstetric ICU admissions (2). The Colombian cohort also differed from the cohorts in this review in the relatively high percentage of pregnant patients on ICU admission (25% vs median 16%), and the low requirement for mechanical ventilation (25% vs median 41%) (2). ICU scoring systems play a number of valuable roles, but in obstetric management, they have also been applied to the evaluation and management of the individual patient. Scoring systems for obstetric-specific conditions have been developed and validated. The Pre-eclampsia Integrated Estimate of Risk score has been shown to allow the early identification of women at risk of complications of preeclampsia, permitting modification of management (12). A Maternal Critical Care Working Group of the Royal College of Obstetricians and Gynaecologists in 2011 recommended monitoring ill pregnant and postpartum women using an Early Warning Score that had been validated in pregnancy (13). In a recent publication using the British Intensive Care National Audit and Research Centre dataset. Carle et al (14) describe the evaluation of several preexisting obstetric early warning scores and the development and validation of a new obstetric score and demonstrate excellent discrimination for this new score (area under ROC curve of 0.995). These scores are able to accurately identify patients at high risk of mortality and are therefore of value in patient management and triage. They are not developed to generate a predicted mortality rate. Other
DOI:10.1097/CCM.0000000000000170 May 2014 • Volume 42 • Number 5 1284
www,ccmjournal,org
Editorials
international efforts are underway to develop a general obstetric severity-of-illness score (15). The very good discrimination and calibration obtained for MPM^^-II by Rojas-Suarez et al (11) suggests that this is a score against which new systems should be compared.
REFERENCES 1. Keegan MT, Gajic O, Afessa B: Severity of iiiness scoring sysfems in the intensive care unit. Crit Care Med 2011 ; 39:163-1 69 2. Poiiock W, Rose L, Dennis CL; Pregnant and postpartum admissions to the intensive care unit: A systematic review. Intensive Care Med 2010; 36:1465-1474 3. el-Soih AA, Grant BJ: A comparison of severity of illness scoring systems for critically iii obstetric patients. Chest ^9Q6; 110:1299-1304 4. Hazelgrove JF, Price C, Pappachan VJ, et al: Multicenter study of obstetric admissions to 14 intensive care units in southern England. Crit Care Med 2001 ; 29:770-775 5. Vasquez DN, Estenssoro E, Canales HS, et al: Ciinical characteristics and outcomes of obsfetric patients requiring ICU admission. Chest 2007; 131:718-724 6. Tempe A, Wadhwa L, Cupta S, et al: Prediction of mortality and morbidity by simplified acute physiology score II in obstetric intensive care unit admissions. Indian J Med Sei 2007; 61:179-185 7. Lapinsky SE, Hallett D, Collop N, ef al; Evaiuation of sfandard and modified severity of illness scores in the obstetric patient. J Crit Care 2011; 26:535.e1-535.e7
8. Lemeshow S, Teres D, Klar J, et al: Morfaiify Probability Models (MPM II) based on an international cohort of intensive care unif patients. JAMA 1993; 270:2478-2486 9. Higgins TL, Kramer AA, Nathanson BH, et al: Prospective vaiidation of fhe infensive care unit admission Mortaiity Probability Model (MPMO-III). Crit Care Med 2009; 37:1619-1623 10. Nathanson BH, Higgins TL, Kramer AA, et al: Subgroup mortality probability models: Are fhey necessary for specialized intensive care units? Crit Care Med 2009; 37:2375-2386 11. Rojas-Suarez J, Pafernin-Caicedo A, Miranda J, ef ai: Comparison of Severity-of-lliness Scores in Criticaiiy III Cbstetric Patients: A 6-Year Retrospective Cohort. Crit Care Med 2014; 42:1047-1054 1 2. von Dadelszen P, Payne B, Li J, et al; PIERS Study Group; Prediction of adverse mafernal outcomes in pre-eclampsia; Developmenf and validation of fhe fullPIERS model. Lancet 2011 ; 377;219-227 13. Mafernai Crificai Care Working Group: Providing Equity of Critical and Maternity Care for the Critically III Pregnant or Recently Pregnant Woman. London, Royal Coilege of Cbstetricians and Gynecologists, 2011. Avaiiabie at: http://vvww.rcog.org.uk/files/rcog-corp/Prov_Eq_ MatandCritCare.pdf. Accessed November 6, 2013 14. Carle C, Alexander P, Coiumb M, et ai: Design and internal validation of an obstetric early warning score: Secondary analysis of fhe Infensive Care Nafional Audit and Research Centre Case Mix Programme database. Anaesthesia 2013; 68:354-367 15. Canadian Research Informafion System; Ciinical prediction models for criticaiiy ill pregnant women: MEOWSandCIPHER.Availableat: http:// webapps.cihr-irsc.gc.ca/cris/detail_e?pResearchld=4339831&p_ version=CRIS&pJanguage=E&p_sessionJd=1333597 Accessed November 6, 2013
ICU Discharge Bias Reveáis Etiiicaiiy Troubiing Pay-for-Performance Benciimark iVietrics* Michael Rie, MD Department of Anesthesia University of Kentucky College of Medicine Lexington, KY
H
ealthcare payment reform is transitioning from fee-for-service payments to an outcomes-driven value-based system. Critical care health services research now confronts "pay-for-performance criteria" from Medicare and insurance funds. In this issue of Critical Care Medicine, Reineck et al ( 1 ) inform us that in-hospital mortality within 30 days or readmission to hospital within 30 days after discharge presently defines substandard quality payment benchmarks
'See also p. 1055. Key Words: intensive care unit discharge bias; mortaiity; nafionai quaiity forum; patient aufonomy; pay-for-performance benchmark mefrics; prognostic scoring systems; rationing intensive oare unit care Dr. Rie served as a board member for Physicians Againsf Drug Shorfages (Patient Advocacy Group) and received support for article preparation from New York Times Cp-ED. Copyright ® 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wiikins
Critical Care Medicine
for Medicare and the National Quality Eorum (NQF) (2). These Pennsylvania investigators previously reported (3) using the same governmental database of the Pennsylvania Health Care Cost Containment Council capturing all nongovernmental hospital admissions. They now extend their research to "discharge bias" across all hospitals in that database. "Discharge bias" is the difference between 30-day mortality and in-hospital mortality rates. Not surprisingly, more critical care patients are found to die in larger than smaller hospitals for many recognized and unrecognized biologic and financial administrative circumstances and protocols. As the report demonstrates a statistical difference in discharge bias between larger and smaller hospitals with greater in-hospital mortality after 30 days, large hospital administrators undoubtedly are concerned that their hospitals will receive decreased value-based payments for all ICU admissions. The Pennsylvania statistical findings are of clinical health policy significance for critical care. However, the opacity of this finding should not dispel the important likelihood that patients not discharged from ICU within the first 30 days who linger before hospital discharge defines a cohort of patients who have been extremely ill with poor survival prognosis. It is time for future research with the major prognostic scoring systems such as Acute Physiology and Chronic Health www.ccmjournal.org
1285
Copyright of Critical Care Medicine is the property of Lippincott Williams & Wilkins and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
S
everity-of-illness scores have been used for over 30 years in critical care for a variety of roles, including case-mix adjustment in research, benchmarking, performance improvement, resource utilization, and clinical decision support (1). A variety of approaches have been used in the development of these scores, but all have made use of large databases of clinical data for the derivation and validation of the score. The performance of the scoring system may be affected by the population of patients used in its derivation, with variability occurring in the geographic region, the type of ICU, and the diagnostic case-mix of patients. Obstetric patients account for only about 2.5% of ICU patients (2), and therefore, the validity of these scoring systems to the pregnant or postpartum patient is in question. Several studies from different geographic regions have evaluated Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score (SAPS) II in the obstetric patient, demonstrating good discrimination between survivors and nonsurvivors but overestimating mortality, with significant variation between studies (3-6). Many of these scoring systems use diagnosis-specific weighting systems for the calculation of predicted mortality— unfortunately such weighting is not available for common obstetric diagnoses. The physiological changes in the pregnant woman may result in alterations in the normal range of values for several physiological and laboratory variables, skewing the points allocated to a score. However, a study that corrected for this altered physiology did not demonstrate an improvement in the operating characteristics of APACHE II or SAPS II (7). The Mortality Prediction Model (MPM) scoring system calculates a score at ICU admission (MPM^), unlike APACHE and SAPS which use data from the first 24 hours in the ICU. The MPM-II and the recalibrated MPM-III scores are simple and easy to apply, using a relatively small number of variables which are easily obtainable at ICU admission (8,9). Evaluation of the MPMij-III score in six diagnostic subgroups (not including obstetric patients, but evaluating cardiovascular, trauma, medical, elective surgery, neurosurgery, and emergency surgery)
'See also p. 1047. Key Words: oritioal care; pregnancy complications; severity of illness index; triage The author has disclosed that he does not have any potential conflicts of interest. Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins
identified excellent performance in each of these subgroups (10). Few studies have evaluated the MPM scoring system in the obstetric patient. Using a small database, el-Solh and Grant (3) found the MPM-II score to accurately predict mortality in critically ill obstetric patients. MPM has potential advantages as an obstetric scoring system, in that its performance appears to be less dependent on the diagnostic subgroup, and it relies less on physiological and laboratory values which may be altered by pregnancy. In this issue of Critical Care Medicine, Rojas-Suarez et al ( 11 ) contribute significantly to this field with a study evaluating several scoring systems in a relatively large database of obstetric patients admitted to their ICU in Cartagena, Colombia. Their single-center study identified 726 obstetric patients admitted to the ICU over a 5-year period, with 22 obstetric-related deaths and nine indirect deaths (overall mortality 4.27%). SAPS-2, SAPS-3, MPM^-II, and MPMj-III were evaluated. Each showed good discrimination, with an area under the receiver-operating characteristic (ROC) curve of greater than 0.86. Calibration, that is, the relationship between the actual and predicted mortality was good only for the MPMjj-II score, with a mortality ratio of 0.88 and supportive Hosmer-Lemeshow statistic. The SAPS scores significantly overestimated mortality and MPM^-III slightly underestimated mortality. However, the results generated by this cohort may not be generalizable to all ICUs that admit obstetric patients— their ICU admission rate of 14.2 per 1,000 deliveries is higher than the median 2.7 per 1,000 deliveries reported in a systematic review of obstetric ICU admissions (2). The Colombian cohort also differed from the cohorts in this review in the relatively high percentage of pregnant patients on ICU admission (25% vs median 16%), and the low requirement for mechanical ventilation (25% vs median 41%) (2). ICU scoring systems play a number of valuable roles, but in obstetric management, they have also been applied to the evaluation and management of the individual patient. Scoring systems for obstetric-specific conditions have been developed and validated. The Pre-eclampsia Integrated Estimate of Risk score has been shown to allow the early identification of women at risk of complications of preeclampsia, permitting modification of management (12). A Maternal Critical Care Working Group of the Royal College of Obstetricians and Gynaecologists in 2011 recommended monitoring ill pregnant and postpartum women using an Early Warning Score that had been validated in pregnancy (13). In a recent publication using the British Intensive Care National Audit and Research Centre dataset. Carle et al (14) describe the evaluation of several preexisting obstetric early warning scores and the development and validation of a new obstetric score and demonstrate excellent discrimination for this new score (area under ROC curve of 0.995). These scores are able to accurately identify patients at high risk of mortality and are therefore of value in patient management and triage. They are not developed to generate a predicted mortality rate. Other
DOI:10.1097/CCM.0000000000000170 May 2014 • Volume 42 • Number 5 1284
www,ccmjournal,org
Editorials
international efforts are underway to develop a general obstetric severity-of-illness score (15). The very good discrimination and calibration obtained for MPM^^-II by Rojas-Suarez et al (11) suggests that this is a score against which new systems should be compared.
REFERENCES 1. Keegan MT, Gajic O, Afessa B: Severity of iiiness scoring sysfems in the intensive care unit. Crit Care Med 2011 ; 39:163-1 69 2. Poiiock W, Rose L, Dennis CL; Pregnant and postpartum admissions to the intensive care unit: A systematic review. Intensive Care Med 2010; 36:1465-1474 3. el-Soih AA, Grant BJ: A comparison of severity of illness scoring systems for critically iii obstetric patients. Chest ^9Q6; 110:1299-1304 4. Hazelgrove JF, Price C, Pappachan VJ, et al: Multicenter study of obstetric admissions to 14 intensive care units in southern England. Crit Care Med 2001 ; 29:770-775 5. Vasquez DN, Estenssoro E, Canales HS, et al: Ciinical characteristics and outcomes of obsfetric patients requiring ICU admission. Chest 2007; 131:718-724 6. Tempe A, Wadhwa L, Cupta S, et al: Prediction of mortality and morbidity by simplified acute physiology score II in obstetric intensive care unit admissions. Indian J Med Sei 2007; 61:179-185 7. Lapinsky SE, Hallett D, Collop N, ef al; Evaiuation of sfandard and modified severity of illness scores in the obstetric patient. J Crit Care 2011; 26:535.e1-535.e7
8. Lemeshow S, Teres D, Klar J, et al: Morfaiify Probability Models (MPM II) based on an international cohort of intensive care unif patients. JAMA 1993; 270:2478-2486 9. Higgins TL, Kramer AA, Nathanson BH, et al: Prospective vaiidation of fhe infensive care unit admission Mortaiity Probability Model (MPMO-III). Crit Care Med 2009; 37:1619-1623 10. Nathanson BH, Higgins TL, Kramer AA, et al: Subgroup mortality probability models: Are fhey necessary for specialized intensive care units? Crit Care Med 2009; 37:2375-2386 11. Rojas-Suarez J, Pafernin-Caicedo A, Miranda J, ef ai: Comparison of Severity-of-lliness Scores in Criticaiiy III Cbstetric Patients: A 6-Year Retrospective Cohort. Crit Care Med 2014; 42:1047-1054 1 2. von Dadelszen P, Payne B, Li J, et al; PIERS Study Group; Prediction of adverse mafernal outcomes in pre-eclampsia; Developmenf and validation of fhe fullPIERS model. Lancet 2011 ; 377;219-227 13. Mafernai Crificai Care Working Group: Providing Equity of Critical and Maternity Care for the Critically III Pregnant or Recently Pregnant Woman. London, Royal Coilege of Cbstetricians and Gynecologists, 2011. Avaiiabie at: http://vvww.rcog.org.uk/files/rcog-corp/Prov_Eq_ MatandCritCare.pdf. Accessed November 6, 2013 14. Carle C, Alexander P, Coiumb M, et ai: Design and internal validation of an obstetric early warning score: Secondary analysis of fhe Infensive Care Nafional Audit and Research Centre Case Mix Programme database. Anaesthesia 2013; 68:354-367 15. Canadian Research Informafion System; Ciinical prediction models for criticaiiy ill pregnant women: MEOWSandCIPHER.Availableat: http:// webapps.cihr-irsc.gc.ca/cris/detail_e?pResearchld=4339831&p_ version=CRIS&pJanguage=E&p_sessionJd=1333597 Accessed November 6, 2013
ICU Discharge Bias Reveáis Etiiicaiiy Troubiing Pay-for-Performance Benciimark iVietrics* Michael Rie, MD Department of Anesthesia University of Kentucky College of Medicine Lexington, KY
H
ealthcare payment reform is transitioning from fee-for-service payments to an outcomes-driven value-based system. Critical care health services research now confronts "pay-for-performance criteria" from Medicare and insurance funds. In this issue of Critical Care Medicine, Reineck et al ( 1 ) inform us that in-hospital mortality within 30 days or readmission to hospital within 30 days after discharge presently defines substandard quality payment benchmarks
'See also p. 1055. Key Words: intensive care unit discharge bias; mortaiity; nafionai quaiity forum; patient aufonomy; pay-for-performance benchmark mefrics; prognostic scoring systems; rationing intensive oare unit care Dr. Rie served as a board member for Physicians Againsf Drug Shorfages (Patient Advocacy Group) and received support for article preparation from New York Times Cp-ED. Copyright ® 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wiikins
Critical Care Medicine
for Medicare and the National Quality Eorum (NQF) (2). These Pennsylvania investigators previously reported (3) using the same governmental database of the Pennsylvania Health Care Cost Containment Council capturing all nongovernmental hospital admissions. They now extend their research to "discharge bias" across all hospitals in that database. "Discharge bias" is the difference between 30-day mortality and in-hospital mortality rates. Not surprisingly, more critical care patients are found to die in larger than smaller hospitals for many recognized and unrecognized biologic and financial administrative circumstances and protocols. As the report demonstrates a statistical difference in discharge bias between larger and smaller hospitals with greater in-hospital mortality after 30 days, large hospital administrators undoubtedly are concerned that their hospitals will receive decreased value-based payments for all ICU admissions. The Pennsylvania statistical findings are of clinical health policy significance for critical care. However, the opacity of this finding should not dispel the important likelihood that patients not discharged from ICU within the first 30 days who linger before hospital discharge defines a cohort of patients who have been extremely ill with poor survival prognosis. It is time for future research with the major prognostic scoring systems such as Acute Physiology and Chronic Health www.ccmjournal.org
1285
Copyright of Critical Care Medicine is the property of Lippincott Williams & Wilkins and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
