Editorials
Severe Sepsis Outcomes: How Are We Doing?* How’m I doing?—Edwin Irving Koch Thomas L. Higgins, MD, MBA, MCCM Baystate Medical Center Springfield, MA
P
erformance assessment is an essential prerequisite to performance improvement. The late Ed Koch, as mayor of New York City, often simplified his own performance assessment to four syllables directed at constituents exiting the subway. Knowing how we are doing with sepsis, however, requires a bit more sophistication. Only in the past 2 decades has sepsis taxonomy become standardized, largely due to the efforts of the Society of Critical Care Medicine (SCCM) and the American College of Chest Physicians (1). The Surviving Sepsis Campaign (SSC), launched by the European Society of Intensive Care Medicine, the International Sepsis Forum, and SCCM demonstrated sustained improvement in the quality of sepsis care over a multiyear period beginning in 2005 (2). The temporally associated improvement in outcomes may not necessarily be cause and effect, and the picture is clouded by changes in diagnostic labeling, capacity to provide critical care outside ICUs, and increased clinician awareness prompting earlier intervention (3). Interpreting variability in the prevalence and mortality of severe sepsis is further complicated by patient factors: age, comorbidities, physiologic abnormalities, and resuscitation status all impact outcomes. Traditionally, mortality outcomes have been risk-adjusted using well-established severity of illness models—Acute Physiology and Chronic Health Evaluation (APACHE) (4), mortality probability model (MPM) (5) or Simplified Acute Physiology Score (SAPS) (6). The data collection requirement for these tools, although minimized by *See also p. 1969. Key Words: Acute Physiology and Chronic Health Evaluation; benchmarking; mortality probability model; severe sepsis; severity of illness; Simplified Acute Physiology Score Dr. Higgins consulted for Cerner Corporation (past consultation for Cerner [mortality probability model (MPM) and Acute Physiology and Chronic Health Evaluation (APACHE)]), lectured for Cerner Corporation (speaker at the Cerner Critical Care Forum at Society of Critical Care Medicine [SCCM] January 2014 and in prior years), has stock options with Cerner Corporation (owns 2300 shares of CERN), and is the lead author on the MPM article and has worked with the developers of APACHE and Simplified Acute Physiology Score, all of which are (necessarily) listed in the editorial. He has previously coauthored articles with Lemeshow, one of the authors of the article editorialized, and knows Shorr, Levy, and Dellinger through his involvement in SCCM and the critical care world. Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins DOI: 10.1097/CCM.0000000000000443
2126
www.ccmjournal.org
electronic health records, has prompted fully automated methods of benchmarking outcome in severe sepsis patients using administrative data (7). So, do we need another tool to know how we are doing? In this issue of Critical Care Medicine, Osborn et al (8) provide a sophisticated secondary analysis of the SSC database. Strengths of this study include size (23,438 patients) and geographic diversity (218 hospitals in 18 countries), both important in developing a robust model. All 34 variables are categorical rather than continuous. Two-way interactions between variables were considered. Given data constraints, the authors could not perform a prospective validation on an independent population, but do provide bootstrapping validation, a resampling method that evaluates performance after randomly removing patients with replacement from the original sample. This attention to detail produces a model with reasonable discrimination (area under the receiver operating characteristic curve of 0.736 in the development set) and robust calibration as measured by Hosmer-Lemeshow goodness-of-fit testing. Although some might advocate hierarchical modeling to better account for clustering of patients within hospitals, evidence suggests that the standard logistic regression approach, as used in this study, is efficient and accurate (9). Although the inclusion of process variables (fluid administration, mechanical ventilation, and vasopressors) may be controversial, these goal-directed therapies comprise elements of the SSC. There is, however, risk to including protocol-driven events which may later be shown to have no effect on outcomes (10). The risk-adjustment model applied to an analysis should closely match the characteristics of the population under study. Murphy-Filkins et al (11) long ago demonstrated that artificially limiting the patient mix reduces the applicability of a general model; models may fail when important patient characteristics exceed a critical percentage. Severe sepsis patients have a high mortality risk (33.4% in this study) and likely merit a customized score. The trade-off is that models optimized for a subpopulation will not necessarily be valid for “all-comer” comparisons. Trauma, pediatric intensive care, and cardiac surgery use customized models because of differing prognostic impact of altered physiology in specialized populations. The concept of a sepsis-specific model is not new; Knaus et al (12) in 1993 customized APACHE for evaluating sepsis patients entering phase 2 and 3 drug studies. The Sepsis Severity Score (SSS) will likely find a niche in clinical research and for international quality comparisons in severe sepsis patients, as defined by the SSC (2). The September 2014 • Volume 42 • Number 9
Editorials
discrimination of the SSS is somewhat lower than current versions of APACHE, MPM, and SAPS, likely because age and comorbidities were not available for consideration. Investigational drug studies, however, would typically track these additional variables. External validation would be welcome, especially with simultaneous comparison to existing models, and with specific attention to calibration, which should be superior in a disease-specific model. As the authors have noted, societal demands for transparency create a pressing need to confidently know how we are doing. The SSS is a welcome addition to a rapidly expanding toolbox.
REFERENCES
1. Bone RC, Sibbald WJ, Sprung CL: The ACCP-SCCM consensus conference on sepsis and organ failure. Chest 1992; 101:1481–1483 2. Levy MM, Dellinger RP, Townsend SR, et al; Surviving Sepsis Campaign: The Surviving Sepsis Campaign: Results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med 2010; 38:367–374 3. Angus DC, van der Poll T: Severe sepsis and septic shock. N Engl J Med 2013; 369:840–851 4. Zimmerman JE, Kramer AA, McNair DS, et al: Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients. Crit Care Med 2006; 34:1297–1310
5. Higgins TL, Teres D, Copes WS, et al: Assessing contemporary intensive care unit outcome: An updated Mortality Probability Admission Model (MPM0-III). Crit Care Med 2007; 35:827–835 6. Moreno RP, Metnitz PG, Almeida E, et al; SAPS 3 Investigators: SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005; 31:1345–1355 7. Lagu T, Lindenauer PK, Rothberg MB, et al: Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis. Crit Care Med 2011; 39:2425–2430 8. Osborn TM, Phillips G, Lemeshow S, et al: Sepsis Severity Score: An Internationally Derived Scoring System From the Surviving Sepsis Campaign Database. Crit Care Med 2014; 42:1969–1976 9. Cohen ME, Dimick JB, Bilimoria KY, et al: Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: A comparison of logistic versus hierarchical modeling. J Am Coll Surg 2009; 209:687–693 10. The ProCESS Investigators, Yealy DM, Kellum JA, et al: A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014; 370:1683–1693 11. Murphy-Filkins R, Teres D, Lemeshow S, et al: Effect of changing patient mix on the performance of an intensive care unit severity-ofillness model: How to distinguish a general from a specialty intensive care unit. Crit Care Med 1996; 24:1968–1973 12. Knaus WA, Harrell FE, Fisher CJ Jr, et al: The clinical evaluation of new drugs for sepsis. A prospective study design based on survival analysis. JAMA 1993; 270:1233–1241
Who Decides Who Should Benefit? Allocating Critical Care in the Context of “Futile Treatment”* Jonna D. Clark, MD, MA Pediatric Critical Care Medicine; Treuman Katz Center for Pediatric Bioethics; and Department of Pediatrics University of Washington School of Medicine Seattle Children’s Hospital Seattle, WA
C
ritical care medicine is a finite resource. Although the degree of scarcity is highly variable among different social contexts, national guidelines regarding the fair allocation of this resource are limited. Critical care physicians often allocate resources at the bedside, using a first-come, first-serve approach while attempting to prioritize care for those who potentially benefit to the greatest degree. In most circumstances, these physicianguided decisions are not explicit; rather, they are made implicitly *See also p. 1977. Key Words: access to care; allocation; futile treatment; futility; triage The author has disclosed that she does not have any potential conflicts of interest. Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins DOI: 10.1097/CCM.0000000000000462
Critical Care Medicine
within the context of cultural norms and local standards of care, without considering the impact on society (1–3). In this issue of Critical Care Medicine, Huynh et al (4) attempt to empirically study the impact of providing “futile” medical care on the allocation of critical care resources. By demonstrating an association between the provision of “futile” treatment and the opportunity cost of delayed access to critical care services by patients who would potentially render greater benefit, they attempt to provide justification for forgoing “futile” medical care. While these authors should be commended for their attempt to study this challenging question empirically, the implications of this study need to be interpreted cautiously for two major reasons. First, the definition of “futile” treatment used by these authors requires closer scrutiny (5, 6). The medical literature of past 3 decades contains an intense debate addressing the concept of futility, and ultimately there is no consensus regarding an objective definition (7). In their study, Huynh et al (4) categorize “futile treatment” based on the perception of one physician for a single patient on a single day. Therefore, whether or not a patient is receiving futile treatment depends on the value system and viewpoint of the treating physician. Within this context, a patient may receive “futile treatment” one day, but not the next, even if the patient is receiving the www.ccmjournal.org
2127
Severe Sepsis Outcomes: How Are We Doing?* How’m I doing?—Edwin Irving Koch Thomas L. Higgins, MD, MBA, MCCM Baystate Medical Center Springfield, MA
P
erformance assessment is an essential prerequisite to performance improvement. The late Ed Koch, as mayor of New York City, often simplified his own performance assessment to four syllables directed at constituents exiting the subway. Knowing how we are doing with sepsis, however, requires a bit more sophistication. Only in the past 2 decades has sepsis taxonomy become standardized, largely due to the efforts of the Society of Critical Care Medicine (SCCM) and the American College of Chest Physicians (1). The Surviving Sepsis Campaign (SSC), launched by the European Society of Intensive Care Medicine, the International Sepsis Forum, and SCCM demonstrated sustained improvement in the quality of sepsis care over a multiyear period beginning in 2005 (2). The temporally associated improvement in outcomes may not necessarily be cause and effect, and the picture is clouded by changes in diagnostic labeling, capacity to provide critical care outside ICUs, and increased clinician awareness prompting earlier intervention (3). Interpreting variability in the prevalence and mortality of severe sepsis is further complicated by patient factors: age, comorbidities, physiologic abnormalities, and resuscitation status all impact outcomes. Traditionally, mortality outcomes have been risk-adjusted using well-established severity of illness models—Acute Physiology and Chronic Health Evaluation (APACHE) (4), mortality probability model (MPM) (5) or Simplified Acute Physiology Score (SAPS) (6). The data collection requirement for these tools, although minimized by *See also p. 1969. Key Words: Acute Physiology and Chronic Health Evaluation; benchmarking; mortality probability model; severe sepsis; severity of illness; Simplified Acute Physiology Score Dr. Higgins consulted for Cerner Corporation (past consultation for Cerner [mortality probability model (MPM) and Acute Physiology and Chronic Health Evaluation (APACHE)]), lectured for Cerner Corporation (speaker at the Cerner Critical Care Forum at Society of Critical Care Medicine [SCCM] January 2014 and in prior years), has stock options with Cerner Corporation (owns 2300 shares of CERN), and is the lead author on the MPM article and has worked with the developers of APACHE and Simplified Acute Physiology Score, all of which are (necessarily) listed in the editorial. He has previously coauthored articles with Lemeshow, one of the authors of the article editorialized, and knows Shorr, Levy, and Dellinger through his involvement in SCCM and the critical care world. Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins DOI: 10.1097/CCM.0000000000000443
2126
www.ccmjournal.org
electronic health records, has prompted fully automated methods of benchmarking outcome in severe sepsis patients using administrative data (7). So, do we need another tool to know how we are doing? In this issue of Critical Care Medicine, Osborn et al (8) provide a sophisticated secondary analysis of the SSC database. Strengths of this study include size (23,438 patients) and geographic diversity (218 hospitals in 18 countries), both important in developing a robust model. All 34 variables are categorical rather than continuous. Two-way interactions between variables were considered. Given data constraints, the authors could not perform a prospective validation on an independent population, but do provide bootstrapping validation, a resampling method that evaluates performance after randomly removing patients with replacement from the original sample. This attention to detail produces a model with reasonable discrimination (area under the receiver operating characteristic curve of 0.736 in the development set) and robust calibration as measured by Hosmer-Lemeshow goodness-of-fit testing. Although some might advocate hierarchical modeling to better account for clustering of patients within hospitals, evidence suggests that the standard logistic regression approach, as used in this study, is efficient and accurate (9). Although the inclusion of process variables (fluid administration, mechanical ventilation, and vasopressors) may be controversial, these goal-directed therapies comprise elements of the SSC. There is, however, risk to including protocol-driven events which may later be shown to have no effect on outcomes (10). The risk-adjustment model applied to an analysis should closely match the characteristics of the population under study. Murphy-Filkins et al (11) long ago demonstrated that artificially limiting the patient mix reduces the applicability of a general model; models may fail when important patient characteristics exceed a critical percentage. Severe sepsis patients have a high mortality risk (33.4% in this study) and likely merit a customized score. The trade-off is that models optimized for a subpopulation will not necessarily be valid for “all-comer” comparisons. Trauma, pediatric intensive care, and cardiac surgery use customized models because of differing prognostic impact of altered physiology in specialized populations. The concept of a sepsis-specific model is not new; Knaus et al (12) in 1993 customized APACHE for evaluating sepsis patients entering phase 2 and 3 drug studies. The Sepsis Severity Score (SSS) will likely find a niche in clinical research and for international quality comparisons in severe sepsis patients, as defined by the SSC (2). The September 2014 • Volume 42 • Number 9
Editorials
discrimination of the SSS is somewhat lower than current versions of APACHE, MPM, and SAPS, likely because age and comorbidities were not available for consideration. Investigational drug studies, however, would typically track these additional variables. External validation would be welcome, especially with simultaneous comparison to existing models, and with specific attention to calibration, which should be superior in a disease-specific model. As the authors have noted, societal demands for transparency create a pressing need to confidently know how we are doing. The SSS is a welcome addition to a rapidly expanding toolbox.
REFERENCES
1. Bone RC, Sibbald WJ, Sprung CL: The ACCP-SCCM consensus conference on sepsis and organ failure. Chest 1992; 101:1481–1483 2. Levy MM, Dellinger RP, Townsend SR, et al; Surviving Sepsis Campaign: The Surviving Sepsis Campaign: Results of an international guideline-based performance improvement program targeting severe sepsis. Crit Care Med 2010; 38:367–374 3. Angus DC, van der Poll T: Severe sepsis and septic shock. N Engl J Med 2013; 369:840–851 4. Zimmerman JE, Kramer AA, McNair DS, et al: Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients. Crit Care Med 2006; 34:1297–1310
5. Higgins TL, Teres D, Copes WS, et al: Assessing contemporary intensive care unit outcome: An updated Mortality Probability Admission Model (MPM0-III). Crit Care Med 2007; 35:827–835 6. Moreno RP, Metnitz PG, Almeida E, et al; SAPS 3 Investigators: SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005; 31:1345–1355 7. Lagu T, Lindenauer PK, Rothberg MB, et al: Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis. Crit Care Med 2011; 39:2425–2430 8. Osborn TM, Phillips G, Lemeshow S, et al: Sepsis Severity Score: An Internationally Derived Scoring System From the Surviving Sepsis Campaign Database. Crit Care Med 2014; 42:1969–1976 9. Cohen ME, Dimick JB, Bilimoria KY, et al: Risk adjustment in the American College of Surgeons National Surgical Quality Improvement Program: A comparison of logistic versus hierarchical modeling. J Am Coll Surg 2009; 209:687–693 10. The ProCESS Investigators, Yealy DM, Kellum JA, et al: A randomized trial of protocol-based care for early septic shock. N Engl J Med 2014; 370:1683–1693 11. Murphy-Filkins R, Teres D, Lemeshow S, et al: Effect of changing patient mix on the performance of an intensive care unit severity-ofillness model: How to distinguish a general from a specialty intensive care unit. Crit Care Med 1996; 24:1968–1973 12. Knaus WA, Harrell FE, Fisher CJ Jr, et al: The clinical evaluation of new drugs for sepsis. A prospective study design based on survival analysis. JAMA 1993; 270:1233–1241
Who Decides Who Should Benefit? Allocating Critical Care in the Context of “Futile Treatment”* Jonna D. Clark, MD, MA Pediatric Critical Care Medicine; Treuman Katz Center for Pediatric Bioethics; and Department of Pediatrics University of Washington School of Medicine Seattle Children’s Hospital Seattle, WA
C
ritical care medicine is a finite resource. Although the degree of scarcity is highly variable among different social contexts, national guidelines regarding the fair allocation of this resource are limited. Critical care physicians often allocate resources at the bedside, using a first-come, first-serve approach while attempting to prioritize care for those who potentially benefit to the greatest degree. In most circumstances, these physicianguided decisions are not explicit; rather, they are made implicitly *See also p. 1977. Key Words: access to care; allocation; futile treatment; futility; triage The author has disclosed that she does not have any potential conflicts of interest. Copyright © 2014 by the Society of Critical Care Medicine and Lippincott Williams & Wilkins DOI: 10.1097/CCM.0000000000000462
Critical Care Medicine
within the context of cultural norms and local standards of care, without considering the impact on society (1–3). In this issue of Critical Care Medicine, Huynh et al (4) attempt to empirically study the impact of providing “futile” medical care on the allocation of critical care resources. By demonstrating an association between the provision of “futile” treatment and the opportunity cost of delayed access to critical care services by patients who would potentially render greater benefit, they attempt to provide justification for forgoing “futile” medical care. While these authors should be commended for their attempt to study this challenging question empirically, the implications of this study need to be interpreted cautiously for two major reasons. First, the definition of “futile” treatment used by these authors requires closer scrutiny (5, 6). The medical literature of past 3 decades contains an intense debate addressing the concept of futility, and ultimately there is no consensus regarding an objective definition (7). In their study, Huynh et al (4) categorize “futile treatment” based on the perception of one physician for a single patient on a single day. Therefore, whether or not a patient is receiving futile treatment depends on the value system and viewpoint of the treating physician. Within this context, a patient may receive “futile treatment” one day, but not the next, even if the patient is receiving the www.ccmjournal.org
2127
