Trauma/Critical Care Alternative end points for trauma studies: A survey of academic trauma surgeons Zeinab M. Alawadi, MD,a,b Eric LeFebvre, MD,c Erin E. Fox, PhD,d Deborah J. Del Junco, PhD,d Bryan A. Cotton, MD,a,d Charles E. Wade, PhD,a,d and John B. Holcomb, MD, FACS,a,d Houston, TX
Introduction. Changing the epidemiology of trauma makes traditional end points like 30-day mortality less than ideal. Many alternative end points have been suggested; however, they are not yet accepted by the trauma community or regulatory bodies. This study characterizes opinions about the adequacy of accepted end points of studies of trauma and the appropriateness of several novel end points. Methods. An electronic survey was administered to all members of the American Association for the Surgery of Trauma. Questions involved demographics, research experience, appropriateness of proposed study end points, and the role of nontraditional, surrogate, and composite end points. Results. Response rate was 16% (141 of 873) with 74% of respondents practicing at Level 1 Trauma Centers. The respondents were very experienced, with 81% reporting >10 years of practice at the attending level and 87% actively involved in research. The majority of respondents rated the following end points favorably: 24-hour survival, 30-day survival, and time to control of acute hemorrhage with approval rates of 82%, 78%, and 76%, respectively. Six-hour survival, intensive care unit-free survival, and days free of multiorgan failure were rated as appropriate or very appropriate less than 66% of the time. Only 45% of respondents judged the currently used end points of trauma to be appropriate. More than 80% respondents disagreed or strongly disagreed that there was no role for of surrogate or composite endpoints in research of trauma resuscitation. Conclusion. There is strong interest in finding efficient end points in trauma research that are both specific and reflect the changing epidemiology of trauma death. The alternative end points of 24-hour survival and time to control of acute hemorrhage had similar approval rates to 30-day mortality. (Surgery 2015;158:1291-6.) From the Department of Surgery,a University of Texas Medical School; University of Texas Health Science Center at Houston Center for Surgical Trials and Evidence-based Practice (C-STEP)b; Department of Emergency Medicine,c University of Texas Medical School; and University of Texas Health Science Center at Houston Center for Translational Injury Research (CeTIR)d, Houston, TX
TRAUMA REMAINS THE THIRD MOST COMMON CAUSE OF DEATH across the entire age range, and specifically for individuals between the ages of 1 and 44 years, it is the most common cause of death.1 During the last 30 years, improvements in trauma care have Presented as a quick shot at the Academic Surgical Congress, San Diego, California on February 4, 2014. Accepted for publication March 18, 2015. Reprint requests: Zeinab M. Alawadi, MD, Department of Surgery, University of Texas Medical School, 6431 Fannin Street, MSB 5.242, Houston, TX 77030. E-mail: Zeinab.alawadi@uth. tmc.edu. 0039-6060/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.surg.2015.03.030
decreased the incidence of late trauma deaths, especially those occurring several or more days after injury.2-4 Yet, many deaths still occur soon after trauma, especially from exsanguination. The unexpected and time-sensitive nature of trauma introduces difficulties unique to trauma research. In 2008 and 2009, leaders in trauma met to characterize and develop solutions to challenges to clinical trials in trauma. Cornerstone to this discussion was the necessity and difficulty of selecting good end points for research in trauma resuscitation and the desirability of efficient end points that are both specific and reflect the changing the epidemiology of trauma death.5 Although traditional primary measures of study outcomes like 30-day mortality performed well in SURGERY 1291
1292 Alawadi et al
identifying interventions that decrease late mortality, these same end points might fail to detect incremental improvements in the care of the acutely bleeding patient, adding more cost to enroll more patients and have greater follow-up, which might be challenging in trauma patients.6-10 Suggestions included developing composite and surrogate outcome measures, a technique that has been applied to great effect in other fields of biomedical research.5,10-12 There was also interest in the use of a shorter-term measure of traditional outcome, eg, 7day mortality instead of 30-day mortality. These novel end points, if validated, have the potential to decrease cost, increase statistical power to detect improvements in morbidity, and to more clearly depict the effects of early treatments. Before undertaking any efforts directed at trauma research that use nontraditional end points, it is important to define clearly the proposed outcome measures and to ensure that the results will be considered valid by the trauma community and by regulatory agencies. Providers who do not accept studies as methodologically sound will not change their clinical practice based on the results. Using the discussion of our working group as a starting point, we reviewed the relevant literature and developed definitions and brief rationales for 8 proposed end points for studies in trauma resuscitation. We then undertook this cross-sectional survey of physician-level trauma providers to better characterize the understanding, familiarity, and acceptance by the surveyed group of several proposed surrogate and composite end points of research in trauma resuscitation. METHODS The study was designed as a web-based census survey of members the American Association for the Surgery of Trauma (AAST). Stringent requirements for membership ensure that members are both active in caring for and/or conducting research to benefit trauma patients and have substantial training in trauma. The survey instrument was developed based on existing recommendations.13,14 Three key categories for inquiry were posited: background/practice environment of the provider, adequacy of traditional end points, and appropriateness of composite or surrogate end points for research. For the first and second domains, a substantial list of potential questions was generated. Each investigator then reviewed the potential questions in each category. Group discussions identified the questions perceived to be most relevant. For the third
Surgery November 2015
category, we undertook an extensive review of previous end points of trauma studies and of the use of surrogate and composite end point in biomedical research. A list of candidate end points was created. Each candidate end point was then given an explicit definition and a description of its rationale. Again, extensive literature search and repeated group discussions were used to select the candidate end points that were judged to be most acceptable or promising.15-19 Finally, high-ranking items from all 3 categories were assembled into a survey designed to be completed easily in approximately 10 minutes. Subsequently, we pilot tested the survey with the trauma faculty and fellows at our local institution. Pilot-test responses are not included in final study analysis. The results of the pilot-tests were used in an iterative process until the survey had strong face validity and acceptable completion time. The final survey is available as Appendix A (online version only). After we obtained approval from the local institutional review board and AAST Multi-Institutional Trials Committee, we sent subjects an e-mail with an invitation to participate and a URL address to a online survey hosted by SurveyMonkey (Portland, OR), where they could indicate their consent and complete the questionnaire. Data were logged anonymously, and only one response was allowed per uniquely identified computer. Data collection began on June 3, 2011, with an invitation to participate distributed in the monthly AAST newsletter. Additional reminders were sent as isolated email messages to all subjects inviting them to participate if they had not already done so 2.5, 7, and 13 weeks after the survey opened. Data collection ceased on November 5, 2011. At the time of completion of data collection, a de-identified data set was downloaded, and responses were searched for patterns, such as all questions answered with the same number, indicating deliberate misrepresentation; such surveys were excluded from analysis. Response rate was calculated as the number of complete surveys divided by the number of deliverable e-mail addresses reported by the AAST. General demographics of the surveyed population are reported as counts and proportions. Responses on a Likert scale to the items were grouped into favorable (response levels 5 and 4), indifferent (response level 3), and unfavorable (response levels 2 and 1) and are reported as counts as well as proportions. By consensus, a priori decision was made by the authors that well-accepted end points with approval rates of 75% or greater would take priority in our recommendations. Sensitivity analyses had been planned,
Alawadi et al 1293
Surgery Volume 158, Number 5
but a marginal response rate limited their utility. Comments offered as free responses, when present, were reviewed by 3 study authors (Z.A., E.L., and J.H.) and grouped into semantic categories. Comments that articulated recurring themes or that were judged to be particularly insightful were considered for inclusion in this report. Analysis was conducted in StataIC 11.2 (College Station, TX). RESULTS There were 1,208 active AAST members during the study. The survey was distributed to between 906 and 908 unique e-mail addresses at each of the 4 mailings and delivered successfully to between 871 and 873 addresses. One hundred sixty-four people started the survey, and 141 completed all sections of the survey. Only the completed surveys were included in the analysis, with an overall response rate of 16.2% (141/873). Ninety-eight percent of the study population practiced in the United States, and the majority worked primarily at Level 1 Trauma centers with academic ties (74%) in urban locations (77%) (Table I). Eighty one percent of respondents had practiced at the attending level for more than 10 years. Eighty-seven percent reported being involved actively in research, and 45% had been a principle investigator of a peer-reviewed, federally funded research project. Most (64%) belonged to 3 or more academic societies. The AAST does not collect routinely similar characteristics of its membership; thus, the representativeness of the sample population could not be assessed rigorously. The majority of respondents rated the following end points favorably: 24-hour survival, 30-day survival, and time to control of acute hemorrhage with approval rates of 82%, 78%, and 76%, respectively (Table II). Responses for the total 24hour transfusion requirement and time to clearance of lactic acidosis were more mixed with approval ratings of 70% and 72% respectively. The end points of 6-hour survival, intensive care unit–free survival, days free of multiorgan failure, and ventilator-free days were rated as appropriate or very appropriate less than 66% of the time. Forty-five percent of respondents judged the current outcome measures of trauma resuscitation to be appropriate (Table III). Twenty-seven percent of respondents agreed or strongly agreed that outcome measures of 30 days or greater for investigational resuscitation interventions were the most clinically meaningful (Table III). Sixtyfive percent of respondents indicated concern
Table I. Characteristics of study population Characteristic
n
%
Total respondents 141 100 US 126 89 International 15 11 Trauma center status Level I 107 76 Level II 27 19 Level III 2 1 Not I, II, or III 5 4 Hospital type Academic 105 74 Private 4 3 Government 5 4 Community 27 19 Hospital location Rural 8 6 Urban 109 77 Suburban 24 17 Military 0 0 Practice at attending level, y 0–5 3 2 6–10 24 17 11–15 28 20 15–20 20 14 >20 66 47 Active in research Yes 122 87 No 19 13 Principle investigator in federally funded research Yes 64 45 No 77 55 Membership in academic societies AAST only 7 5 1–2 49 35 3–4 80 57 5+ 5 3 US, United States.
regarding overreliance on 30-day mortality (Table III). Greater than 80% of respondents disagreed or strongly disagreed that there was no role for of surrogate or composite outcome measures in research of trauma resuscitation. (Table III). DISCUSSION This survey of novel, short-term, surrogate, and composite outcome measures for research in trauma resuscitation was undertaken as a continuation of the discussions held at the Center for Translational Injury Research in 2008 and 2009 among trauma experts regarding the ways to increase amount of Level I data related to the care of trauma patients.5 During that meeting, the need for clearly defined outcome measures that addressed the preponderance of early death in
1294 Alawadi et al
Surgery November 2015
Table II. Appropriateness of the proposed end points for trauma resuscitation studies End point 30-d survival 24-h survival 6-h survival Intensive care unit–free survival Ventilator-free days Days free of multiorgan failure Time to clearance of lactic acidosis Total 24-h transfusion requirement Time to control of acute hemorrhage
Appropriate/very appropriate, %
Neutral, %
Inappropriate/very inappropriate, %
78 82 46 54 66 63 72 70 76
7 9 27 27 18 24 17 18 14
16 10 27 19 16 13 11 12 10
Table III. Agreement with end point statements Statement Current outcome measures for trauma resuscitation interventions appropriate 30+ day outcomes most clinically meaningful for resuscitation interventions Overreliance on 30-day morality as the primary end point is a barrier to effective trauma research No role for surrogate outcome measures in research in trauma resuscitation No role for composite outcome measures in research in trauma resuscitation
trauma was addressed. To that end, 873 AAST members were given a web-based survey that focused on the appropriateness and specific definitions of 10 novel end points for studies of early trauma interventions (Appendix A). One hundred forty-one AAST members returned a completed survey, for an overall response rate of 16.2%. Although this low response rate prohibits broad generalization, it should be an adequate number to generate hypotheses and provide some insight into the opinions of the trauma communities about novel and surrogate end points. In addition, as can be seen from demographics of the study population, these providers represent a highly experienced, academic cohort that is heavily involved in research. The 24-hour survival and the time to control of acute hemorrhage were the 2, non-30-day-mortality end points with rates of approval on par with the 78% approval of this well-established end point. For an end point to be considered a surrogate two requirements must be met: the relationship between it and the accepted clinical outcome
Strongly agree/agree, %
Neutral, %
Disagree/strongly disagree, %
45
29
26
27
20
53
65
20
15
7
12
81
5
11
84
must be biologically plausible,10 and there must be evidence from clinical trials that treatment effects on the surrogate outcome correspond to treatment effects on the acceptable clinical outcome.9 Although the second requirement is not met for the majority of these novel end points, and despite that the correlation between short-term surrogate end points and long-term survival is conditional on the survival after surrogate end point is reached, which may not be the case in all trauma patients, it only makes sense that the earlier a patient’s hemorrhaging is controlled, the better the chance of survival. Likewise, if the intervention of interest can prevent early mortality, patient care will be shifted from the field to the hospital, allowing more sophisticated interventions and ultimately, improving patient survival. Despite the fact that the use of surrogate and composite end points is very common in medical literature, some would argue that the use of these alternative end points may produce biased or misleading results.20-22 As an example, patients
Alawadi et al 1295
Surgery Volume 158, Number 5
surviving early, hemorrhage-related mortality might still be at increased risk for late death from multiorgan failure. In contrast, the use of long-term mortality as the primary and only end point when one is assessing intervention taking place early in the course of treatment also may not be appropriate. In addition to the added cost and the large sample size affecting the feasibility of trials, this long-term mortality may be well confounded by several factors, such as the severity and mechanism of injury (blunt vs penetrating), the prognosis of individual patients, the complexity of care and variation in treatment after the initial experimental intervention, the cause of death, and the existence of do-not-resuscitate order.8,23-28 Thus, the use of multiple or composite end points may be warranted. Limitations. Our study has several limitations. Most notable is the generally low response rate. This low response rate is not totally unexpected, given the difficulty of conducting survey research in the medical community,29 but a low response rate does complicate interpretation. With a substantial portion of the AAST members unrepresented, this survey may be subject to response bias in which individuals with strong opinions preferentially respond, making generalization inappropriate. However, we believe that a sample of a wide range of trauma providers belonging to a professional organization and who are involved heavily in research would serve as an adequate representation of trauma providers in the best position to conduct, evaluate, and disseminate outcomes in trauma research. It is critical to note, however, that this survey was designed to elicit acceptance of the novel end points, not evidence-based knowledge. The general response consensus of this survey could support an end point as promising, but this promise may not be born out in trials. Any new end point will require validation. Second, the international community largely was underrepresented in this survey. Although we realize that the opinions and experiences of the international community may be useful, this survey is very United States-centric and may not be applicable to the international community. Other countries with different practice patterns may have different attitudes and opinions toward trauma research and may not face similar barriers to trauma research as faced by our local community. Finally, because there is no clear consensus on the definition of novel end points, we were forced to develop definitions to implement the survey. Hopefully, our pilot testing ensured that our
definitions were clear and had good face validity; however, we may have overlooked promising end points or our definitions may not be the ones that are acceptable to the trauma community at large. In summary, in addition to 30-day mortality, the trauma surgeons who responded to the survey approved of 2 novel outcome measures for trauma resuscitation: 24-hour survival and time to control of acute hemorrhage. Going forward, it is our recommendation that investigators consider including 1 or 2 of the proposed end points with high favorability ratings in large resuscitation studies as a secondary outcome measure to assess validity. SUPPLEMENTARY DATA Supplementary data related to this article can be found online at http://dx.doi.org/10.1016/j.surg.2015. 03.030. REFERENCES 1. Minino AM, Heron MP, Murphy SL, Kochanek KD. Deaths: final data for 2004. Natl Vital Stat Rep 2007;55:1-119. 2. Trunkey DD. Trauma. Accidental and intentional injuries account for more years of life lost in the U.S. than cancer and heart disease. Among the prescribed remedies are improved preventive efforts, speedier surgery and further research. Sci Am 1983;249:28-35. 3. Demetriades D, Kimbrell B, Salim A, Velmahos G, Rhee P, Preston C, et al. Trauma deaths in a mature urban trauma system: is ‘‘trimodal’’ distribution a valid concept? J Am Coll Surg 2005;201:343-8. 4. Gunst M, Ghaemmaghami V, Gruszecki A, Urban J, Frankel H, Shafi S. Changing epidemiology of trauma deaths leads to a bimodal distribution. Proc (Bayl Univ Med Cent) 2010; 23:349-54. 5. Holcomb JB, Weiskopf R, Champion H, Gould SA, Sauer RM, Brasel K, et al. Challenges to effective research in acute trauma resuscitation: consent and endpoints. Shock 2011; 35:107-13. 6. Wade CE, Holcomb JB. Endpoints in clinical trials of fluid resuscitation of patients with traumatic injuries. Transfusion 2005;45(1 Suppl):4S-8S. 7. Gross I, Shander A, Sweeney J. Patient blood management and outcome, too early or not? Best Pract Res Clin Anaesthesiol 2013;27:161-72. 8. Riou B, Landais P, Vivien B, Stell P, Labbene I, Carli P. Distribution of the probability of survival is a strategic issue for randomized trials in critically ill patients. Anesthesiology 2001;95:56-63. 9. Lee EC, Whitehead AL, Jacques RM, Julious SA. The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Med Res Methodol 2014;14:41. 10. Prentice RL. Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med 1989;8:431-40. 11. Hughes MD, Daniels MJ, Fischl MA, Kim S, Schooley RT. CD4 cell count as a surrogate endpoint in HIV clinical trials: a meta-analysis of studies of the AIDS Clinical Trials Group. AIDS 1998;12:1823-32. 12. Ridker PM, Danielson E, Fonseca FA, Genest J, Gotto AM Jr, Kastelein JJ, et al. Rosuvastatin to prevent vascular events in
1296 Alawadi et al
13.
14. 15.
16.
17.
18.
19.
20. 21.
22.
men and women with elevated C-reactive protein. N Engl J Med 2008;359:2195-207. Burns KE, Duffett M, Kho ME, Meade MO, Adhikari NK, Sinuff T, et al. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ 2008;179: 245-52. Passmore C, Dobbie AE, Parchman M, Tysinger J. Guidelines for constructing a survey. Fam Med 2002;34:281-6. Rahbar E, Fox EE, del Junco DJ, Harvin JA, Holcomb JB, Wade CE, et al. Early resuscitation intensity as a surrogate for bleeding severity and early mortality in the PROMMTT study. J Trauma Acute Care Surg 2013;75(1 Suppl 1):S16-23. Martini WZ, Holcomb JB. Acidosis and coagulopathy: the differential effects on fibrinogen synthesis and breakdown in pigs. Ann Surg 2007;246:831-5. Schreiber MA, Perkins J, Kiraly L, Underwood S, Wade C, Holcomb JB. Early predictors of massive transfusion in combat casualties. J Am Coll Surg 2007;205:541-5. Eastridge BJ, Malone D, Holcomb JB. Early predictors of transfusion and mortality after injury: a review of the databased literature. J Trauma 2006;60(6 Suppl):S20-5. Niles SE, McLaughlin DF, Perkins JG, Wade CE, Li Y, Spinella PC, et al. Increased mortality associated with the early coagulopathy of trauma in combat casualties. J Trauma 2008;64:1459-63; discussion 63-5. Garattini S, Bertele V. Ethics in clinical research. J Hepatol 2009;51:792-7. Wang Y, Mogg R, Lunceford J. Evaluating correlation-based metric for surrogate marker qualification within a causal correlation framework. Biometrics 2012;68:617-27. Hochman M, McCormick D. Endpoint selection and relative (versus absolute) risk reporting in published medication trials. J Gen Intern Med 2011;26:1246-52.
Surgery November 2015
23. Vassar MJ, Perry CA, Gannaway WL, Holcroft JW. 7.5% sodium chloride/dextran for resuscitation of trauma patients undergoing helicopter transport. Arch Surg 1991;126:1065-72. 24. Balogh Z, McKinley BA, Holcomb JB, Miller CC, Cocanour CS, Kozar RA, et al. Both primary and secondary abdominal compartment syndrome can be predicted early and are harbingers of multiple organ failure. J Trauma 2003;54:848-59; discussion 59-61. 25. Adams SD, Cotton BA, Wade CE, Kozar RA, Dipasupil E, Podbielski JM, et al. Do not resuscitate status, not age, affects outcomes after injury: an evaluation of 15,227 consecutive trauma patients. J Trauma Acute Care Surg 2013;74:1327-30. 26. Wade CE, del Junco DJ, Fox EE, Cotton BA, Cohen MJ, Muskat P, et al. Do-not-resuscitate orders in trauma patients may bias mortality-based effect estimates: an evaluation using the PROMMTT study. J Trauma Acute Care Surg 2013; 75(1 Suppl 1):S89-96. 27. Adams SD, Cotton BA, McGuire MF, Dipasupil E, Podbielski JM, Zaharia A, et al. Unique pattern of complications in elderly trauma patients at a Level I trauma center. J Trauma Acute Care Surg 2012;72:112-8. 28. Wade CE, del Junco DJ, Holcomb JB, Trauma Outcomes G, Holcomb JB, Wade CE, et al. Variations between level I trauma centers in 24-hour mortality in severely injured patients requiring a massive transfusion. J Trauma 2011; 71(2 Suppl 3):S389-93. 29. Flanigan TS, McFarlane E, Cook S. Conducting survey research among physicians and other medical professionals: a review of the current literature. Available from: http://www. amstat.org/sections/SRMS/Proceedings/y2008/Files/ flanigan.pdf.
Introduction. Changing the epidemiology of trauma makes traditional end points like 30-day mortality less than ideal. Many alternative end points have been suggested; however, they are not yet accepted by the trauma community or regulatory bodies. This study characterizes opinions about the adequacy of accepted end points of studies of trauma and the appropriateness of several novel end points. Methods. An electronic survey was administered to all members of the American Association for the Surgery of Trauma. Questions involved demographics, research experience, appropriateness of proposed study end points, and the role of nontraditional, surrogate, and composite end points. Results. Response rate was 16% (141 of 873) with 74% of respondents practicing at Level 1 Trauma Centers. The respondents were very experienced, with 81% reporting >10 years of practice at the attending level and 87% actively involved in research. The majority of respondents rated the following end points favorably: 24-hour survival, 30-day survival, and time to control of acute hemorrhage with approval rates of 82%, 78%, and 76%, respectively. Six-hour survival, intensive care unit-free survival, and days free of multiorgan failure were rated as appropriate or very appropriate less than 66% of the time. Only 45% of respondents judged the currently used end points of trauma to be appropriate. More than 80% respondents disagreed or strongly disagreed that there was no role for of surrogate or composite endpoints in research of trauma resuscitation. Conclusion. There is strong interest in finding efficient end points in trauma research that are both specific and reflect the changing epidemiology of trauma death. The alternative end points of 24-hour survival and time to control of acute hemorrhage had similar approval rates to 30-day mortality. (Surgery 2015;158:1291-6.) From the Department of Surgery,a University of Texas Medical School; University of Texas Health Science Center at Houston Center for Surgical Trials and Evidence-based Practice (C-STEP)b; Department of Emergency Medicine,c University of Texas Medical School; and University of Texas Health Science Center at Houston Center for Translational Injury Research (CeTIR)d, Houston, TX
TRAUMA REMAINS THE THIRD MOST COMMON CAUSE OF DEATH across the entire age range, and specifically for individuals between the ages of 1 and 44 years, it is the most common cause of death.1 During the last 30 years, improvements in trauma care have Presented as a quick shot at the Academic Surgical Congress, San Diego, California on February 4, 2014. Accepted for publication March 18, 2015. Reprint requests: Zeinab M. Alawadi, MD, Department of Surgery, University of Texas Medical School, 6431 Fannin Street, MSB 5.242, Houston, TX 77030. E-mail: Zeinab.alawadi@uth. tmc.edu. 0039-6060/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.surg.2015.03.030
decreased the incidence of late trauma deaths, especially those occurring several or more days after injury.2-4 Yet, many deaths still occur soon after trauma, especially from exsanguination. The unexpected and time-sensitive nature of trauma introduces difficulties unique to trauma research. In 2008 and 2009, leaders in trauma met to characterize and develop solutions to challenges to clinical trials in trauma. Cornerstone to this discussion was the necessity and difficulty of selecting good end points for research in trauma resuscitation and the desirability of efficient end points that are both specific and reflect the changing the epidemiology of trauma death.5 Although traditional primary measures of study outcomes like 30-day mortality performed well in SURGERY 1291
1292 Alawadi et al
identifying interventions that decrease late mortality, these same end points might fail to detect incremental improvements in the care of the acutely bleeding patient, adding more cost to enroll more patients and have greater follow-up, which might be challenging in trauma patients.6-10 Suggestions included developing composite and surrogate outcome measures, a technique that has been applied to great effect in other fields of biomedical research.5,10-12 There was also interest in the use of a shorter-term measure of traditional outcome, eg, 7day mortality instead of 30-day mortality. These novel end points, if validated, have the potential to decrease cost, increase statistical power to detect improvements in morbidity, and to more clearly depict the effects of early treatments. Before undertaking any efforts directed at trauma research that use nontraditional end points, it is important to define clearly the proposed outcome measures and to ensure that the results will be considered valid by the trauma community and by regulatory agencies. Providers who do not accept studies as methodologically sound will not change their clinical practice based on the results. Using the discussion of our working group as a starting point, we reviewed the relevant literature and developed definitions and brief rationales for 8 proposed end points for studies in trauma resuscitation. We then undertook this cross-sectional survey of physician-level trauma providers to better characterize the understanding, familiarity, and acceptance by the surveyed group of several proposed surrogate and composite end points of research in trauma resuscitation. METHODS The study was designed as a web-based census survey of members the American Association for the Surgery of Trauma (AAST). Stringent requirements for membership ensure that members are both active in caring for and/or conducting research to benefit trauma patients and have substantial training in trauma. The survey instrument was developed based on existing recommendations.13,14 Three key categories for inquiry were posited: background/practice environment of the provider, adequacy of traditional end points, and appropriateness of composite or surrogate end points for research. For the first and second domains, a substantial list of potential questions was generated. Each investigator then reviewed the potential questions in each category. Group discussions identified the questions perceived to be most relevant. For the third
Surgery November 2015
category, we undertook an extensive review of previous end points of trauma studies and of the use of surrogate and composite end point in biomedical research. A list of candidate end points was created. Each candidate end point was then given an explicit definition and a description of its rationale. Again, extensive literature search and repeated group discussions were used to select the candidate end points that were judged to be most acceptable or promising.15-19 Finally, high-ranking items from all 3 categories were assembled into a survey designed to be completed easily in approximately 10 minutes. Subsequently, we pilot tested the survey with the trauma faculty and fellows at our local institution. Pilot-test responses are not included in final study analysis. The results of the pilot-tests were used in an iterative process until the survey had strong face validity and acceptable completion time. The final survey is available as Appendix A (online version only). After we obtained approval from the local institutional review board and AAST Multi-Institutional Trials Committee, we sent subjects an e-mail with an invitation to participate and a URL address to a online survey hosted by SurveyMonkey (Portland, OR), where they could indicate their consent and complete the questionnaire. Data were logged anonymously, and only one response was allowed per uniquely identified computer. Data collection began on June 3, 2011, with an invitation to participate distributed in the monthly AAST newsletter. Additional reminders were sent as isolated email messages to all subjects inviting them to participate if they had not already done so 2.5, 7, and 13 weeks after the survey opened. Data collection ceased on November 5, 2011. At the time of completion of data collection, a de-identified data set was downloaded, and responses were searched for patterns, such as all questions answered with the same number, indicating deliberate misrepresentation; such surveys were excluded from analysis. Response rate was calculated as the number of complete surveys divided by the number of deliverable e-mail addresses reported by the AAST. General demographics of the surveyed population are reported as counts and proportions. Responses on a Likert scale to the items were grouped into favorable (response levels 5 and 4), indifferent (response level 3), and unfavorable (response levels 2 and 1) and are reported as counts as well as proportions. By consensus, a priori decision was made by the authors that well-accepted end points with approval rates of 75% or greater would take priority in our recommendations. Sensitivity analyses had been planned,
Alawadi et al 1293
Surgery Volume 158, Number 5
but a marginal response rate limited their utility. Comments offered as free responses, when present, were reviewed by 3 study authors (Z.A., E.L., and J.H.) and grouped into semantic categories. Comments that articulated recurring themes or that were judged to be particularly insightful were considered for inclusion in this report. Analysis was conducted in StataIC 11.2 (College Station, TX). RESULTS There were 1,208 active AAST members during the study. The survey was distributed to between 906 and 908 unique e-mail addresses at each of the 4 mailings and delivered successfully to between 871 and 873 addresses. One hundred sixty-four people started the survey, and 141 completed all sections of the survey. Only the completed surveys were included in the analysis, with an overall response rate of 16.2% (141/873). Ninety-eight percent of the study population practiced in the United States, and the majority worked primarily at Level 1 Trauma centers with academic ties (74%) in urban locations (77%) (Table I). Eighty one percent of respondents had practiced at the attending level for more than 10 years. Eighty-seven percent reported being involved actively in research, and 45% had been a principle investigator of a peer-reviewed, federally funded research project. Most (64%) belonged to 3 or more academic societies. The AAST does not collect routinely similar characteristics of its membership; thus, the representativeness of the sample population could not be assessed rigorously. The majority of respondents rated the following end points favorably: 24-hour survival, 30-day survival, and time to control of acute hemorrhage with approval rates of 82%, 78%, and 76%, respectively (Table II). Responses for the total 24hour transfusion requirement and time to clearance of lactic acidosis were more mixed with approval ratings of 70% and 72% respectively. The end points of 6-hour survival, intensive care unit–free survival, days free of multiorgan failure, and ventilator-free days were rated as appropriate or very appropriate less than 66% of the time. Forty-five percent of respondents judged the current outcome measures of trauma resuscitation to be appropriate (Table III). Twenty-seven percent of respondents agreed or strongly agreed that outcome measures of 30 days or greater for investigational resuscitation interventions were the most clinically meaningful (Table III). Sixtyfive percent of respondents indicated concern
Table I. Characteristics of study population Characteristic
n
%
Total respondents 141 100 US 126 89 International 15 11 Trauma center status Level I 107 76 Level II 27 19 Level III 2 1 Not I, II, or III 5 4 Hospital type Academic 105 74 Private 4 3 Government 5 4 Community 27 19 Hospital location Rural 8 6 Urban 109 77 Suburban 24 17 Military 0 0 Practice at attending level, y 0–5 3 2 6–10 24 17 11–15 28 20 15–20 20 14 >20 66 47 Active in research Yes 122 87 No 19 13 Principle investigator in federally funded research Yes 64 45 No 77 55 Membership in academic societies AAST only 7 5 1–2 49 35 3–4 80 57 5+ 5 3 US, United States.
regarding overreliance on 30-day mortality (Table III). Greater than 80% of respondents disagreed or strongly disagreed that there was no role for of surrogate or composite outcome measures in research of trauma resuscitation. (Table III). DISCUSSION This survey of novel, short-term, surrogate, and composite outcome measures for research in trauma resuscitation was undertaken as a continuation of the discussions held at the Center for Translational Injury Research in 2008 and 2009 among trauma experts regarding the ways to increase amount of Level I data related to the care of trauma patients.5 During that meeting, the need for clearly defined outcome measures that addressed the preponderance of early death in
1294 Alawadi et al
Surgery November 2015
Table II. Appropriateness of the proposed end points for trauma resuscitation studies End point 30-d survival 24-h survival 6-h survival Intensive care unit–free survival Ventilator-free days Days free of multiorgan failure Time to clearance of lactic acidosis Total 24-h transfusion requirement Time to control of acute hemorrhage
Appropriate/very appropriate, %
Neutral, %
Inappropriate/very inappropriate, %
78 82 46 54 66 63 72 70 76
7 9 27 27 18 24 17 18 14
16 10 27 19 16 13 11 12 10
Table III. Agreement with end point statements Statement Current outcome measures for trauma resuscitation interventions appropriate 30+ day outcomes most clinically meaningful for resuscitation interventions Overreliance on 30-day morality as the primary end point is a barrier to effective trauma research No role for surrogate outcome measures in research in trauma resuscitation No role for composite outcome measures in research in trauma resuscitation
trauma was addressed. To that end, 873 AAST members were given a web-based survey that focused on the appropriateness and specific definitions of 10 novel end points for studies of early trauma interventions (Appendix A). One hundred forty-one AAST members returned a completed survey, for an overall response rate of 16.2%. Although this low response rate prohibits broad generalization, it should be an adequate number to generate hypotheses and provide some insight into the opinions of the trauma communities about novel and surrogate end points. In addition, as can be seen from demographics of the study population, these providers represent a highly experienced, academic cohort that is heavily involved in research. The 24-hour survival and the time to control of acute hemorrhage were the 2, non-30-day-mortality end points with rates of approval on par with the 78% approval of this well-established end point. For an end point to be considered a surrogate two requirements must be met: the relationship between it and the accepted clinical outcome
Strongly agree/agree, %
Neutral, %
Disagree/strongly disagree, %
45
29
26
27
20
53
65
20
15
7
12
81
5
11
84
must be biologically plausible,10 and there must be evidence from clinical trials that treatment effects on the surrogate outcome correspond to treatment effects on the acceptable clinical outcome.9 Although the second requirement is not met for the majority of these novel end points, and despite that the correlation between short-term surrogate end points and long-term survival is conditional on the survival after surrogate end point is reached, which may not be the case in all trauma patients, it only makes sense that the earlier a patient’s hemorrhaging is controlled, the better the chance of survival. Likewise, if the intervention of interest can prevent early mortality, patient care will be shifted from the field to the hospital, allowing more sophisticated interventions and ultimately, improving patient survival. Despite the fact that the use of surrogate and composite end points is very common in medical literature, some would argue that the use of these alternative end points may produce biased or misleading results.20-22 As an example, patients
Alawadi et al 1295
Surgery Volume 158, Number 5
surviving early, hemorrhage-related mortality might still be at increased risk for late death from multiorgan failure. In contrast, the use of long-term mortality as the primary and only end point when one is assessing intervention taking place early in the course of treatment also may not be appropriate. In addition to the added cost and the large sample size affecting the feasibility of trials, this long-term mortality may be well confounded by several factors, such as the severity and mechanism of injury (blunt vs penetrating), the prognosis of individual patients, the complexity of care and variation in treatment after the initial experimental intervention, the cause of death, and the existence of do-not-resuscitate order.8,23-28 Thus, the use of multiple or composite end points may be warranted. Limitations. Our study has several limitations. Most notable is the generally low response rate. This low response rate is not totally unexpected, given the difficulty of conducting survey research in the medical community,29 but a low response rate does complicate interpretation. With a substantial portion of the AAST members unrepresented, this survey may be subject to response bias in which individuals with strong opinions preferentially respond, making generalization inappropriate. However, we believe that a sample of a wide range of trauma providers belonging to a professional organization and who are involved heavily in research would serve as an adequate representation of trauma providers in the best position to conduct, evaluate, and disseminate outcomes in trauma research. It is critical to note, however, that this survey was designed to elicit acceptance of the novel end points, not evidence-based knowledge. The general response consensus of this survey could support an end point as promising, but this promise may not be born out in trials. Any new end point will require validation. Second, the international community largely was underrepresented in this survey. Although we realize that the opinions and experiences of the international community may be useful, this survey is very United States-centric and may not be applicable to the international community. Other countries with different practice patterns may have different attitudes and opinions toward trauma research and may not face similar barriers to trauma research as faced by our local community. Finally, because there is no clear consensus on the definition of novel end points, we were forced to develop definitions to implement the survey. Hopefully, our pilot testing ensured that our
definitions were clear and had good face validity; however, we may have overlooked promising end points or our definitions may not be the ones that are acceptable to the trauma community at large. In summary, in addition to 30-day mortality, the trauma surgeons who responded to the survey approved of 2 novel outcome measures for trauma resuscitation: 24-hour survival and time to control of acute hemorrhage. Going forward, it is our recommendation that investigators consider including 1 or 2 of the proposed end points with high favorability ratings in large resuscitation studies as a secondary outcome measure to assess validity. SUPPLEMENTARY DATA Supplementary data related to this article can be found online at http://dx.doi.org/10.1016/j.surg.2015. 03.030. REFERENCES 1. Minino AM, Heron MP, Murphy SL, Kochanek KD. Deaths: final data for 2004. Natl Vital Stat Rep 2007;55:1-119. 2. Trunkey DD. Trauma. Accidental and intentional injuries account for more years of life lost in the U.S. than cancer and heart disease. Among the prescribed remedies are improved preventive efforts, speedier surgery and further research. Sci Am 1983;249:28-35. 3. Demetriades D, Kimbrell B, Salim A, Velmahos G, Rhee P, Preston C, et al. Trauma deaths in a mature urban trauma system: is ‘‘trimodal’’ distribution a valid concept? J Am Coll Surg 2005;201:343-8. 4. Gunst M, Ghaemmaghami V, Gruszecki A, Urban J, Frankel H, Shafi S. Changing epidemiology of trauma deaths leads to a bimodal distribution. Proc (Bayl Univ Med Cent) 2010; 23:349-54. 5. Holcomb JB, Weiskopf R, Champion H, Gould SA, Sauer RM, Brasel K, et al. Challenges to effective research in acute trauma resuscitation: consent and endpoints. Shock 2011; 35:107-13. 6. Wade CE, Holcomb JB. Endpoints in clinical trials of fluid resuscitation of patients with traumatic injuries. Transfusion 2005;45(1 Suppl):4S-8S. 7. Gross I, Shander A, Sweeney J. Patient blood management and outcome, too early or not? Best Pract Res Clin Anaesthesiol 2013;27:161-72. 8. Riou B, Landais P, Vivien B, Stell P, Labbene I, Carli P. Distribution of the probability of survival is a strategic issue for randomized trials in critically ill patients. Anesthesiology 2001;95:56-63. 9. Lee EC, Whitehead AL, Jacques RM, Julious SA. The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Med Res Methodol 2014;14:41. 10. Prentice RL. Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med 1989;8:431-40. 11. Hughes MD, Daniels MJ, Fischl MA, Kim S, Schooley RT. CD4 cell count as a surrogate endpoint in HIV clinical trials: a meta-analysis of studies of the AIDS Clinical Trials Group. AIDS 1998;12:1823-32. 12. Ridker PM, Danielson E, Fonseca FA, Genest J, Gotto AM Jr, Kastelein JJ, et al. Rosuvastatin to prevent vascular events in
1296 Alawadi et al
13.
14. 15.
16.
17.
18.
19.
20. 21.
22.
men and women with elevated C-reactive protein. N Engl J Med 2008;359:2195-207. Burns KE, Duffett M, Kho ME, Meade MO, Adhikari NK, Sinuff T, et al. A guide for the design and conduct of self-administered surveys of clinicians. CMAJ 2008;179: 245-52. Passmore C, Dobbie AE, Parchman M, Tysinger J. Guidelines for constructing a survey. Fam Med 2002;34:281-6. Rahbar E, Fox EE, del Junco DJ, Harvin JA, Holcomb JB, Wade CE, et al. Early resuscitation intensity as a surrogate for bleeding severity and early mortality in the PROMMTT study. J Trauma Acute Care Surg 2013;75(1 Suppl 1):S16-23. Martini WZ, Holcomb JB. Acidosis and coagulopathy: the differential effects on fibrinogen synthesis and breakdown in pigs. Ann Surg 2007;246:831-5. Schreiber MA, Perkins J, Kiraly L, Underwood S, Wade C, Holcomb JB. Early predictors of massive transfusion in combat casualties. J Am Coll Surg 2007;205:541-5. Eastridge BJ, Malone D, Holcomb JB. Early predictors of transfusion and mortality after injury: a review of the databased literature. J Trauma 2006;60(6 Suppl):S20-5. Niles SE, McLaughlin DF, Perkins JG, Wade CE, Li Y, Spinella PC, et al. Increased mortality associated with the early coagulopathy of trauma in combat casualties. J Trauma 2008;64:1459-63; discussion 63-5. Garattini S, Bertele V. Ethics in clinical research. J Hepatol 2009;51:792-7. Wang Y, Mogg R, Lunceford J. Evaluating correlation-based metric for surrogate marker qualification within a causal correlation framework. Biometrics 2012;68:617-27. Hochman M, McCormick D. Endpoint selection and relative (versus absolute) risk reporting in published medication trials. J Gen Intern Med 2011;26:1246-52.
Surgery November 2015
23. Vassar MJ, Perry CA, Gannaway WL, Holcroft JW. 7.5% sodium chloride/dextran for resuscitation of trauma patients undergoing helicopter transport. Arch Surg 1991;126:1065-72. 24. Balogh Z, McKinley BA, Holcomb JB, Miller CC, Cocanour CS, Kozar RA, et al. Both primary and secondary abdominal compartment syndrome can be predicted early and are harbingers of multiple organ failure. J Trauma 2003;54:848-59; discussion 59-61. 25. Adams SD, Cotton BA, Wade CE, Kozar RA, Dipasupil E, Podbielski JM, et al. Do not resuscitate status, not age, affects outcomes after injury: an evaluation of 15,227 consecutive trauma patients. J Trauma Acute Care Surg 2013;74:1327-30. 26. Wade CE, del Junco DJ, Fox EE, Cotton BA, Cohen MJ, Muskat P, et al. Do-not-resuscitate orders in trauma patients may bias mortality-based effect estimates: an evaluation using the PROMMTT study. J Trauma Acute Care Surg 2013; 75(1 Suppl 1):S89-96. 27. Adams SD, Cotton BA, McGuire MF, Dipasupil E, Podbielski JM, Zaharia A, et al. Unique pattern of complications in elderly trauma patients at a Level I trauma center. J Trauma Acute Care Surg 2012;72:112-8. 28. Wade CE, del Junco DJ, Holcomb JB, Trauma Outcomes G, Holcomb JB, Wade CE, et al. Variations between level I trauma centers in 24-hour mortality in severely injured patients requiring a massive transfusion. J Trauma 2011; 71(2 Suppl 3):S389-93. 29. Flanigan TS, McFarlane E, Cook S. Conducting survey research among physicians and other medical professionals: a review of the current literature. Available from: http://www. amstat.org/sections/SRMS/Proceedings/y2008/Files/ flanigan.pdf.
