E EDITORIAL
Perioperative Cognitive Aids in Anesthesia: What, Who, How, and Why Bother? David M. Gaba, MD
T
wo articles in this issue1,2 concern a very timely topic: the utility, design, and implementation of “cognitive aids.” This term, in the context of dynamic arenas of health care, encompasses a host of physical (or now virtual) items aimed to assist professionals in executing the complex decision making of diagnosis and therapy. Although cognitive aids of various forms have been around for years, the health care community’s and public’s awareness of the issue and interest in such artifacts have been heightened greatly by the publication and popularity of the WHO Surgical Safety checklist3 and the publication of Atul Gawande’s book The Checklist Manifesto: How to Get Things Right.4 A recent New England Journal of Medicine article described the simulation testing of a set of intraoperative emergency checklists,5 which are cognitive aids whose design and use are intended specifically for use during an emergency (as the name implies), and thus are different than routine cognitive aids. The cover story of the Spring 2013 issue of the Anesthesia Patient Safety Foundation Newsletter highlights the topic of such emergency manuals and emphasizes the proliferation of such aids and the growing interest in their adoption and use. Before I start my commentary, I wish to acknowledge that in many different ways I am not a neutral observer to the issue of cognitive aids. The Patient Safety Section Editor of Anesthesia & Analgesia invited me to write this editorial even after I reminded him of all of the disclosures that I make below. The 2 articles in this issue are complementary in content and scope, although both do focus on the emergency manual types of cognitive aids. The article by Marshall1 is a review of the published empirical testing of cognitive aid use in anesthesia, combined with a suggestion to consider such aids to be analogous to medical devices in terms of advisable processes for their design, testing, and preparation for their use. The other article, by From the Center for Immersive and Simulation-based Learning and Department of Anesthesia, Stanford University School of Medicine; and Anesthesiology and Perioperative Care Service, VA Palo Alto Health Care System, Palo Alto, California. Accepted for publication July 5, 2013. Funding: There was no funding for this work. Conflict of Interest: See Disclosures at the end of the article. Reprints will not be available from the author. Address correspondence to David M. Gaba, MD, Department of Anesthesia, Stanford University, Mail Code 5588, 3801 Miranda Ave., Stanford, CA 943055588. Address e-mail to [email protected]. Copyright © 2013 International Anesthesia Research Society DOI: 10.1213/ANE.0b013e3182a571e3
November 2013 • Volume 117 • Number 5
Goldhaber-Fiebert and Howard,2 is a review of the concept of cognitive aids, concentrating on emergency manuals, and providing a case study of the Stanford implementation experience. The authors of both articles review some of the confusing terminology in this arena. As in many developing fields, terms are overlapping and often are used by others in ways differing from the intent of their original creators or disseminators. I would like to point out that my own introduction of the term “cognitive aids” to this field was an adaptation from the term “cognitive artifact” first applied by Don Norman, PhD, in the fields of cognitive psychology and human factors engineering.6 Commonly used terms for written cognitive aids include “checklists,” “manuals,” “handbooks,” and “reference cards.” The term cognitive aid or cognitive artifact also includes use of calculators to aid computation and the use of written or verbal mnemonics. Historically, there are many pejorative terms for such aids, such as “cheat sheets,” “crib sheets,” “crutches,” and “cookbooks.” While in the past the use of these aids was viewed as a sign of weakness and lack of intelligence, many of us strive to emphasize to students, trainees, and experienced clinicians that their use is actually a sign of strength and wisdom and that failing to use them is a sign of weakness and perhaps unwarranted hubris. The field is unlikely to see a definitive shakeout of terminology anytime soon, and I think that any attempts to force a definitive taxonomy could be counterproductive. As long as clinicians and teams in practice have clear terminology in their local environment, and as long as journal article readers can tell by clear description, rather than just by a word, what a particular cognitive aid is, the exact nomenclature may not be very important. It seems obvious that many different types of aids can be used within a single case. Consider, for example, the use of the following: a verbal or written mnemonic for the elements of anesthesia operating room setup (e.g., MSMAID = “machine, suction, monitors, airway, IV, drugs”); the World Health Organization presurgery safety checklist; a preanesthetic-induction checklist; a perioperative emergency manual (if needed in the case); an electronic calculator to compute drug doses; and (in a rare event) the hotline of the Malignant Hyperthermia Association of the United States. Again, however, the main focus of the 2 articles in this issue is on perioperative emergency manuals. Marshall1 points out that in the reviewed articles describing empirical testing of cognitive aids, their development history is either unknown or not well described. Indeed, this may reflect the fact that even today, with the growing www.anesthesia-analgesia.org 1033
E Editorial availability of aids published and disseminated in various fashions, in the United States, only a few such aids are in widespread use. These are the Malignant Hyperthermia Association of the United States malignant hyperthermia cognitive aids (including a poster, a pocket card, hotline mnemonic, etc.), which are believed to have wide inclusion in special malignant hyperthermia carts, and the American Heart Association’s Advanced Cardiac Life Support small size booklet that appears on many institution’s crash carts. A emergency response cognitive aid for local anesthetic systemic toxicity, promulgated by the American Society for Regional Anesthesia,7 is also gaining in popularity. It is perhaps no surprise that these particular aids were developed through formal consensus processes. Marshall1 points out that in Australia, the Australian Patient Safety Foundation perioperative crisis management manual8 has an extensively documented development history including 7 national consensus meetings, corroborated by review of thousands of incident reports in a national database.9 However, the bulk of emergency manuals use local experts who typically draw from existing national guidelines and from their own literature reviews. With several decades passing since the introduction of these kinds of aids, is it surprising that no national body in the United States and many other countries, and no international body, has attempted to create a more comprehensive set of perioperative cognitive aids through formal consensus approaches? One can only conjecture that the task is daunting logistically, politically, and financially. Hence, individual groups or institutions have taken this bull by the horns and developed their own aids. In the Stanford case, while the authors sought evidence from the literature for the articulation of how to recognize or treat the many events in the original Catalog of Critical Incidents appearing first in 1990 in the Anesthesia Crisis Resource Management course syllabus, and then in the subsequent book Crisis Management in Anesthesiology,10 there were no definitive reviews or guidelines available for a host of events that can and do occur during perioperative anesthesia care. It is likely that many uncommon events will never be studied thoroughly and hence that high levels of evidence will be lacking about their management. Large official bodies may be loath to provide guidelines or emergency manuals for the many situations where the evidence base is thin. However, clinicians in the field need reasonable guidance for what they should do when such events do occur. Creating such emergency manuals must balance several equally important needs. These are articulated in different ways in the 2 articles. The content of the advice must be reasonable and must apply the best evidence known where it exists. At the same time, advice must be presented for the suite of events whose management is critical even when unequivocal evidence is lacking. Considerations of human factors engineering are crucial for optimizing the design of the aids, and for how they are disseminated, implemented, and used. Many kinds of guidelines and practice parameters that are created through highly rigorous processes have not resulted in materials that can be used effectively by clinicians in the heat of the moment. The Advanced Cardiac Life Support booklet is a good example. It summarizes literature and recommendations thoroughly, but experience shows
1034 www.anesthesia-analgesia.org
that even its graphical flowcharts are very hard to read and to use. There is no particular reason why national or international bodies could not balance all these needs, much as the individual institutions have, although the various political, organizational, and proprietary interests of different stakeholders will be challenging to harness. Both articles recommend a process of user-centered design that combines substantial input by those familiar with human factors and cognition as well as graphical design. Both favor intensive iterative testing of the evolving aids with actual users, ideally in both simulated cases and actual clinical emergencies. The literature review by Marshall,1 the Stanford Case Study, and articles referenced in both publications demonstrate the utility of simulation testing. But even simulation studies are difficult to accomplish, especially when the key target population must be experienced personnel and not students or trainees. To obtain solid iterative testing for national or international consensus, emergency manuals would be complex, challenging, and expensive. Who is likely to step up to finance such studies?11 Will national clinical and scientific funding agencies see this kind of development as a priority, especially in the foreseeable future of shrinking budgets? It is possible that other stakeholders, such as liability insurers, stand to accrue greater direct benefits from the promulgation, adoption, and effective use of cognitive aids, but historically in health care, they themselves have rarely financed research, development, or implementation of safety interventions. The study of the use and impact of cognitive aids in and after real clinical crises will be equally difficult. Marshall1 and Goldhaber-Fiebert and Howard,2 each reference some of the studies that have been tried, particularly the Australian experience. These suffer from voluntary reporting (so that many cases with or without cognitive aid use are never analyzed); selection-biased reporting (those who use cognitive aids might be more likely to report events, or they might be more safety focused in the first place); inadequate information about what was really wrong with the patient and what exactly was done; and sparse information about how the aid was used, not used, or misused. Thus, it is reasonable to conclude that we may never be able to prove with Level 1a evidence (multiple wellcontrolled randomized trials) that using cognitive aids in managing critical events improves patient outcome. Even if funders could be identified, the studies might need to last, say, a decade and might involve hundreds or thousands of institutions, and tens of thousands of clinicians. Which institutions and clinicians would agree to be randomized to the control group? Is there really clinical equipoise about this issue? Are those individuals who claim equipoise actually a true cross section of the population of perioperative clinicians as a whole? Frankly, I think the answer is no for all these questions. Both articles rightly point out that evidence about cognitive aid creation, dissemination, and practical use is also weak or nonexistent at this point. Experience from the other industries, such as aviation and nuclear power, is impressive in concept and useful as a guide, but even there, Level 1a evidence is lacking. Moreover, exactly how best to translate successes from those industries to the practical aspects
anesthesia & analgesia
Perioperative Cognitive Aids
of optimizing the creation, use, or evaluation of such aids in health care remain uncertain. But absence of evidence is not evidence of absence, and the growing number of groups that have decided to vote with their feet by creating or adopting what they see as the best available set of cognitive aids attests to the desire of clinicians to avoid paralysis by analysis and to do something sensible to close the gap from nothing to something. As delineated in the 2 articles, it is not as simple as finding a suitable emergency manual and placing copies of it in the clinical environment. The process of adoption, placement, training of personnel, and creating and sustaining a culture that emphasizes and rewards their use is necessary but complex. As suggested by Goldhaber-Fiebert and Howard,2 the domain of “implementation science” is applicable both for studying how these processes transpire in different locales and settings, and for providing advice to those wishing to optimize their site’s use of cognitive aids. As hinted by the Australian experience, the use of data from event reporting and analysis systems holds out hope for 1 pathway of gathering at least anecdotal data on the use, successes, and failures or pitfalls of cognitive aids. In Australia, reporting of anesthetic events to the Australian Patient Safety Foundation now is only on an ad hoc basis. In the United States, the new Anesthesia Incident Reporting System operated by the Anesthesia Quality Institute can be a vehicle for collecting such reports (https://www.aqihq. org/airs/airsIntro.aspx). While such data may help drive the evolution of the aids and how best to use them, event reports can never be definitive and will never convince skeptics. Marshall1 suggests that poor aids, or poor use of aids, may distract clinicians and lead to worse performance than without them. That this could occur in some cases seems selfevident because it posits a suboptimal system from the start (poor aids, aids used poorly). Standards for how to properly create “good” aids and how best to prepare clinicians to use them “wisely” might be established for now, based on experience rather than on empirical evidence. How often worse clinical performance will result from the appropriate use of a set of well-designed aids by anesthesia professionals trained to use them is currently unknown. I think that this probability is likely to be very low. Simulation studies might answer this question, but again to acquire data from a real cross section of clinicians will be very challenging, and if worse performance is uncommon, very large studies would be needed to demonstrate its occurrence. Marshall1 also notes that in the Burden study of a “reader” role (who reads the aid to the team and checks on their progress in managing the event),12 there could be a danger implied by the observed results that “total communication” was reduced with a reader in place. However, since the effectiveness of the teams with a reader was much greater than in those without a reader, it seems far more likely (as acknowledged as a possibility by Marshall) that much of the communication during codes is spurious or redundant anyway, and the use of a reader might focus the team’s communication on the issues that are of greatest importance. Further studies could confirm whether the residual communication
November 2013 • Volume 117 • Number 5
is indeed tighter and whether all the critical communications still take place. My opinion is that what we already know about wellcrafted emergency manuals and the use of a reader role (when sufficient number of on-scene personnel allows it) is sufficient justification to warrant widespread adoption. Sites that fail to see value after a fair trial, even if only by anecdotal reports, will be free to abandon them, at least until more evidence has accumulated. Those that quickly find value in their use may well come to that conclusion because they have seen at least 1 life, heart, or brain saved in part by the use of a cognitive aid. There is a saying so good that it appears in various forms in both the Hebrew Talmud (Sanhedrin 4:5) and the Muslim Quran (5:32): “Whoever saves a life, it is as if he has saved the entire world.” If a set of plasticized pages or their equivalent can do that only once, the effort to adopt them and train people in their use should be well worth it.E DISCLOSURES
Name: David M. Gaba, MD. Contribution: This author wrote the manuscript. Attestation: David Gaba approved the final manuscript, and he is the archival author. Conflicts of Interest: First, I am a strong believer in the use of cognitive aids. Both articles reference our textbook Crisis Management in Anesthesiology as an early source of systematic presentation of information on how to recognize and manage a variety of different perioperative events. That book, and some of my prior articles, also includes textual material about the importance of cognitive aids as one of the vital “resources” for “crisis resource management.” The authors of the book receive small royalties on the original edition and on future editions (the 2nd edition is expected in early 2014). Second, Drs. Goldhaber-Fiebert and Howard, authors of the second article in this issue, are close academic associates of mine, Dr. Howard for >20 years. I participated peripherally in the drafting of the Emergency Manual described in their article. The Stanford Anesthesia Cognitive Aid Group, of which I am a member, collectively decided to provide the Stanford Emergency Manual to the world for free, in PDF form under a Creative Commons License, so as to maximize its availability to interested parties. Hence, we derive no financial gain from the dissemination of the Manual. This manuscript was handled by: Sorin J. Brull, MD, FCARCSI (Hon). REFERENCES 1. Marshall S. The use of cognitive aids during emergencies in anesthesia: A review of the literature. Anesth Analg 2013;117:1162–71 2. Goldhaber-Fiebert S, Howard S. Implementing emergency manuals: can cognitive aids help translate best practices for patient care during acute events. Anesth Analg 2013;117:1149–61 3. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AA; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9 4. Gawande A. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books, Henry Holt and Company, LLC, 2009 5. Arriaga AF, Bader AM, Wong JM, Lipsitz SR, Berry WR, Ziewacz JE, Hepner DL, Boorman DJ, Pozner CN, Smink DS,
www.anesthesia-analgesia.org 1035
E Editorial Gawande AA. Simulation-based trial of surgical-crisis checklists. N Engl J Med 2013;368:246–53 6. Norman D. Cognitive Artifacts, Designing Interaction: Psychology at the Human-Computer Interface. In: Carroll J. Cambridge, ed. Cambridge: Cambridge University Press, 1991:17–38 7. Neal JM, Bernards CM, Butterworth JF 4th, Di Gregorio G, Drasner K, Hejtmanek MR, Mulroy MF, Rosenquist RW, Weinberg GL. ASRA practice advisory on local anesthetic systemic toxicity. Reg Anesth Pain Med 2010;35:152–61 8. Australian Patient Safety Foundation. APSF Crisis Management Manual: “COVER ABCD A SWIFT CHECK” Adelaide, Australia: Australian Patient Safety Foundation, 1996
1036 www.anesthesia-analgesia.org
9. Runciman WB, Kluger MT, Morris RW, Paix AD, Watterson LM, Webb RK. Crisis management during anaesthesia: the development of an anaesthetic crisis management manual. Qual Saf Health Care 2005;14:e1 10. Gaba DM, Fish KJ, Howard SK. Crisis Management in Anesthesiology. New York, NY: Churchill Livingstone, 1994 11. Gaba DM. The pharmaceutical analogy for simulation: a policy perspective. Simul Healthc 2010;5:5–7 12. Burden AR, Carr ZJ, Staman GW, Littman JJ, Torjman MC. Does every code need a “reader?” improvement of rare event management with a cognitive aid “reader” during a simulated emergency: a pilot study. Simul Healthc 2012;7:1–9
anesthesia & analgesia
Perioperative Cognitive Aids in Anesthesia: What, Who, How, and Why Bother? David M. Gaba, MD
T
wo articles in this issue1,2 concern a very timely topic: the utility, design, and implementation of “cognitive aids.” This term, in the context of dynamic arenas of health care, encompasses a host of physical (or now virtual) items aimed to assist professionals in executing the complex decision making of diagnosis and therapy. Although cognitive aids of various forms have been around for years, the health care community’s and public’s awareness of the issue and interest in such artifacts have been heightened greatly by the publication and popularity of the WHO Surgical Safety checklist3 and the publication of Atul Gawande’s book The Checklist Manifesto: How to Get Things Right.4 A recent New England Journal of Medicine article described the simulation testing of a set of intraoperative emergency checklists,5 which are cognitive aids whose design and use are intended specifically for use during an emergency (as the name implies), and thus are different than routine cognitive aids. The cover story of the Spring 2013 issue of the Anesthesia Patient Safety Foundation Newsletter highlights the topic of such emergency manuals and emphasizes the proliferation of such aids and the growing interest in their adoption and use. Before I start my commentary, I wish to acknowledge that in many different ways I am not a neutral observer to the issue of cognitive aids. The Patient Safety Section Editor of Anesthesia & Analgesia invited me to write this editorial even after I reminded him of all of the disclosures that I make below. The 2 articles in this issue are complementary in content and scope, although both do focus on the emergency manual types of cognitive aids. The article by Marshall1 is a review of the published empirical testing of cognitive aid use in anesthesia, combined with a suggestion to consider such aids to be analogous to medical devices in terms of advisable processes for their design, testing, and preparation for their use. The other article, by From the Center for Immersive and Simulation-based Learning and Department of Anesthesia, Stanford University School of Medicine; and Anesthesiology and Perioperative Care Service, VA Palo Alto Health Care System, Palo Alto, California. Accepted for publication July 5, 2013. Funding: There was no funding for this work. Conflict of Interest: See Disclosures at the end of the article. Reprints will not be available from the author. Address correspondence to David M. Gaba, MD, Department of Anesthesia, Stanford University, Mail Code 5588, 3801 Miranda Ave., Stanford, CA 943055588. Address e-mail to [email protected]. Copyright © 2013 International Anesthesia Research Society DOI: 10.1213/ANE.0b013e3182a571e3
November 2013 • Volume 117 • Number 5
Goldhaber-Fiebert and Howard,2 is a review of the concept of cognitive aids, concentrating on emergency manuals, and providing a case study of the Stanford implementation experience. The authors of both articles review some of the confusing terminology in this arena. As in many developing fields, terms are overlapping and often are used by others in ways differing from the intent of their original creators or disseminators. I would like to point out that my own introduction of the term “cognitive aids” to this field was an adaptation from the term “cognitive artifact” first applied by Don Norman, PhD, in the fields of cognitive psychology and human factors engineering.6 Commonly used terms for written cognitive aids include “checklists,” “manuals,” “handbooks,” and “reference cards.” The term cognitive aid or cognitive artifact also includes use of calculators to aid computation and the use of written or verbal mnemonics. Historically, there are many pejorative terms for such aids, such as “cheat sheets,” “crib sheets,” “crutches,” and “cookbooks.” While in the past the use of these aids was viewed as a sign of weakness and lack of intelligence, many of us strive to emphasize to students, trainees, and experienced clinicians that their use is actually a sign of strength and wisdom and that failing to use them is a sign of weakness and perhaps unwarranted hubris. The field is unlikely to see a definitive shakeout of terminology anytime soon, and I think that any attempts to force a definitive taxonomy could be counterproductive. As long as clinicians and teams in practice have clear terminology in their local environment, and as long as journal article readers can tell by clear description, rather than just by a word, what a particular cognitive aid is, the exact nomenclature may not be very important. It seems obvious that many different types of aids can be used within a single case. Consider, for example, the use of the following: a verbal or written mnemonic for the elements of anesthesia operating room setup (e.g., MSMAID = “machine, suction, monitors, airway, IV, drugs”); the World Health Organization presurgery safety checklist; a preanesthetic-induction checklist; a perioperative emergency manual (if needed in the case); an electronic calculator to compute drug doses; and (in a rare event) the hotline of the Malignant Hyperthermia Association of the United States. Again, however, the main focus of the 2 articles in this issue is on perioperative emergency manuals. Marshall1 points out that in the reviewed articles describing empirical testing of cognitive aids, their development history is either unknown or not well described. Indeed, this may reflect the fact that even today, with the growing www.anesthesia-analgesia.org 1033
E Editorial availability of aids published and disseminated in various fashions, in the United States, only a few such aids are in widespread use. These are the Malignant Hyperthermia Association of the United States malignant hyperthermia cognitive aids (including a poster, a pocket card, hotline mnemonic, etc.), which are believed to have wide inclusion in special malignant hyperthermia carts, and the American Heart Association’s Advanced Cardiac Life Support small size booklet that appears on many institution’s crash carts. A emergency response cognitive aid for local anesthetic systemic toxicity, promulgated by the American Society for Regional Anesthesia,7 is also gaining in popularity. It is perhaps no surprise that these particular aids were developed through formal consensus processes. Marshall1 points out that in Australia, the Australian Patient Safety Foundation perioperative crisis management manual8 has an extensively documented development history including 7 national consensus meetings, corroborated by review of thousands of incident reports in a national database.9 However, the bulk of emergency manuals use local experts who typically draw from existing national guidelines and from their own literature reviews. With several decades passing since the introduction of these kinds of aids, is it surprising that no national body in the United States and many other countries, and no international body, has attempted to create a more comprehensive set of perioperative cognitive aids through formal consensus approaches? One can only conjecture that the task is daunting logistically, politically, and financially. Hence, individual groups or institutions have taken this bull by the horns and developed their own aids. In the Stanford case, while the authors sought evidence from the literature for the articulation of how to recognize or treat the many events in the original Catalog of Critical Incidents appearing first in 1990 in the Anesthesia Crisis Resource Management course syllabus, and then in the subsequent book Crisis Management in Anesthesiology,10 there were no definitive reviews or guidelines available for a host of events that can and do occur during perioperative anesthesia care. It is likely that many uncommon events will never be studied thoroughly and hence that high levels of evidence will be lacking about their management. Large official bodies may be loath to provide guidelines or emergency manuals for the many situations where the evidence base is thin. However, clinicians in the field need reasonable guidance for what they should do when such events do occur. Creating such emergency manuals must balance several equally important needs. These are articulated in different ways in the 2 articles. The content of the advice must be reasonable and must apply the best evidence known where it exists. At the same time, advice must be presented for the suite of events whose management is critical even when unequivocal evidence is lacking. Considerations of human factors engineering are crucial for optimizing the design of the aids, and for how they are disseminated, implemented, and used. Many kinds of guidelines and practice parameters that are created through highly rigorous processes have not resulted in materials that can be used effectively by clinicians in the heat of the moment. The Advanced Cardiac Life Support booklet is a good example. It summarizes literature and recommendations thoroughly, but experience shows
1034 www.anesthesia-analgesia.org
that even its graphical flowcharts are very hard to read and to use. There is no particular reason why national or international bodies could not balance all these needs, much as the individual institutions have, although the various political, organizational, and proprietary interests of different stakeholders will be challenging to harness. Both articles recommend a process of user-centered design that combines substantial input by those familiar with human factors and cognition as well as graphical design. Both favor intensive iterative testing of the evolving aids with actual users, ideally in both simulated cases and actual clinical emergencies. The literature review by Marshall,1 the Stanford Case Study, and articles referenced in both publications demonstrate the utility of simulation testing. But even simulation studies are difficult to accomplish, especially when the key target population must be experienced personnel and not students or trainees. To obtain solid iterative testing for national or international consensus, emergency manuals would be complex, challenging, and expensive. Who is likely to step up to finance such studies?11 Will national clinical and scientific funding agencies see this kind of development as a priority, especially in the foreseeable future of shrinking budgets? It is possible that other stakeholders, such as liability insurers, stand to accrue greater direct benefits from the promulgation, adoption, and effective use of cognitive aids, but historically in health care, they themselves have rarely financed research, development, or implementation of safety interventions. The study of the use and impact of cognitive aids in and after real clinical crises will be equally difficult. Marshall1 and Goldhaber-Fiebert and Howard,2 each reference some of the studies that have been tried, particularly the Australian experience. These suffer from voluntary reporting (so that many cases with or without cognitive aid use are never analyzed); selection-biased reporting (those who use cognitive aids might be more likely to report events, or they might be more safety focused in the first place); inadequate information about what was really wrong with the patient and what exactly was done; and sparse information about how the aid was used, not used, or misused. Thus, it is reasonable to conclude that we may never be able to prove with Level 1a evidence (multiple wellcontrolled randomized trials) that using cognitive aids in managing critical events improves patient outcome. Even if funders could be identified, the studies might need to last, say, a decade and might involve hundreds or thousands of institutions, and tens of thousands of clinicians. Which institutions and clinicians would agree to be randomized to the control group? Is there really clinical equipoise about this issue? Are those individuals who claim equipoise actually a true cross section of the population of perioperative clinicians as a whole? Frankly, I think the answer is no for all these questions. Both articles rightly point out that evidence about cognitive aid creation, dissemination, and practical use is also weak or nonexistent at this point. Experience from the other industries, such as aviation and nuclear power, is impressive in concept and useful as a guide, but even there, Level 1a evidence is lacking. Moreover, exactly how best to translate successes from those industries to the practical aspects
anesthesia & analgesia
Perioperative Cognitive Aids
of optimizing the creation, use, or evaluation of such aids in health care remain uncertain. But absence of evidence is not evidence of absence, and the growing number of groups that have decided to vote with their feet by creating or adopting what they see as the best available set of cognitive aids attests to the desire of clinicians to avoid paralysis by analysis and to do something sensible to close the gap from nothing to something. As delineated in the 2 articles, it is not as simple as finding a suitable emergency manual and placing copies of it in the clinical environment. The process of adoption, placement, training of personnel, and creating and sustaining a culture that emphasizes and rewards their use is necessary but complex. As suggested by Goldhaber-Fiebert and Howard,2 the domain of “implementation science” is applicable both for studying how these processes transpire in different locales and settings, and for providing advice to those wishing to optimize their site’s use of cognitive aids. As hinted by the Australian experience, the use of data from event reporting and analysis systems holds out hope for 1 pathway of gathering at least anecdotal data on the use, successes, and failures or pitfalls of cognitive aids. In Australia, reporting of anesthetic events to the Australian Patient Safety Foundation now is only on an ad hoc basis. In the United States, the new Anesthesia Incident Reporting System operated by the Anesthesia Quality Institute can be a vehicle for collecting such reports (https://www.aqihq. org/airs/airsIntro.aspx). While such data may help drive the evolution of the aids and how best to use them, event reports can never be definitive and will never convince skeptics. Marshall1 suggests that poor aids, or poor use of aids, may distract clinicians and lead to worse performance than without them. That this could occur in some cases seems selfevident because it posits a suboptimal system from the start (poor aids, aids used poorly). Standards for how to properly create “good” aids and how best to prepare clinicians to use them “wisely” might be established for now, based on experience rather than on empirical evidence. How often worse clinical performance will result from the appropriate use of a set of well-designed aids by anesthesia professionals trained to use them is currently unknown. I think that this probability is likely to be very low. Simulation studies might answer this question, but again to acquire data from a real cross section of clinicians will be very challenging, and if worse performance is uncommon, very large studies would be needed to demonstrate its occurrence. Marshall1 also notes that in the Burden study of a “reader” role (who reads the aid to the team and checks on their progress in managing the event),12 there could be a danger implied by the observed results that “total communication” was reduced with a reader in place. However, since the effectiveness of the teams with a reader was much greater than in those without a reader, it seems far more likely (as acknowledged as a possibility by Marshall) that much of the communication during codes is spurious or redundant anyway, and the use of a reader might focus the team’s communication on the issues that are of greatest importance. Further studies could confirm whether the residual communication
November 2013 • Volume 117 • Number 5
is indeed tighter and whether all the critical communications still take place. My opinion is that what we already know about wellcrafted emergency manuals and the use of a reader role (when sufficient number of on-scene personnel allows it) is sufficient justification to warrant widespread adoption. Sites that fail to see value after a fair trial, even if only by anecdotal reports, will be free to abandon them, at least until more evidence has accumulated. Those that quickly find value in their use may well come to that conclusion because they have seen at least 1 life, heart, or brain saved in part by the use of a cognitive aid. There is a saying so good that it appears in various forms in both the Hebrew Talmud (Sanhedrin 4:5) and the Muslim Quran (5:32): “Whoever saves a life, it is as if he has saved the entire world.” If a set of plasticized pages or their equivalent can do that only once, the effort to adopt them and train people in their use should be well worth it.E DISCLOSURES
Name: David M. Gaba, MD. Contribution: This author wrote the manuscript. Attestation: David Gaba approved the final manuscript, and he is the archival author. Conflicts of Interest: First, I am a strong believer in the use of cognitive aids. Both articles reference our textbook Crisis Management in Anesthesiology as an early source of systematic presentation of information on how to recognize and manage a variety of different perioperative events. That book, and some of my prior articles, also includes textual material about the importance of cognitive aids as one of the vital “resources” for “crisis resource management.” The authors of the book receive small royalties on the original edition and on future editions (the 2nd edition is expected in early 2014). Second, Drs. Goldhaber-Fiebert and Howard, authors of the second article in this issue, are close academic associates of mine, Dr. Howard for >20 years. I participated peripherally in the drafting of the Emergency Manual described in their article. The Stanford Anesthesia Cognitive Aid Group, of which I am a member, collectively decided to provide the Stanford Emergency Manual to the world for free, in PDF form under a Creative Commons License, so as to maximize its availability to interested parties. Hence, we derive no financial gain from the dissemination of the Manual. This manuscript was handled by: Sorin J. Brull, MD, FCARCSI (Hon). REFERENCES 1. Marshall S. The use of cognitive aids during emergencies in anesthesia: A review of the literature. Anesth Analg 2013;117:1162–71 2. Goldhaber-Fiebert S, Howard S. Implementing emergency manuals: can cognitive aids help translate best practices for patient care during acute events. Anesth Analg 2013;117:1149–61 3. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AA; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9 4. Gawande A. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books, Henry Holt and Company, LLC, 2009 5. Arriaga AF, Bader AM, Wong JM, Lipsitz SR, Berry WR, Ziewacz JE, Hepner DL, Boorman DJ, Pozner CN, Smink DS,
www.anesthesia-analgesia.org 1035
E Editorial Gawande AA. Simulation-based trial of surgical-crisis checklists. N Engl J Med 2013;368:246–53 6. Norman D. Cognitive Artifacts, Designing Interaction: Psychology at the Human-Computer Interface. In: Carroll J. Cambridge, ed. Cambridge: Cambridge University Press, 1991:17–38 7. Neal JM, Bernards CM, Butterworth JF 4th, Di Gregorio G, Drasner K, Hejtmanek MR, Mulroy MF, Rosenquist RW, Weinberg GL. ASRA practice advisory on local anesthetic systemic toxicity. Reg Anesth Pain Med 2010;35:152–61 8. Australian Patient Safety Foundation. APSF Crisis Management Manual: “COVER ABCD A SWIFT CHECK” Adelaide, Australia: Australian Patient Safety Foundation, 1996
1036 www.anesthesia-analgesia.org
9. Runciman WB, Kluger MT, Morris RW, Paix AD, Watterson LM, Webb RK. Crisis management during anaesthesia: the development of an anaesthetic crisis management manual. Qual Saf Health Care 2005;14:e1 10. Gaba DM, Fish KJ, Howard SK. Crisis Management in Anesthesiology. New York, NY: Churchill Livingstone, 1994 11. Gaba DM. The pharmaceutical analogy for simulation: a policy perspective. Simul Healthc 2010;5:5–7 12. Burden AR, Carr ZJ, Staman GW, Littman JJ, Torjman MC. Does every code need a “reader?” improvement of rare event management with a cognitive aid “reader” during a simulated emergency: a pilot study. Simul Healthc 2012;7:1–9
anesthesia & analgesia
