Downloaded from http://gut.bmj.com/ on October 23, 2014 - Published by group.bmj.com
Commentary
Endoscopy training: time to stop counting procedures? Michael Bretthauer,1,2,3 Aleidis Skard Brandrud4 Until recently, training of endoscopists was largely up to the local environment, often unstructured and only sporadically 1
Institute of Health and Society, University of Oslo, Oslo, Norway; 2Department of Transplantation Medicine, Oslo University Hospital, Oslo, Norway; 3 Department of Medicine, Sørlandet Hospital, Kristiansand, Norway; 4Quality Department, Vestre Viken Hospital Trust, Drammen, Norway Correspondence to Dr Michael Bretthauer, Institute of Health and Society, Department of Health Management and Health Economics, University of Oslo, Postbox 1089 Blindern, Oslo 0318, Norway; [email protected] 1686
supervised. Consequently, there were large variations in the performance of endoscopy, and the quality of the service was very often too poor.1 During recent years, some countries have made great efforts to formalise and structure endoscopist training, often within the framework of continual quality improvement initiatives. This has led to impressive and sustained improvement in the quality of endoscopy services, as recently shown for colonoscopies in the UK.2 The number of procedures performed during training or practice has long been
used as a measure of competency. Clearly, competency of technical skills is related to how often someone does something. Repetition (increasing the number of times one does something within a certain amount of time) is a well recognised tool of learning and training, from simple tasks in elementary school to complex technical procedures such as colonoscopy. Thus, it is not surprising that counting numbers of procedures is a central tool to measure achievement of competency levels. Most gastroenterology training programmes require minimum numbers of procedures performed to gain certification. Some countries also have requirements for numbers of procedures after initial certification to be eligible to charge reimbursement fees, such as in the German colonoscopy screening programme which requires a minimum number of colonoscopies and polypectomies performed each Gut November 2014 Vol 63 No 11
Downloaded from http://gut.bmj.com/ on October 23, 2014 - Published by group.bmj.com
Commentary year to maintain access to the programme, and thus, reimbursement.3 Although number of procedures is a popular quality indicator (because it is easily measured and surveyed) it is a very crude proxy for quality. Number of procedures cannot be translated one-to-one to quality. What we really want to measure are important patient outcomes, such as post-colonoscopy colorectal cancer, serious adverse events and complications, and patient burden. These measures are called result or outcome indicators, as compared with number of procedures which is a proxy and called process indicator. Outcome indicators are the ones which matter most in the long run. However, they are difficult to ascertain on an individual endoscopist level. Ward and colleagues present very interesting data on the value of number of procedures performed during colonoscopy training as a proxy for quality.4 Using a nationwide UK database (the so-called JETS e-portfolio), they show that there is wide variation in the individual learning curve of colonoscopy trainees. As they have shown in figure 3 of their paper, some colonoscopy trainees were competent (as defined by caecum intubation rate of ≥90%) after 100 procedures, while others needed 300 procedures to achieve competency. Using the LC-Cusum method, 41% of trainees were competent after 200 procedures, while 76% needed 250 procedures.4 The LC-Cusum analysis is more difficult to follow, while the moving average method is more straightforward. With both methods, there is considerable variation in the individual
Gut November 2014 Vol 63 No 11
number of procedures to be performed for achieving competency. One of the crucial issues in quality improvement is the establishment of thresholds for good quality. These thresholds are required to define competency, for example, for certification as accredited endscopists in cancer screening programmes. The methods applied by Ward and co-workers to define competency (caecum intubation rate of ≥90%) are well-known in quality improvement. As expected, depending on the method applied, the definition of thresholds and error margins, one gets slightly different results, but all seem valid. The method chosen should be transparent, easy-tounderstand and simple to implement locally. The moving average method may have advantages in this respect. Getting rid of counting proxies such as numbers of procedures and instead directly measuring more important quality indicators such as caecum intubation rate, adenoma detection rate and ultimately individual outcome indicators as the ones mentioned above is the obvious next step. This seems within short-term reach because studies as the one by Ward et al demonstrate how modern quality improvement tools can be applied to endoscopy training. One hurdle, however, needs to be overcome first (and was not achieved in the Ward study); the implementation of mandatory registration of quality indicators for all colonoscopies performed. This is preferably achieved by implementation of structured colonoscopy reports within hospital’s electronic patient record systems.5
Contributors Both authors contributed equally at this manuscript. Competing interests None. Provenance and peer review Commissioned; internally peer reviewed.
To cite Bretthauer M, Brandrud AS. Gut 2014;63:1686–1687. Accepted 10 February 2014 Published Online First 6 March 2014
▸ http://dx.doi.org/10.1136/gutjnl-2013-305973 Gut 2014;63:1686–1687. doi:10.1136/gutjnl-2014-306765
REFERENCES 1
2
3
4
5
Bowles CJ, Leicester R, Romaya C, et al. A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow? Gut 2004;53:277–83. Gavin DR, Valori RM, Anderson JT, et al. The national colonoscopy audit: a nationwide assessment of the quality and safety of colonoscopy in the UK. Gut 2013;62:242–9. http://www.ziberlin.de/cms/fileadmin/images/content/ PDFs_alle/Darmkrebsfrueherk_Bericht.pdf (accessed 1 Feb 2014). Ward ST, Mohammed MA, Walt R, et al. An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut 2014;63:1746–54. van Doorn SC, van Vliet J, Fockens P, et al. A novel colonoscopy reporting system enabling quality assurance. Endoscopy 2014;46:181–7.
1687
Downloaded from http://gut.bmj.com/ on October 23, 2014 - Published by group.bmj.com
Endoscopy training: time to stop counting procedures? Michael Bretthauer and Aleidis Skard Brandrud Gut 2014 63: 1686-1687 originally published online March 6, 2014
doi: 10.1136/gutjnl-2014-306765 Updated information and services can be found at: http://gut.bmj.com/content/63/11/1686
These include:
References Email alerting service
This article cites 3 articles, 2 of which you can access for free at: http://gut.bmj.com/content/63/11/1686#BIBL Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
Commentary
Endoscopy training: time to stop counting procedures? Michael Bretthauer,1,2,3 Aleidis Skard Brandrud4 Until recently, training of endoscopists was largely up to the local environment, often unstructured and only sporadically 1
Institute of Health and Society, University of Oslo, Oslo, Norway; 2Department of Transplantation Medicine, Oslo University Hospital, Oslo, Norway; 3 Department of Medicine, Sørlandet Hospital, Kristiansand, Norway; 4Quality Department, Vestre Viken Hospital Trust, Drammen, Norway Correspondence to Dr Michael Bretthauer, Institute of Health and Society, Department of Health Management and Health Economics, University of Oslo, Postbox 1089 Blindern, Oslo 0318, Norway; [email protected] 1686
supervised. Consequently, there were large variations in the performance of endoscopy, and the quality of the service was very often too poor.1 During recent years, some countries have made great efforts to formalise and structure endoscopist training, often within the framework of continual quality improvement initiatives. This has led to impressive and sustained improvement in the quality of endoscopy services, as recently shown for colonoscopies in the UK.2 The number of procedures performed during training or practice has long been
used as a measure of competency. Clearly, competency of technical skills is related to how often someone does something. Repetition (increasing the number of times one does something within a certain amount of time) is a well recognised tool of learning and training, from simple tasks in elementary school to complex technical procedures such as colonoscopy. Thus, it is not surprising that counting numbers of procedures is a central tool to measure achievement of competency levels. Most gastroenterology training programmes require minimum numbers of procedures performed to gain certification. Some countries also have requirements for numbers of procedures after initial certification to be eligible to charge reimbursement fees, such as in the German colonoscopy screening programme which requires a minimum number of colonoscopies and polypectomies performed each Gut November 2014 Vol 63 No 11
Downloaded from http://gut.bmj.com/ on October 23, 2014 - Published by group.bmj.com
Commentary year to maintain access to the programme, and thus, reimbursement.3 Although number of procedures is a popular quality indicator (because it is easily measured and surveyed) it is a very crude proxy for quality. Number of procedures cannot be translated one-to-one to quality. What we really want to measure are important patient outcomes, such as post-colonoscopy colorectal cancer, serious adverse events and complications, and patient burden. These measures are called result or outcome indicators, as compared with number of procedures which is a proxy and called process indicator. Outcome indicators are the ones which matter most in the long run. However, they are difficult to ascertain on an individual endoscopist level. Ward and colleagues present very interesting data on the value of number of procedures performed during colonoscopy training as a proxy for quality.4 Using a nationwide UK database (the so-called JETS e-portfolio), they show that there is wide variation in the individual learning curve of colonoscopy trainees. As they have shown in figure 3 of their paper, some colonoscopy trainees were competent (as defined by caecum intubation rate of ≥90%) after 100 procedures, while others needed 300 procedures to achieve competency. Using the LC-Cusum method, 41% of trainees were competent after 200 procedures, while 76% needed 250 procedures.4 The LC-Cusum analysis is more difficult to follow, while the moving average method is more straightforward. With both methods, there is considerable variation in the individual
Gut November 2014 Vol 63 No 11
number of procedures to be performed for achieving competency. One of the crucial issues in quality improvement is the establishment of thresholds for good quality. These thresholds are required to define competency, for example, for certification as accredited endscopists in cancer screening programmes. The methods applied by Ward and co-workers to define competency (caecum intubation rate of ≥90%) are well-known in quality improvement. As expected, depending on the method applied, the definition of thresholds and error margins, one gets slightly different results, but all seem valid. The method chosen should be transparent, easy-tounderstand and simple to implement locally. The moving average method may have advantages in this respect. Getting rid of counting proxies such as numbers of procedures and instead directly measuring more important quality indicators such as caecum intubation rate, adenoma detection rate and ultimately individual outcome indicators as the ones mentioned above is the obvious next step. This seems within short-term reach because studies as the one by Ward et al demonstrate how modern quality improvement tools can be applied to endoscopy training. One hurdle, however, needs to be overcome first (and was not achieved in the Ward study); the implementation of mandatory registration of quality indicators for all colonoscopies performed. This is preferably achieved by implementation of structured colonoscopy reports within hospital’s electronic patient record systems.5
Contributors Both authors contributed equally at this manuscript. Competing interests None. Provenance and peer review Commissioned; internally peer reviewed.
To cite Bretthauer M, Brandrud AS. Gut 2014;63:1686–1687. Accepted 10 February 2014 Published Online First 6 March 2014
▸ http://dx.doi.org/10.1136/gutjnl-2013-305973 Gut 2014;63:1686–1687. doi:10.1136/gutjnl-2014-306765
REFERENCES 1
2
3
4
5
Bowles CJ, Leicester R, Romaya C, et al. A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow? Gut 2004;53:277–83. Gavin DR, Valori RM, Anderson JT, et al. The national colonoscopy audit: a nationwide assessment of the quality and safety of colonoscopy in the UK. Gut 2013;62:242–9. http://www.ziberlin.de/cms/fileadmin/images/content/ PDFs_alle/Darmkrebsfrueherk_Bericht.pdf (accessed 1 Feb 2014). Ward ST, Mohammed MA, Walt R, et al. An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut 2014;63:1746–54. van Doorn SC, van Vliet J, Fockens P, et al. A novel colonoscopy reporting system enabling quality assurance. Endoscopy 2014;46:181–7.
1687
Downloaded from http://gut.bmj.com/ on October 23, 2014 - Published by group.bmj.com
Endoscopy training: time to stop counting procedures? Michael Bretthauer and Aleidis Skard Brandrud Gut 2014 63: 1686-1687 originally published online March 6, 2014
doi: 10.1136/gutjnl-2014-306765 Updated information and services can be found at: http://gut.bmj.com/content/63/11/1686
These include:
References Email alerting service
This article cites 3 articles, 2 of which you can access for free at: http://gut.bmj.com/content/63/11/1686#BIBL Receive free email alerts when new articles cite this article. Sign up in the box at the top right corner of the online article.
Notes
To request permissions go to: http://group.bmj.com/group/rights-licensing/permissions To order reprints go to: http://journals.bmj.com/cgi/reprintform To subscribe to BMJ go to: http://group.bmj.com/subscribe/
