ORIGINAL ARTICLE: Clinical Endoscopy
The light at the end of the tunnel: a single-operator learning curve analysis for per oral endoscopic myotomy Kumkum Sarkar Patel, MD, MPH,1 Rose Calixte, PhD,2 Rani J. Modayil, MD,3 David Friedel, MD,3 Collin E. Brathwaite, MD, FACS,4 Stavros N. Stavropoulos, MD3 Mineola, New York, USA
Background: Per oral endoscopic myotomy (POEM) represents a natural orifice transluminal endoscopic surgery approach to Heller myotomy. Our center was the first to offer POEM outside of Japan, allowing us to accumulate what is likely the highest single-operator POEM volume in the United States. Objective: To define the POEM learning curve of a gastroenterologist by using a larger data set and more detailed statistical analysis than used in 2 other reports of POEM performed by surgeons. Design: Prospective cohort study. Setting: Tertiary-care academic medical center. Patients: We analyzed the first 93 consecutive POEMs on patients with achalasia aged O18 years without contraindications to POEM performed by a single operator from October 2009 to November 2013. Interventions: (1) Efficiency estimation via cumulative sum (CUSUM) analysis, (2) mastery estimation via penalized basis-spline regression and CUSUM analysis, (3) correlation of operator experience with clinical outcomes (Eckardt score improvement, lower esophageal sphincter pressure reduction) and technical errors (accidental mucosotomy rate), and (4) unadjusted and adjusted regression analysis to assess how patient characteristics affected procedure time by using a generalized linear model. Main Outcome Measurements: Clinical outcomes, procedure time, technical errors. Results: Efficiency was attained after 40 POEMs and mastery after 60 POEMs. When we used the adjusted regression analysis, only case number (operator experience) significantly affected procedure time (P! .0001). Improvements in clinical outcomes were excellent but not significantly affected by operator experience, as was the case with accidental mucosotomies. Procedure time was not significantly affected by age, sex, achalasia stage, baseline lower esophageal sphincter pressure, baseline Eckardt score, prior treatment of achalasia, prior botulinum toxin injection, incidence of accidental mucosotomies, length of myotomy, or type of knife used (all P O .05). Limitations: Our analysis may underestimate the number of POEMs required to achieve mastery for operators with limited or no endoscopic submucosal dissection experience. Conclusion: These results offer thresholds for efficiency and mastery of a single gastroenterologist operator that may guide the efforts of novice POEM operators. (Gastrointest Endosc 2015;81:1181-7.)
Abbreviations: AIC, Akaike information criterion; BIC, Bayesian information criterion; B-spline, basis-spline; ESD, endoscopic submucosal dissection; GEJ, gastroesophageal junction; LES, lower esophageal sphincter; NOTES, natural orifice transluminal endoscopic surgery; POEM, per oral endoscopic myotomy; TT, triangular tip. DISCLOSURE: C. Brathwaite is a consultant and speaker for Intuitive Surgical, Inc and a consultant for Covidien. S. Stavropoulos received an honorarium from ERBE and is a consultant for Boston Scientific. All other authors disclosed no financial relationships relevant to this article. Copyright ª 2015 by the American Society for Gastrointestinal Endoscopy
www.giejournal.org
0016-5107/$36.00 http://dx.doi.org/10.1016/j.gie.2014.10.002 Received June 17, 2014. Accepted October 2, 2014. Current affiliations: Department of Internal Medicine (1), Department of Biostatistics (2), Division of Gastroenterology (3), Division of Surgery, Winthrop University Hospital, Mineola, New York, USA (4). Reprint requests: Kumkum Sarkar Patel, MD, MPH, Winthrop University Hospital, Department of Internal Medicine, 260 First Street, Apt B13, Mineola, NY 11501.
Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1181
Single-operator learning curve analysis for per oral endoscopic myotomy
Patel et al
Per oral endoscopic myotomy (POEM) represents a natural orifice transluminal endoscopic surgery (NOTES) approach to Heller myotomy for treatment of achalasia. POEM was first performed in 2008, and the excellent initial outcomes by pioneering centers resulted in now rapidly expanding adoption.1 Defining the learning curve for this novel, challenging, high-risk procedure would be of importance to new physicians as they move from training to early clinical experience. Two prior reports on the POEM learning curve were based on a small number of POEMs (36 and 40, respectively) performed by multiple operators.2,3 Furthermore, in one of the studies, a simple block analysis was used rather than more detailed statistical techniques such as basis-spline (B-spline) regression and cumulative sum (CUSUM) analysis. Our center was the first to perform POEM after the initial human POEMs by Inoue et al1 in Japan, allowing us to accumulate what is likely the highest single-operator POEM volume in the United States.4-6 This substantial number of POEMs was analyzed with both B-spline regression and CUSUM analysis to define the learning curve for POEM. B-spline regression is a technique used to smooth a regression curves.7 CUSUM is a useful tool for examining performance improvement in surgical procedures.8,9 CUSUM analysis examines outcomes like procedure time to assess performance and uses deviations from the median or mean to graph the learning curve.9 CUSUM analysis has been used previously to examine procedural competence in a variety of procedures including regional anesthesia, endoscopic thyroidectomy, laparoscopic colectomy, endovascular aneurysm repair, video-assisted thorascopic lobectomy, and colonoscopy.9-14 However, there are no published reports describing a learning curve on POEM by using B-spline regression or CUSUM analysis. We present our experience in defining the learning curve of POEM. All procedures were completed by one operator (S.N.S.) who had extensive prior experience with endoscopic submucosal dissection (ESD) before initiation of POEM. His first POEM procedure in October 2009 was preceded by live animal laboratory procedures in ESD starting in 2004 and human cases starting in 2006, with approximately 100 hours of ESD training in live animals and 60 human ESD procedures completed before the first POEM.
mance refinements that lead to gradual decrease in procedure time, whereas mastery is defined as the point at which procedure time becomes consistent, and no further change in mean procedure time is observed.13 We used a penalized B-spline regression and CUSUM analysis to visually estimate the points at which efficiency and mastery were achieved. In addition to procedure time, we also included an analysis of technical errors (accidental mucosotomies) and clinical outcomes in our study (Eckardt score improvement and lower esophageal sphincter [LES] pressure reduction). We hypothesized that as the operator gained experience, not only would procedure time decrease but also technical errors would decrease, and the procedure would be performed better in some fashion that would be detectable by improved clinical outcomes. A secondary objective was to evaluate how certain patient and technique factors affect procedure time. We examined the following variables: age, sex, baseline LES pressure, baseline Eckardt score, achalasia stage, prior treatment for achalasia, prior botulinum toxin treatment, type of knife used, incidence of accidental mucosotomies, and length of myotomy. This study was a non-randomized, non-blinded, prospective cohort study. Sample size was determined by the number of patients who underwent POEM for achalasia from October 2009 to November 2013. For LES pressures, complete data analyses were done.
POEM technique
From October 2009 to November 2013, patients with achalasia aged O18 years, able to consent, and without contraindications to POEM were enrolled in a prospective internal review board–approved study. Data were collected on the first 93 consecutive POEM procedures performed at Winthrop University Hospital by a single operator (S.N.S.). Efficiency has been defined as the point in the learning curve in which the operator starts engaging in perfor-
The POEM technique used for our patients was similar to the technique first described by Inoue et al,1,15 with some important differences described later. A highdefinition gastroscope with a straight short transparent distal attachment was used along with CO2 insufflation.15 After determining the distance to the gastroesophageal junction (GEJ), we selected a location usually 10 to 15 cm proximal to it as the site of the initial submucosal entry.15,16 Then, a saline solution containing indigo carmine dye was injected to create a mucosal bleb before the initial 2-cm longitudinal mucosotomy. The submucosal tunnel was extended by using the ESD technique 2 to 3 cm beyond the GEJ into the cardia followed by myotomy and closure of the entry site and any accidental mucosotomies.1,4,15,16 In our case series, patients 1 to 7 underwent tunnel dissection by using a 12-mm CRE dilation balloon (Boston Scientific, Natick, Mass) followed by use of an Olympus flat knife (Olympus, Center Valley, Pa) to complete the tunnel dissection and the myotomy. This knife was used because of the absence of U.S. Food and Drug Administration–approved specialized ESD knives in the United States in 2009 and 2010. Patients 8 to 18 underwent tunnel dissection and myotomy with the triangular-tip (TT) knife (Olympus) as described by Inoue et al.1 Patients 19 to 93 underwent tunnel dissection and myotomy with the use
1182 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 5 : 2015
www.giejournal.org
METHODS
Patel et al
Single-operator learning curve analysis for per oral endoscopic myotomy
of the ERBE T-type hybrid knife (ERBE, Tübingen, Germany).5,16,17
model to find the best possible grouping for the 93 sequential procedure times and adjusted for any covariates with a P value ! .1000. We used the Tukey method to adjust for multiple comparisons of group means. Summary statistics were provided for categorical data by using frequency tables. For continuous and score data, we provided all measures of central tendency and all measures of spread to provide a better picture of the data. All data analyses for this article were done by using SAS 9.2 (Cary, NC, USA). Additionally, we used R 2.15.0 (Vienna, Austria) to perform the Marascuilo comparison.27
B-spline regression for learning curve We fit a penalized B-spline regression on the procedure time as a function of procedure number, with 4 knots and a polynomial of degree 4, and we searched for the best smoothing parameter lambda (l).7 Models with different l values were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).18,19 Because BIC tends to underestimate and AIC tends to overestimate the dimension of a model, both were used to select the l value that provided the best parameter fit.20
CUSUM analysis for learning curve We performed CUSUM analysis on the data by using the median procedure time as the targeted time to complete the procedure.21 Median procedure time was a better fit for our case series because our distribution of procedure time was positively skewed. We defined the residual time as the difference between the actual procedure time and the median procedure time. If the endoscopist reached mastery in doing POEMs after completing a certain number of procedures, then after that procedure, the change in the cumulative sum of the residual should be negligible. We plotted the cumulative sum of the residuals and, through graphical plateauing, estimated efficiency (the point after which the difference in the cumulative sum was negative) and mastery (the point after which the difference in the cumulative sum was negligible, indicating consistency in procedure time).10,11,13 We used locally weighted scatter plot smoothing regression to plot a smooth curve on the CUSUM values, with a low-degree polynomial being fitted to a subset of the data at each point in the data.22,23
Analysis of technical errors and clinical outcomes in relation to operator experience We used a Wilcoxon signed rank test to see if there was a significant improvement in postprocedure values. We used the permutation test to adjust for multiple comparisons based on rank of each change in score for sequential groups of patients.24 We compared the rate of accidental mucosotomies among sequential groups of patients by using the exact chi-square test and the method of Marascuilo for pair-wise comparison of the rate of accidental mucosotomies.25
Unadjusted and adjusted regression analysis to assess the effect of patient and technique characteristics on procedure time A Box-Cox transformation on procedure time had a lambda (l) value of 0, meaning that procedure time followed a lognormal distribution.26 For the second objective of this study, we used a generalized linear www.giejournal.org
RESULTS Baseline population and procedure characteristics Ninety-three consecutive patients (58 male, 35 female) with a mean age of 52 years (range 18-93 years) underwent POEM by the same operator. Of the 93 patients, 38 had prior treatment for achalasia, with 17 having had botulinum toxin as their prior treatment. Twenty-four patients (26%) were American Society of Anesthesiologists Physical Status Classification System class 3 (severe systemic disease) (Tables 1 and 2). When we used the previously reported achalasia staging shown in Table 3, the median stage was stage II, making up 61% of patients, with 22% of patients being stage IV.28 All attempted POEMs were completed with no aborted, discontinued, or incomplete procedures or surgical conversions. There were no deaths. Adverse events occurred in 2 patients (2%) and were minor: an 88-year-old man with congestive heart failure and severe aortic stenosis developed atrial fibrillation that was controlled medically (discharged on postoperative day 6; successfully underwent transaortic valve replacement 2 years after POEM); another patient with emphysema had atelectasis after extubation, which was asymptomatic and resolved within hours without intervention. There were no episodes of bleeding, no transfusions, no leaks, no surgical or radiologic interventions. There were 2 brief (%48 hours) readmissions not related to POEMdpacemaker lead malfunction in an 89-year-old patient and subjective fever and anxiety in a 35-year-old patient. At mean follow-up of 22 months (range 9-58 months), there was 96% clinical success (4 patients had increase of Eckardt score to O3 on follow-up after an initial response). The mean ( standard deviation) procedure time was 107.5 51.2 minutes. The median (interquartile range) for procedure time was 97 (65-140) minutes. Twenty-four patients (26%) sustained accidental mucosotomies, all repaired during the procedure without sequelae.
B-spline regression for learning curve We used penalized B-spline regression to determine the number of procedures needed to reach mastery (Fig. 1, Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1183
Single-operator learning curve analysis for per oral endoscopic myotomy
Patel et al
CUSUM analysis for learning curve TABLE 1. Patient demographics and characteristics Patient characteristic
No. (%)
Sex Female
35 (38)
Male
58 (62)
Stage of achalasia I
7 (8)
II
57 (61)
III
8 (9)
IV
21 (22)
ASA score 1
13 (14)
2
56 (60)
3
24 (26)
Sigmoid characterization Nonsigmoid
72 (77)
Sigmoid
21 (23)
We used the CUSUM analysis to estimate efficiency and mastery points for POEM. During the first 40 cases, actual procedure time was in part greater than the median (Fig. 2). However, after 40 procedures, there was a downward shift in the graph, which indicated that the procedure time was becoming less than the median. Clinically, the steep upward gradient in the curve represents the accrual of additional technical expertise and increase in competence in performing the POEM procedure.10 The peak of the CUSUM curvedat 40 proceduresdrepresents the point at which efficiency was achieved, because no further skills could be accrued. The downward gradient in the curve represented no further accrual of skills but rather what has been termed approach to mastery.10,11 During this phase, the operator engages in refinements that result in decreasing procedure time. The plateau of the curve at 60 procedures represents consistency in the procedure time, thus indicating achievement of mastery.10,11 Although there were some cases in which the median procedure time was less than the actual procedure time, the relative change in the curve was minimal after case 60, indicating mastery.
Prior botulinum toxin No
76 (82)
Yes
17 (18)
Prior treatment No
55 (59)
Yes
38 (41)
Type of knife Flat
7 (8)
Triangular tip
11 (12)
Hybrid
75 (80)
Accidental mucosotomy No
69 (74)
Yes
24 (26)
ASA, American Society of Anesthesiologists Physical Status Classification System.
Analysis of technical errors and clinical outcomes in relation to operator experience The summary statistics for LES pressures and Eckardt scores after POEM are presented in Table 2. The result of the Wilcoxon signed rank test indicated that postprocedure Eckardt scores and postprocedure LES pressures were significantly lower than baseline measurements (P ! .0001). When we used a permutation test on rank values, there was no difference among the sequential group of patients for change in Eckardt score and change in LES pressure (P O .93). There were no differences in the rate of accidental mucosotomies among the 5 sequential groups (P Z .22), based on the exact chi-square test. Further analysis by using the Marascuilo method for pair-wise comparison did not reveal any significant differences.
Unadjusted and adjusted regression analysis to assess the effect of patient and technique characteristics on procedure time
showing the regression line in a plot of procedure time as a function of case number). The smoothing parameter that minimized the expected value for procedure time was l Z 1.0, based on both the BIC and the AIC. The fitted penalized B-spline regression showed that after 59 procedures, the expected times to complete the procedure were fairly equal, demonstrating that the endoscopist had reached mastery. The 95% prediction limits showed that only 5 of the 93 procedures fell outside the prediction limits.
We fit a generalized linear model on procedure time by using each predictor. In the unadjusted analysis, neither age (P Z .12), sex (P Z .11), stage (P Z .33), baseline LES pressure (P Z .96), baseline Eckardt score (P Z .46), prior treatment of achalasia (P Z .74), prior botulinum toxin injections (P Z .27), nor incidence of accidental mucosotomies (P Z .53) significantly affected procedure time. However, case number, length of myotomy, and type of knife used significantly affected procedure time (P ! .001). For each unit increase in length of myotomy,
1184 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 5 : 2015
www.giejournal.org
Patel et al
Single-operator learning curve analysis for per oral endoscopic myotomy
TABLE 2. Baseline and outcome measurements Patients characteristic
No.
Mean
SD
Median
IQR
Range
Age, y
93
52
16
52
41-63
18-93
Length of myotomy, cm
93
11
5
11
9-14
3-26
Baseline LES pressure, mm/Hg
93
43
18
40
30-2
5.4-102.5
Baseline Eckardt score
93
8
2
8
7-9
4-12
Procedure time, min
93
108
51
97
65-140
38-240
LES pressure after POEM, mm/Hg
64
18
12
15
10.05-25
0-66
Eckardt score after POEM
93
0.44
0.91
0
0-1
0-7
SD, Standard deviation; IQR, interquartile range; LES, lower esophageal sphincter; POEM, per oral endoscopic myotomy.
250
TABLE 3. Stages of achalasia Maximum esophageal diameter and presence of sigmoidization on barium esophagram
1
%3 cm nonsigmoid
2
O3 cm to !6 cm nonsigmoid
3
6 cm to !8 cm nonsigmoid
4
R8 cm nonsigmoid or any size with sigmoidization
Procedure Duration (min)
Stage
59 cases 200
150
100
50
Modified from Tsiaoussis et al.28 0 0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Case
there was a 3.5% decrease in procedure time. Additionally, there was a 38.4% decrease and a 46.5% decrease in mean procedure time provided by the use of the hybrid knife as compare with the flat knife and TT knife, respectively (adjusted P ! .05). No significant difference was seen between the TT knife and the flat knife. In Figure 3, we showed the graph of the median interquartile range of procedure time for each sequential group of POEM procedures. Categorizing the data into 5 groups (20 patients in each of the first 4 groups and the remaining 13 patients in the 5th group) provided the best fit. After adjustment for multiple comparisons, the mean procedure time of group 1 was significantly higher than the mean procedure time of groups 3, 4, and 5 (P ! .0001). The mean procedure time of group 2 was significantly higher than the mean procedure time of groups 3, 4, and 5 (P ! .05). No other significant different pairs were observed. We fit an adjusted generalized linear model on procedure time by using the 5 sequential groups, adjusted for length of myotomy and type of knife used. In the adjusted model, only case number was significant (P ! .0001). Length of myotomy and knife used were no longer significant (P Z .22 and P Z .53). www.giejournal.org
95% Prediction Limits
95% Confidence Limits
Penalized B-spline, Smooth=1
Figure 1. Penalized B-spline regression with 95% prediction limits showing mastery achieved after 59 procedures.
DISCUSSION In our single-center, single-operator POEM experience, we analyzed procedure time and how it was affected by patient characteristics, and ultimately, we defined a learning curve by B-spline regression and CUSUM analysis. By using CUSUM analysis, efficiency in performing POEMs was attained after 40 procedures. Mastery in performing POEMs was reached after 59 procedures via B-spline regression and after 60 procedures via CUSUM. The achievement of efficiency and mastery was further validated by the overall significant decrease in procedure time from group 1 to group 5 (P ! .0001). The variability in procedure times resulted from the 5 cases that did not fit within the 95% prediction limits. Three of these 5 cases (cases 34, 77, and 90) had a severely dilated sigmoid esophagus and very high procedure times, thus becoming outliers. Length of myotomy and type of knife did significantly affect procedure time, but after adjusting for case number, Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1185
Single-operator learning curve analysis for per oral endoscopic myotomy
Patel et al
180
1500
160
40 cases 1000
Achievement of efficiency & approach to mastery
Achievement of
mastery
Procedure Time (min)
Cummulative sum of distance to median procedure time
200
60 cases
Median
152.5
140 120
Q1
120
100
Q3 83.5
80 68
60
59
Poly.(Median)
40
500
Accrual of additional
20
expertise
0 1-20
21-40
41-60
61-80
81-93
0 0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Case Lowess Smoothing Regression
Figure 2. CUSUM plot of procedure time with a locally weighted scatter plot smoothing regression curve. CUSUM analysis showing efficiency after 40 procedures and mastery after 60 procedures.
Figure 3. Sequential grouping of procedure time. Median (interquartile range) procedure times (minutes) for each sequential group showing overall decrease in procedure time.
neither factor was significant (P O .05). The small but paradoxical effect of myotomy length on procedure time in the unadjusted analysis that disappeared in the adjusted analysis may be caused by the following: We initially performed short 3 to 6–cm myotomies in our first 10 patients das done by Inoue et al1 in their early casesdbut quickly followed their lead (starting with case 11 in our series) when, once preliminary experience with the technique and its safety accrued, they progressed to at least standard Heller myotomy lengths of 8 to 10 cm.1 With regard to the knife used, a recent randomized trial found 36% shorter procedure time when the hybrid knife was used rather than the TT knife.17 Although our unadjusted analysis did indicate substantially shorter procedure times with the hybrid knife, knife selection was not random in our series, but sequential with all but the initial 18 POEMs performed with the hybrid knife. Thus, when adjusted for case number, this effect was not significant. In our series, clinical outcomes such as reduction in Eckardt scores and LES pressure and the rate of technical errors such as accidental mucosotomies did not correlate with operator experience (as measured by case number). This may be explained by the fact that our approach to POEM involves slow, meticulous dissection, as proposed by Inoue et al,1 followed by extensive intraprocedural assessment to ensure that an adequate myotomy had been performed, including use of the Endolumenal Functional Lumen Imaging Probe (EndoFLIP; Crospon, Carlsbad, Calif) and use of multiple other methods to confirm adequate myotomy. Additional dissection was performed if necessary until satisfactory ablation of the high-pressure zone was achieved. This approach may result in significantly longer procedure times when the operator is less experienced but ensures consistently high rates of clinical success and a low rate of technical errors.
We should note here that the 4 “failures” in our series, patients with symptom relapse at 47, 5, 14, and 15 months after POEM, had POEM early in our experience (case numbers 1, 3, 5, and 31, respectively). However, the lessdurable response in the first 3 of these 4 patients was almost certainly because of the short 3-cm myotomy performed in these early patients (as discussed earlier) rather than the more limited operator experience in these early cases. Another reason that may explain the lack of correlation between accidental mucosotomies and POEM operator experience in our study may be the significant ESD experience of the operator. The dissection of the submucosal tunneldwhen accidental mucosotomies usually occurdis indistinguishable from esophageal ESD for mucosal neoplasms. Therefore, the operator’s prior ESD experience may explain the relatively low, constant rate of 26%. Because we count even minimal thermal injuries to the mucosa causing subtle “blanching” as “accidental mucosotomies,” our rate of 26% overestimates the rate of true perforations requiring treatment. Notably, in the Kurian et al3 study, which demonstrated an association between accidental mucosotomies and operator experience, the rate of accidental mucosotomies in their first block of procedures was 100%. This is unusually high and may have been related to limited prior ESD experience by at least some of the surgeons in that study. We compared our study findings to the findings by Kurian et al.3 These authors defined mastery of the POEM technique to be attained after 20 cases, as evidenced by decrease in overall procedure time. However, the decrease in procedure time alone does not indicate attainment of mastery without also a plateau in procedure time, which was not demonstrated, possibly because of the small number of procedures analyzed.3 In our case series, there was also an overall decrease in median procedure time from group 1 to group 5 but it was the achievement of a plateau phase in procedure time (Figs. 1 and 2), that signaled achievement of both efficiency and mastery in performing POEMs.
1186 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 5 : 2015
www.giejournal.org
Patel et al
Single-operator learning curve analysis for per oral endoscopic myotomy
In comparing our data analysis to that of Teitelbaum et al,2 we noted that instead of CUSUM analysis, they used inverse regression analysis to graph the data of a small number of procedures (n Z 36) distributed among two operators, and they assessed characteristics such as age, body mass index (BMI), prior treatment of achalasia, and Eckardt score to analyze procedure difficulty. In contrast, our study used only one operator throughout all of the cases. Furthermore, Teitelbaum et al performed separate analyses and found the learning rate of 7 procedures for completing two portions of the POEM (achieving submucosal access and completing the myotomy). However, their study lacked power to define the learning rate for the entire POEM procedure with all 4 of its components or even for the submucosal tunnel dissection component of POEMdprobably the most arduous component to learn.2 A limitation of our study includes the fact that we did not compare more than 1 expert operator when creating the learning curve.2,3,13,14 This learning curve analysis is based on a gastroenterologist operator with significant prior advanced endoscopy and ESD experience and, therefore, it may underestimate the number of POEMs required to achieve mastery for operators without such experience.
4. Stavropoulos SN, Harris MD, Hida S, et al. Endoscopic submucosal myotomy for the treatment of achalasia (with video). Gastrointest Endosc 2010;72:1309-11. 5. Stavropoulos SN, Modayil RJ, Friedel D, et al. The International Per Oral Endoscopic Myotomy Survey (IPOEMS): a snapshot of the global POEM experience. Surg Endosc 2013;27:3322-38. 6. Friedel D, Modayil R, Iqbal S, et al. Per-oral endoscopic myotomy for achalasia: An American perspective. World J Gastrointest Endosc 2013;5:420-7. 7. Eilers PH, Marx BD. Flexible smoothing with B-splines and penalties. Stat Sci 1996;11:89-102. 8. MacDonald D, Davies R, Henderson GR. Bringing data to life with posthoc CUSUM charts. Case studies in business, industry and government statistics 2010;3:60. 9. Starkie T, Drake EJ. Assessment of procedural skills training and performance in anesthesia using cumulative sum analysis (cusum). Can J Anaesth 2013;60:1228-39. 10. Liao HJ, Dong C, Kong FJ, et al. The CUSUM analysis of the learning curve for endoscopic thyroidectomy by the breast approach. Surg Innov 2014;21:221-8. 11. Haas EM, Nieto J, Ragupathi M, et al. Critical appraisal of learning curve for single incision laparoscopic right colectomy. Surg Endosc 2013;27: 4499-503. 12. Kalteis M, Benedikt P, Huber F, et al. Looking for a learning curve in EVAR based on the Zenith stent graft. Int J Angiol 2012;21:223-8. 13. Li X, Wang J, Ferguson MK. Competence versus mastery: the time course for developing proficiency in video-assisted thoracoscopic lobectomy. J Thorac Cardiovasc Surg 2014;147:1150-4. 14. Ward ST, Mohammed MA, Walt R, et al. An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut 2014;63:1746-54. 15. Inoue H, Tianle KM, Ikea H, et al. Peroral endoscopic myotomy for esophageal achalasia: technique, indication, and outcomes. Thorac Surg Clin 2011;21:519-25. 16. Stavropoulos SN, Desilets DJ, Fuchs KH, et al. Per-oral endoscopic myotomy white paper summary. Surg Endosc 2014;28:2005-19. 17. Cai MY, Zhou PH, Yao LQ, et al. Peroral endoscopic myotomy for idiopathic achalasia: randomized comparison of water-jet assisted versus conventional dissection technique. Surg Endosc 2014;28:1158-65. 18. Akaike, H. A new look at statistical-model identification. IEEE Xplore Digital Library. Automatic Control, IEEE Transactions on. 1974;19:716-23. 19. Schwarz G. Estimating the dimension of a model. Ann Statis 1978;6: 461-4. 20. Dziak JJ, et al. Sensitivity and specificity of information criteria. The Methodology Center and Department of Statistics, The Pennsylvania State University, 2012; 12-25.e1. 21. Page E. Continuous inspection schemes. Biometrika 1954;41:100-15. 22. Cleveland WS. Robust locally weighted regression and smoothing scatterplots. J Am Statistic Assoc 1979;74:829-36. 23. Cleveland WS, Devlin SJ. Locally weighted regression: an approach to regression analysis by local fitting. J Am Statistic Assoc 1988;83: 596-610. 24. Westfall PH, Young SS. P value adjustments for multiple tests in multivariate binomial models. J Am Statistic Assoc 1989;84:780-6. 25. Marascuilo LA. Large-sample multiple comparisons. Psychol Bull 1966;65:280-90. 26. Box GEP, Cox DR. An analysis of transformations. J Royal Statistic Soc 1964;26:211-52; Series B (methodological). 27. Team RDC. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2012. 28. Tsiaoussis J, Athanasakis E, Pechlivanides G, et al. Long-term functional results after laparoscopic surgery for esophageal achalasia. Am J Surg 2007;193:26-31.
Conclusion POEM is a major advance in the treatment of achalasia that offers a less invasive, NOTES approach to Heller myotomy that can be performed in the endoscopy unit with lower resource utilization.16 By allowing for longer lengths of myotomy extending into the proximal body of the esophagus, POEM also offers the promise of superior therapeutic effect in patients with spastic achalasia in comparison to Heller myotomy.16 Given the rarity of achalasia, a relatively short learning curve would be achievable in centers with large referral centers for motility disorders. However, the converse may be true in centers with low volume of referrals. Our study determined a number of procedures for achieving efficiency and mastery that appears realistic for this complex procedure and can be used to guide the efforts of other novice operators, particularly in centers with low referral volumes.
REFERENCES 1. Inoue H, Minami H, Kobayashi Y, et al. Peroral endoscopic myotomy (POEM) for esophageal achalasia. Endoscopy 2010;42:265-71. 2. Teitelbaum EN, Soper NJ, Arafat FO, et al. Analysis of a learning curve and predictors of intraoperative difficulty for peroral esophageal myotomy (POEM). J Gastrointest Surg 2014;18:92-9. 3. Kurian AA, Dunst CM, Sharata A, et al. Peroral endoscopic esophageal myotomy: defining the learning curve. Gastrointest Endosc 2013;77: 719-25.
www.giejournal.org
Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1187
The light at the end of the tunnel: a single-operator learning curve analysis for per oral endoscopic myotomy Kumkum Sarkar Patel, MD, MPH,1 Rose Calixte, PhD,2 Rani J. Modayil, MD,3 David Friedel, MD,3 Collin E. Brathwaite, MD, FACS,4 Stavros N. Stavropoulos, MD3 Mineola, New York, USA
Background: Per oral endoscopic myotomy (POEM) represents a natural orifice transluminal endoscopic surgery approach to Heller myotomy. Our center was the first to offer POEM outside of Japan, allowing us to accumulate what is likely the highest single-operator POEM volume in the United States. Objective: To define the POEM learning curve of a gastroenterologist by using a larger data set and more detailed statistical analysis than used in 2 other reports of POEM performed by surgeons. Design: Prospective cohort study. Setting: Tertiary-care academic medical center. Patients: We analyzed the first 93 consecutive POEMs on patients with achalasia aged O18 years without contraindications to POEM performed by a single operator from October 2009 to November 2013. Interventions: (1) Efficiency estimation via cumulative sum (CUSUM) analysis, (2) mastery estimation via penalized basis-spline regression and CUSUM analysis, (3) correlation of operator experience with clinical outcomes (Eckardt score improvement, lower esophageal sphincter pressure reduction) and technical errors (accidental mucosotomy rate), and (4) unadjusted and adjusted regression analysis to assess how patient characteristics affected procedure time by using a generalized linear model. Main Outcome Measurements: Clinical outcomes, procedure time, technical errors. Results: Efficiency was attained after 40 POEMs and mastery after 60 POEMs. When we used the adjusted regression analysis, only case number (operator experience) significantly affected procedure time (P! .0001). Improvements in clinical outcomes were excellent but not significantly affected by operator experience, as was the case with accidental mucosotomies. Procedure time was not significantly affected by age, sex, achalasia stage, baseline lower esophageal sphincter pressure, baseline Eckardt score, prior treatment of achalasia, prior botulinum toxin injection, incidence of accidental mucosotomies, length of myotomy, or type of knife used (all P O .05). Limitations: Our analysis may underestimate the number of POEMs required to achieve mastery for operators with limited or no endoscopic submucosal dissection experience. Conclusion: These results offer thresholds for efficiency and mastery of a single gastroenterologist operator that may guide the efforts of novice POEM operators. (Gastrointest Endosc 2015;81:1181-7.)
Abbreviations: AIC, Akaike information criterion; BIC, Bayesian information criterion; B-spline, basis-spline; ESD, endoscopic submucosal dissection; GEJ, gastroesophageal junction; LES, lower esophageal sphincter; NOTES, natural orifice transluminal endoscopic surgery; POEM, per oral endoscopic myotomy; TT, triangular tip. DISCLOSURE: C. Brathwaite is a consultant and speaker for Intuitive Surgical, Inc and a consultant for Covidien. S. Stavropoulos received an honorarium from ERBE and is a consultant for Boston Scientific. All other authors disclosed no financial relationships relevant to this article. Copyright ª 2015 by the American Society for Gastrointestinal Endoscopy
www.giejournal.org
0016-5107/$36.00 http://dx.doi.org/10.1016/j.gie.2014.10.002 Received June 17, 2014. Accepted October 2, 2014. Current affiliations: Department of Internal Medicine (1), Department of Biostatistics (2), Division of Gastroenterology (3), Division of Surgery, Winthrop University Hospital, Mineola, New York, USA (4). Reprint requests: Kumkum Sarkar Patel, MD, MPH, Winthrop University Hospital, Department of Internal Medicine, 260 First Street, Apt B13, Mineola, NY 11501.
Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1181
Single-operator learning curve analysis for per oral endoscopic myotomy
Patel et al
Per oral endoscopic myotomy (POEM) represents a natural orifice transluminal endoscopic surgery (NOTES) approach to Heller myotomy for treatment of achalasia. POEM was first performed in 2008, and the excellent initial outcomes by pioneering centers resulted in now rapidly expanding adoption.1 Defining the learning curve for this novel, challenging, high-risk procedure would be of importance to new physicians as they move from training to early clinical experience. Two prior reports on the POEM learning curve were based on a small number of POEMs (36 and 40, respectively) performed by multiple operators.2,3 Furthermore, in one of the studies, a simple block analysis was used rather than more detailed statistical techniques such as basis-spline (B-spline) regression and cumulative sum (CUSUM) analysis. Our center was the first to perform POEM after the initial human POEMs by Inoue et al1 in Japan, allowing us to accumulate what is likely the highest single-operator POEM volume in the United States.4-6 This substantial number of POEMs was analyzed with both B-spline regression and CUSUM analysis to define the learning curve for POEM. B-spline regression is a technique used to smooth a regression curves.7 CUSUM is a useful tool for examining performance improvement in surgical procedures.8,9 CUSUM analysis examines outcomes like procedure time to assess performance and uses deviations from the median or mean to graph the learning curve.9 CUSUM analysis has been used previously to examine procedural competence in a variety of procedures including regional anesthesia, endoscopic thyroidectomy, laparoscopic colectomy, endovascular aneurysm repair, video-assisted thorascopic lobectomy, and colonoscopy.9-14 However, there are no published reports describing a learning curve on POEM by using B-spline regression or CUSUM analysis. We present our experience in defining the learning curve of POEM. All procedures were completed by one operator (S.N.S.) who had extensive prior experience with endoscopic submucosal dissection (ESD) before initiation of POEM. His first POEM procedure in October 2009 was preceded by live animal laboratory procedures in ESD starting in 2004 and human cases starting in 2006, with approximately 100 hours of ESD training in live animals and 60 human ESD procedures completed before the first POEM.
mance refinements that lead to gradual decrease in procedure time, whereas mastery is defined as the point at which procedure time becomes consistent, and no further change in mean procedure time is observed.13 We used a penalized B-spline regression and CUSUM analysis to visually estimate the points at which efficiency and mastery were achieved. In addition to procedure time, we also included an analysis of technical errors (accidental mucosotomies) and clinical outcomes in our study (Eckardt score improvement and lower esophageal sphincter [LES] pressure reduction). We hypothesized that as the operator gained experience, not only would procedure time decrease but also technical errors would decrease, and the procedure would be performed better in some fashion that would be detectable by improved clinical outcomes. A secondary objective was to evaluate how certain patient and technique factors affect procedure time. We examined the following variables: age, sex, baseline LES pressure, baseline Eckardt score, achalasia stage, prior treatment for achalasia, prior botulinum toxin treatment, type of knife used, incidence of accidental mucosotomies, and length of myotomy. This study was a non-randomized, non-blinded, prospective cohort study. Sample size was determined by the number of patients who underwent POEM for achalasia from October 2009 to November 2013. For LES pressures, complete data analyses were done.
POEM technique
From October 2009 to November 2013, patients with achalasia aged O18 years, able to consent, and without contraindications to POEM were enrolled in a prospective internal review board–approved study. Data were collected on the first 93 consecutive POEM procedures performed at Winthrop University Hospital by a single operator (S.N.S.). Efficiency has been defined as the point in the learning curve in which the operator starts engaging in perfor-
The POEM technique used for our patients was similar to the technique first described by Inoue et al,1,15 with some important differences described later. A highdefinition gastroscope with a straight short transparent distal attachment was used along with CO2 insufflation.15 After determining the distance to the gastroesophageal junction (GEJ), we selected a location usually 10 to 15 cm proximal to it as the site of the initial submucosal entry.15,16 Then, a saline solution containing indigo carmine dye was injected to create a mucosal bleb before the initial 2-cm longitudinal mucosotomy. The submucosal tunnel was extended by using the ESD technique 2 to 3 cm beyond the GEJ into the cardia followed by myotomy and closure of the entry site and any accidental mucosotomies.1,4,15,16 In our case series, patients 1 to 7 underwent tunnel dissection by using a 12-mm CRE dilation balloon (Boston Scientific, Natick, Mass) followed by use of an Olympus flat knife (Olympus, Center Valley, Pa) to complete the tunnel dissection and the myotomy. This knife was used because of the absence of U.S. Food and Drug Administration–approved specialized ESD knives in the United States in 2009 and 2010. Patients 8 to 18 underwent tunnel dissection and myotomy with the triangular-tip (TT) knife (Olympus) as described by Inoue et al.1 Patients 19 to 93 underwent tunnel dissection and myotomy with the use
1182 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 5 : 2015
www.giejournal.org
METHODS
Patel et al
Single-operator learning curve analysis for per oral endoscopic myotomy
of the ERBE T-type hybrid knife (ERBE, Tübingen, Germany).5,16,17
model to find the best possible grouping for the 93 sequential procedure times and adjusted for any covariates with a P value ! .1000. We used the Tukey method to adjust for multiple comparisons of group means. Summary statistics were provided for categorical data by using frequency tables. For continuous and score data, we provided all measures of central tendency and all measures of spread to provide a better picture of the data. All data analyses for this article were done by using SAS 9.2 (Cary, NC, USA). Additionally, we used R 2.15.0 (Vienna, Austria) to perform the Marascuilo comparison.27
B-spline regression for learning curve We fit a penalized B-spline regression on the procedure time as a function of procedure number, with 4 knots and a polynomial of degree 4, and we searched for the best smoothing parameter lambda (l).7 Models with different l values were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).18,19 Because BIC tends to underestimate and AIC tends to overestimate the dimension of a model, both were used to select the l value that provided the best parameter fit.20
CUSUM analysis for learning curve We performed CUSUM analysis on the data by using the median procedure time as the targeted time to complete the procedure.21 Median procedure time was a better fit for our case series because our distribution of procedure time was positively skewed. We defined the residual time as the difference between the actual procedure time and the median procedure time. If the endoscopist reached mastery in doing POEMs after completing a certain number of procedures, then after that procedure, the change in the cumulative sum of the residual should be negligible. We plotted the cumulative sum of the residuals and, through graphical plateauing, estimated efficiency (the point after which the difference in the cumulative sum was negative) and mastery (the point after which the difference in the cumulative sum was negligible, indicating consistency in procedure time).10,11,13 We used locally weighted scatter plot smoothing regression to plot a smooth curve on the CUSUM values, with a low-degree polynomial being fitted to a subset of the data at each point in the data.22,23
Analysis of technical errors and clinical outcomes in relation to operator experience We used a Wilcoxon signed rank test to see if there was a significant improvement in postprocedure values. We used the permutation test to adjust for multiple comparisons based on rank of each change in score for sequential groups of patients.24 We compared the rate of accidental mucosotomies among sequential groups of patients by using the exact chi-square test and the method of Marascuilo for pair-wise comparison of the rate of accidental mucosotomies.25
Unadjusted and adjusted regression analysis to assess the effect of patient and technique characteristics on procedure time A Box-Cox transformation on procedure time had a lambda (l) value of 0, meaning that procedure time followed a lognormal distribution.26 For the second objective of this study, we used a generalized linear www.giejournal.org
RESULTS Baseline population and procedure characteristics Ninety-three consecutive patients (58 male, 35 female) with a mean age of 52 years (range 18-93 years) underwent POEM by the same operator. Of the 93 patients, 38 had prior treatment for achalasia, with 17 having had botulinum toxin as their prior treatment. Twenty-four patients (26%) were American Society of Anesthesiologists Physical Status Classification System class 3 (severe systemic disease) (Tables 1 and 2). When we used the previously reported achalasia staging shown in Table 3, the median stage was stage II, making up 61% of patients, with 22% of patients being stage IV.28 All attempted POEMs were completed with no aborted, discontinued, or incomplete procedures or surgical conversions. There were no deaths. Adverse events occurred in 2 patients (2%) and were minor: an 88-year-old man with congestive heart failure and severe aortic stenosis developed atrial fibrillation that was controlled medically (discharged on postoperative day 6; successfully underwent transaortic valve replacement 2 years after POEM); another patient with emphysema had atelectasis after extubation, which was asymptomatic and resolved within hours without intervention. There were no episodes of bleeding, no transfusions, no leaks, no surgical or radiologic interventions. There were 2 brief (%48 hours) readmissions not related to POEMdpacemaker lead malfunction in an 89-year-old patient and subjective fever and anxiety in a 35-year-old patient. At mean follow-up of 22 months (range 9-58 months), there was 96% clinical success (4 patients had increase of Eckardt score to O3 on follow-up after an initial response). The mean ( standard deviation) procedure time was 107.5 51.2 minutes. The median (interquartile range) for procedure time was 97 (65-140) minutes. Twenty-four patients (26%) sustained accidental mucosotomies, all repaired during the procedure without sequelae.
B-spline regression for learning curve We used penalized B-spline regression to determine the number of procedures needed to reach mastery (Fig. 1, Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1183
Single-operator learning curve analysis for per oral endoscopic myotomy
Patel et al
CUSUM analysis for learning curve TABLE 1. Patient demographics and characteristics Patient characteristic
No. (%)
Sex Female
35 (38)
Male
58 (62)
Stage of achalasia I
7 (8)
II
57 (61)
III
8 (9)
IV
21 (22)
ASA score 1
13 (14)
2
56 (60)
3
24 (26)
Sigmoid characterization Nonsigmoid
72 (77)
Sigmoid
21 (23)
We used the CUSUM analysis to estimate efficiency and mastery points for POEM. During the first 40 cases, actual procedure time was in part greater than the median (Fig. 2). However, after 40 procedures, there was a downward shift in the graph, which indicated that the procedure time was becoming less than the median. Clinically, the steep upward gradient in the curve represents the accrual of additional technical expertise and increase in competence in performing the POEM procedure.10 The peak of the CUSUM curvedat 40 proceduresdrepresents the point at which efficiency was achieved, because no further skills could be accrued. The downward gradient in the curve represented no further accrual of skills but rather what has been termed approach to mastery.10,11 During this phase, the operator engages in refinements that result in decreasing procedure time. The plateau of the curve at 60 procedures represents consistency in the procedure time, thus indicating achievement of mastery.10,11 Although there were some cases in which the median procedure time was less than the actual procedure time, the relative change in the curve was minimal after case 60, indicating mastery.
Prior botulinum toxin No
76 (82)
Yes
17 (18)
Prior treatment No
55 (59)
Yes
38 (41)
Type of knife Flat
7 (8)
Triangular tip
11 (12)
Hybrid
75 (80)
Accidental mucosotomy No
69 (74)
Yes
24 (26)
ASA, American Society of Anesthesiologists Physical Status Classification System.
Analysis of technical errors and clinical outcomes in relation to operator experience The summary statistics for LES pressures and Eckardt scores after POEM are presented in Table 2. The result of the Wilcoxon signed rank test indicated that postprocedure Eckardt scores and postprocedure LES pressures were significantly lower than baseline measurements (P ! .0001). When we used a permutation test on rank values, there was no difference among the sequential group of patients for change in Eckardt score and change in LES pressure (P O .93). There were no differences in the rate of accidental mucosotomies among the 5 sequential groups (P Z .22), based on the exact chi-square test. Further analysis by using the Marascuilo method for pair-wise comparison did not reveal any significant differences.
Unadjusted and adjusted regression analysis to assess the effect of patient and technique characteristics on procedure time
showing the regression line in a plot of procedure time as a function of case number). The smoothing parameter that minimized the expected value for procedure time was l Z 1.0, based on both the BIC and the AIC. The fitted penalized B-spline regression showed that after 59 procedures, the expected times to complete the procedure were fairly equal, demonstrating that the endoscopist had reached mastery. The 95% prediction limits showed that only 5 of the 93 procedures fell outside the prediction limits.
We fit a generalized linear model on procedure time by using each predictor. In the unadjusted analysis, neither age (P Z .12), sex (P Z .11), stage (P Z .33), baseline LES pressure (P Z .96), baseline Eckardt score (P Z .46), prior treatment of achalasia (P Z .74), prior botulinum toxin injections (P Z .27), nor incidence of accidental mucosotomies (P Z .53) significantly affected procedure time. However, case number, length of myotomy, and type of knife used significantly affected procedure time (P ! .001). For each unit increase in length of myotomy,
1184 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 5 : 2015
www.giejournal.org
Patel et al
Single-operator learning curve analysis for per oral endoscopic myotomy
TABLE 2. Baseline and outcome measurements Patients characteristic
No.
Mean
SD
Median
IQR
Range
Age, y
93
52
16
52
41-63
18-93
Length of myotomy, cm
93
11
5
11
9-14
3-26
Baseline LES pressure, mm/Hg
93
43
18
40
30-2
5.4-102.5
Baseline Eckardt score
93
8
2
8
7-9
4-12
Procedure time, min
93
108
51
97
65-140
38-240
LES pressure after POEM, mm/Hg
64
18
12
15
10.05-25
0-66
Eckardt score after POEM
93
0.44
0.91
0
0-1
0-7
SD, Standard deviation; IQR, interquartile range; LES, lower esophageal sphincter; POEM, per oral endoscopic myotomy.
250
TABLE 3. Stages of achalasia Maximum esophageal diameter and presence of sigmoidization on barium esophagram
1
%3 cm nonsigmoid
2
O3 cm to !6 cm nonsigmoid
3
6 cm to !8 cm nonsigmoid
4
R8 cm nonsigmoid or any size with sigmoidization
Procedure Duration (min)
Stage
59 cases 200
150
100
50
Modified from Tsiaoussis et al.28 0 0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Case
there was a 3.5% decrease in procedure time. Additionally, there was a 38.4% decrease and a 46.5% decrease in mean procedure time provided by the use of the hybrid knife as compare with the flat knife and TT knife, respectively (adjusted P ! .05). No significant difference was seen between the TT knife and the flat knife. In Figure 3, we showed the graph of the median interquartile range of procedure time for each sequential group of POEM procedures. Categorizing the data into 5 groups (20 patients in each of the first 4 groups and the remaining 13 patients in the 5th group) provided the best fit. After adjustment for multiple comparisons, the mean procedure time of group 1 was significantly higher than the mean procedure time of groups 3, 4, and 5 (P ! .0001). The mean procedure time of group 2 was significantly higher than the mean procedure time of groups 3, 4, and 5 (P ! .05). No other significant different pairs were observed. We fit an adjusted generalized linear model on procedure time by using the 5 sequential groups, adjusted for length of myotomy and type of knife used. In the adjusted model, only case number was significant (P ! .0001). Length of myotomy and knife used were no longer significant (P Z .22 and P Z .53). www.giejournal.org
95% Prediction Limits
95% Confidence Limits
Penalized B-spline, Smooth=1
Figure 1. Penalized B-spline regression with 95% prediction limits showing mastery achieved after 59 procedures.
DISCUSSION In our single-center, single-operator POEM experience, we analyzed procedure time and how it was affected by patient characteristics, and ultimately, we defined a learning curve by B-spline regression and CUSUM analysis. By using CUSUM analysis, efficiency in performing POEMs was attained after 40 procedures. Mastery in performing POEMs was reached after 59 procedures via B-spline regression and after 60 procedures via CUSUM. The achievement of efficiency and mastery was further validated by the overall significant decrease in procedure time from group 1 to group 5 (P ! .0001). The variability in procedure times resulted from the 5 cases that did not fit within the 95% prediction limits. Three of these 5 cases (cases 34, 77, and 90) had a severely dilated sigmoid esophagus and very high procedure times, thus becoming outliers. Length of myotomy and type of knife did significantly affect procedure time, but after adjusting for case number, Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1185
Single-operator learning curve analysis for per oral endoscopic myotomy
Patel et al
180
1500
160
40 cases 1000
Achievement of efficiency & approach to mastery
Achievement of
mastery
Procedure Time (min)
Cummulative sum of distance to median procedure time
200
60 cases
Median
152.5
140 120
Q1
120
100
Q3 83.5
80 68
60
59
Poly.(Median)
40
500
Accrual of additional
20
expertise
0 1-20
21-40
41-60
61-80
81-93
0 0
5
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Case Lowess Smoothing Regression
Figure 2. CUSUM plot of procedure time with a locally weighted scatter plot smoothing regression curve. CUSUM analysis showing efficiency after 40 procedures and mastery after 60 procedures.
Figure 3. Sequential grouping of procedure time. Median (interquartile range) procedure times (minutes) for each sequential group showing overall decrease in procedure time.
neither factor was significant (P O .05). The small but paradoxical effect of myotomy length on procedure time in the unadjusted analysis that disappeared in the adjusted analysis may be caused by the following: We initially performed short 3 to 6–cm myotomies in our first 10 patients das done by Inoue et al1 in their early casesdbut quickly followed their lead (starting with case 11 in our series) when, once preliminary experience with the technique and its safety accrued, they progressed to at least standard Heller myotomy lengths of 8 to 10 cm.1 With regard to the knife used, a recent randomized trial found 36% shorter procedure time when the hybrid knife was used rather than the TT knife.17 Although our unadjusted analysis did indicate substantially shorter procedure times with the hybrid knife, knife selection was not random in our series, but sequential with all but the initial 18 POEMs performed with the hybrid knife. Thus, when adjusted for case number, this effect was not significant. In our series, clinical outcomes such as reduction in Eckardt scores and LES pressure and the rate of technical errors such as accidental mucosotomies did not correlate with operator experience (as measured by case number). This may be explained by the fact that our approach to POEM involves slow, meticulous dissection, as proposed by Inoue et al,1 followed by extensive intraprocedural assessment to ensure that an adequate myotomy had been performed, including use of the Endolumenal Functional Lumen Imaging Probe (EndoFLIP; Crospon, Carlsbad, Calif) and use of multiple other methods to confirm adequate myotomy. Additional dissection was performed if necessary until satisfactory ablation of the high-pressure zone was achieved. This approach may result in significantly longer procedure times when the operator is less experienced but ensures consistently high rates of clinical success and a low rate of technical errors.
We should note here that the 4 “failures” in our series, patients with symptom relapse at 47, 5, 14, and 15 months after POEM, had POEM early in our experience (case numbers 1, 3, 5, and 31, respectively). However, the lessdurable response in the first 3 of these 4 patients was almost certainly because of the short 3-cm myotomy performed in these early patients (as discussed earlier) rather than the more limited operator experience in these early cases. Another reason that may explain the lack of correlation between accidental mucosotomies and POEM operator experience in our study may be the significant ESD experience of the operator. The dissection of the submucosal tunneldwhen accidental mucosotomies usually occurdis indistinguishable from esophageal ESD for mucosal neoplasms. Therefore, the operator’s prior ESD experience may explain the relatively low, constant rate of 26%. Because we count even minimal thermal injuries to the mucosa causing subtle “blanching” as “accidental mucosotomies,” our rate of 26% overestimates the rate of true perforations requiring treatment. Notably, in the Kurian et al3 study, which demonstrated an association between accidental mucosotomies and operator experience, the rate of accidental mucosotomies in their first block of procedures was 100%. This is unusually high and may have been related to limited prior ESD experience by at least some of the surgeons in that study. We compared our study findings to the findings by Kurian et al.3 These authors defined mastery of the POEM technique to be attained after 20 cases, as evidenced by decrease in overall procedure time. However, the decrease in procedure time alone does not indicate attainment of mastery without also a plateau in procedure time, which was not demonstrated, possibly because of the small number of procedures analyzed.3 In our case series, there was also an overall decrease in median procedure time from group 1 to group 5 but it was the achievement of a plateau phase in procedure time (Figs. 1 and 2), that signaled achievement of both efficiency and mastery in performing POEMs.
1186 GASTROINTESTINAL ENDOSCOPY Volume 81, No. 5 : 2015
www.giejournal.org
Patel et al
Single-operator learning curve analysis for per oral endoscopic myotomy
In comparing our data analysis to that of Teitelbaum et al,2 we noted that instead of CUSUM analysis, they used inverse regression analysis to graph the data of a small number of procedures (n Z 36) distributed among two operators, and they assessed characteristics such as age, body mass index (BMI), prior treatment of achalasia, and Eckardt score to analyze procedure difficulty. In contrast, our study used only one operator throughout all of the cases. Furthermore, Teitelbaum et al performed separate analyses and found the learning rate of 7 procedures for completing two portions of the POEM (achieving submucosal access and completing the myotomy). However, their study lacked power to define the learning rate for the entire POEM procedure with all 4 of its components or even for the submucosal tunnel dissection component of POEMdprobably the most arduous component to learn.2 A limitation of our study includes the fact that we did not compare more than 1 expert operator when creating the learning curve.2,3,13,14 This learning curve analysis is based on a gastroenterologist operator with significant prior advanced endoscopy and ESD experience and, therefore, it may underestimate the number of POEMs required to achieve mastery for operators without such experience.
4. Stavropoulos SN, Harris MD, Hida S, et al. Endoscopic submucosal myotomy for the treatment of achalasia (with video). Gastrointest Endosc 2010;72:1309-11. 5. Stavropoulos SN, Modayil RJ, Friedel D, et al. The International Per Oral Endoscopic Myotomy Survey (IPOEMS): a snapshot of the global POEM experience. Surg Endosc 2013;27:3322-38. 6. Friedel D, Modayil R, Iqbal S, et al. Per-oral endoscopic myotomy for achalasia: An American perspective. World J Gastrointest Endosc 2013;5:420-7. 7. Eilers PH, Marx BD. Flexible smoothing with B-splines and penalties. Stat Sci 1996;11:89-102. 8. MacDonald D, Davies R, Henderson GR. Bringing data to life with posthoc CUSUM charts. Case studies in business, industry and government statistics 2010;3:60. 9. Starkie T, Drake EJ. Assessment of procedural skills training and performance in anesthesia using cumulative sum analysis (cusum). Can J Anaesth 2013;60:1228-39. 10. Liao HJ, Dong C, Kong FJ, et al. The CUSUM analysis of the learning curve for endoscopic thyroidectomy by the breast approach. Surg Innov 2014;21:221-8. 11. Haas EM, Nieto J, Ragupathi M, et al. Critical appraisal of learning curve for single incision laparoscopic right colectomy. Surg Endosc 2013;27: 4499-503. 12. Kalteis M, Benedikt P, Huber F, et al. Looking for a learning curve in EVAR based on the Zenith stent graft. Int J Angiol 2012;21:223-8. 13. Li X, Wang J, Ferguson MK. Competence versus mastery: the time course for developing proficiency in video-assisted thoracoscopic lobectomy. J Thorac Cardiovasc Surg 2014;147:1150-4. 14. Ward ST, Mohammed MA, Walt R, et al. An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut 2014;63:1746-54. 15. Inoue H, Tianle KM, Ikea H, et al. Peroral endoscopic myotomy for esophageal achalasia: technique, indication, and outcomes. Thorac Surg Clin 2011;21:519-25. 16. Stavropoulos SN, Desilets DJ, Fuchs KH, et al. Per-oral endoscopic myotomy white paper summary. Surg Endosc 2014;28:2005-19. 17. Cai MY, Zhou PH, Yao LQ, et al. Peroral endoscopic myotomy for idiopathic achalasia: randomized comparison of water-jet assisted versus conventional dissection technique. Surg Endosc 2014;28:1158-65. 18. Akaike, H. A new look at statistical-model identification. IEEE Xplore Digital Library. Automatic Control, IEEE Transactions on. 1974;19:716-23. 19. Schwarz G. Estimating the dimension of a model. Ann Statis 1978;6: 461-4. 20. Dziak JJ, et al. Sensitivity and specificity of information criteria. The Methodology Center and Department of Statistics, The Pennsylvania State University, 2012; 12-25.e1. 21. Page E. Continuous inspection schemes. Biometrika 1954;41:100-15. 22. Cleveland WS. Robust locally weighted regression and smoothing scatterplots. J Am Statistic Assoc 1979;74:829-36. 23. Cleveland WS, Devlin SJ. Locally weighted regression: an approach to regression analysis by local fitting. J Am Statistic Assoc 1988;83: 596-610. 24. Westfall PH, Young SS. P value adjustments for multiple tests in multivariate binomial models. J Am Statistic Assoc 1989;84:780-6. 25. Marascuilo LA. Large-sample multiple comparisons. Psychol Bull 1966;65:280-90. 26. Box GEP, Cox DR. An analysis of transformations. J Royal Statistic Soc 1964;26:211-52; Series B (methodological). 27. Team RDC. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2012. 28. Tsiaoussis J, Athanasakis E, Pechlivanides G, et al. Long-term functional results after laparoscopic surgery for esophageal achalasia. Am J Surg 2007;193:26-31.
Conclusion POEM is a major advance in the treatment of achalasia that offers a less invasive, NOTES approach to Heller myotomy that can be performed in the endoscopy unit with lower resource utilization.16 By allowing for longer lengths of myotomy extending into the proximal body of the esophagus, POEM also offers the promise of superior therapeutic effect in patients with spastic achalasia in comparison to Heller myotomy.16 Given the rarity of achalasia, a relatively short learning curve would be achievable in centers with large referral centers for motility disorders. However, the converse may be true in centers with low volume of referrals. Our study determined a number of procedures for achieving efficiency and mastery that appears realistic for this complex procedure and can be used to guide the efforts of other novice operators, particularly in centers with low referral volumes.
REFERENCES 1. Inoue H, Minami H, Kobayashi Y, et al. Peroral endoscopic myotomy (POEM) for esophageal achalasia. Endoscopy 2010;42:265-71. 2. Teitelbaum EN, Soper NJ, Arafat FO, et al. Analysis of a learning curve and predictors of intraoperative difficulty for peroral esophageal myotomy (POEM). J Gastrointest Surg 2014;18:92-9. 3. Kurian AA, Dunst CM, Sharata A, et al. Peroral endoscopic esophageal myotomy: defining the learning curve. Gastrointest Endosc 2013;77: 719-25.
www.giejournal.org
Volume 81, No. 5 : 2015 GASTROINTESTINAL ENDOSCOPY 1187
