World J Urol DOI 10.1007/s00345-016-1774-x

ORIGINAL ARTICLE

A comparison between an in vitro ureteroscopic stone size estimation and the stone size measurement with the help of a scale on stone baskets Jens Cordes1   · Lisa Teske1 · Felix Nguyen1 · Wolfhard Pinkowski2 · Karl‑Dietrich Sievert1 · Reinhard Vonthein3,4 

Received: 9 April 2015 / Accepted: 25 January 2016 © Springer-Verlag Berlin Heidelberg 2016

Abstract  Introduction  Endoscopic treatment of ureter stones and renal calculi relies on the surgeon’s estimation of the stone size for both lithotripsy and removal of stones or stone fragments. We therefore compared precision and reliability of the endoscopic estimation of stone size by the surgeon with measurements on a scale on a stone basket. Materials and methods  Two surgeons (one high experienced and one low experienced) first estimated, then measured the size of 12 stones differing in size and color using different stone baskets (2.5, 3.0, 4.0 Ch) each via a semirigid renoscope in an artificial ureter under water repeatedly on two different days. All together, we had 288 measurements and 288 estimations. Results  On the whole, the accuracy of the estimation diminished with bigger stones. There is an increasing underestimation with increasing stone size. Factors, which significantly influence the estimation, are the operating surgeon, the color of the stone, the time sequence, and the size of the closed basket, which was held beside the stone. The accuracy of the measurement of the stone baskets is not as good as the estimation. The small 2.5-Ch basket is the most

* Jens Cordes [email protected]; [email protected] 1

Urology Clinic and Policlinic, University Medical Center Schleswig–Holstein (UKSH), Campus Lübeck, University of Lübeck, Ratzeburger Allee 160, 23538 Lübeck, Germany

2

Urotech GmbH, Achenmuehle, Rohrdorf, Germany

3

Institute of Medical Biometry and Statistics, University Medical Center Schleswig–Holstein (UKSH), Campus Lübeck, University of Lübeck, Lübeck, Germany

4

Center for Clinical Trials, University of Lübeck, Lübeck, Germany







accurate in measuring big stones (>6 mm), the 3.5 Ch in intermediate stones (3–6 mm), the big basket (4.0 Ch) in small stones (5 mm in width, a positive history of ureteral surgery, a dilated proximal ureter, kidneys that failed to

13



excrete contrast medium, stones above the ischial spines, and involvement of junior urologist were factors associated with a statistically significant higher incidence of intraoperative complications [3]. In 2013, we established a scale on three automatically fixating stone baskets (2.5, 3.0, 4.0 Ch) [4] (Fig. 1). We now studied the validity, accuracy, and reliability of measurements using stone baskets and surgeon’s estimations via a renoscope in the same artificial ureter model.

Fig. 1  Nonlinear millimeter scales colored like streetlights to indicate extractability for basket sizes indicated

Fig. 2  Prototypes of three different sizes of automatically fixating stone baskets (2.5, 3.0, 4.0 Ch) with colored millimeter scales in sterile packaging

World J Urol

Materials and methods All stone baskets tried were automatically fixating stone baskets with four helical wires of nitinol, as these were considered state of the art, manufactured by Urotech®. Three sizes: 2.5, 3.0, 4.0 Ch, were studied to cover the range needed in practice and to investigate generalizability. The handle was developed by Urotech® together with Prof. S. Lahme (Pforzheim, Germany). This handle is characterized by several unique design elements. It has an automatic fixation of the stone in the basket with a spring mechanism and a dis- and reconnectable handle so that the endoscope can be removed while the retrieval basket is still situated in the ureter and the handle can be reconnected again. We established a nonlinear scale in millimeter on the handle [4] (Fig. 2). The experimental setup (Fig. 3) was closely aligned with the clinical situation as established previously [5]: a metal container was filled with 0.9 % sodium chloride, and a catheter with a diameter of 26 Ch (8.7 mm) was attached as a simulated ureter to the rim of the container. The catheter was wrapped with tape to prevent any scattered light penetrating from outside. This then resulted in an endoscopic image as in the ureter. The respective stone extraction basket was guided by means of a rigid URS video device (11.5 Ch, Wolf), and different pebbles were captured. The entire setup was now inserted into the artificial ureter until the stone extraction basket, which enclosed the stone, and the URS device were completely under water. Stone sizes were chosen to evenly cover the range of sizes one would consider for extraction in toto. Stone colors were chosen to represent the typical range, as the contrast to background is known to influence subjective size assessment. The perception depends on the shape, color, distance to the renoscope, and dilatation of the ureter. This is the so-called binding problem [6],

Fig. 3  Schematic drawing of the experimental setup shows the URS device with one of the stone baskets holding one of the 12 different stones in a 26-Ch catheter under water

13

World J Urol

because shape, color, and direction of motion are processed separately by different population of neurons [7]. Selection of fitting pebbles resulted in a pair of light and dark stones measuring 2  ×  2, 3   ×  2, 4   ×  3, 6   ×  4, and 7  ×  5 mm each and 5  ×  3 and 5  ×  4 mm, respectively. True measures were taken by sliding caliper, repeated, and verified by another person. Stone sizes were categorized as too small to warrant extraction (under 3 mm), perfect for extraction (3–6 mm), and probably too large for extraction (over 6 mm) in some of the analyses. The operating surgeons selected for little (2.5 years) and great (13 years) experience assessed these stones while blinded to true size in a randomized sequence repeated once to assess repeatability. In an identically repeated session about 2 month later, the operating surgeons demonstrated reliability and learning effects. Before a surgeon caught a stone with the basket, he or she had to estimate the stone size. Having caught the stone, he or she identified the direction of the stone in the basket (transverse or longitudinal) to define the reference and measured the stone size using the scale on the stone basket. All together, each surgeon made 144 estimations and 144 measurements.

Table 1  Location and scatter of differences between estimates, measurements, and reference and within and between simulated interventions

Statistics Accuracy was assessed as specified in international standard ISO 5275-1 as the opposite of trueness, bias, estimated by the mean of differences of estimates or measurements and reference true size, and the opposite of precision, estimated by repeatability standard deviation from observations by the same surgeon on the same day and reproducibility standard deviation of observations by a different surgeon on a different day, i.e., after the model was set up once more. Validity was studied by analysis of covariance (ANCOVA) of difference to the reference by factors size, surgeon, basket, color, and day. Reliability and consistency can be judged from the estimated effects. Diagnostic accuracy for indication to extraction was described using relevant stone size categories. Analyses were reported for subgroups of data acquired with baskets of a size. Software JMP 9.0.2 (SAS Inst., Inc., Cary, NC, USA 2010) was used for calculations.

Results All measurements were acquired as planned, although light stones were detected only at a second try in a few instances,

mm

Difference

Basket

Mean

SD

Median

MAD*

Validity, trueness (n = 288)

Measurement—reference

Pooled 2.5 Ch 3.0 Ch 4.0 Ch Pooled 2.5 Ch 3.0 Ch 4.0 Ch Pooled 2.5 Ch 3.0 Ch 4.0 Ch Pooled 2.5 Ch 3.0 Ch 4.0 Ch Pooled 2.5 Ch 3.0 Ch 4.0 Ch Pooled 2.5 Ch 3.0 Ch

0.1 1.1 −0.3 −0.6 −0.7 −0.2 −0.8 −1.1 0.0 −0.1 0.1 0.0 −0.1 −0.2 −0.1 0.1 0.0 0.1 −0.2 0.3 −0.1 0.2 −0.2

1.33 0.99 0.99 1.29 1.06 1.19 0.81 0.90 0.65 0.57 0.66 0.70 0.57 0.51 0.56 0.60 0.90 0.97 0.62 1.00 1.15 1.47 0.87

0 1 0 −0.5 −0.75 −0.5 −0.5 −1 0 0 0 0 0 0 0 0 0 0 0 0 0 0.25 0

1.0 1.3 0.8 1.0 1.0 1.0 0.9 1.2 0.4 0.2 0.4 0.4 0.3 0.3 0.3 0.4 0.6 0.6 0.4 0.8 0.8 1.1 0.7

Estimate—reference

Reliability, repeatability (n   =  288)

Measurement Same session Same surgeon Estimate Same session Same surgeon

Consistency, reproducibility (n   =  144)

Measurement Other day Other surgeon Estimate Other day Other surgeon

4.0 Ch

−0.2

1.00

0

0.7

13



World J Urol

and one prototype had to be repaired after the first few measurements. Trueness, precision, and agreement between estimates, measurements, and reference are given in Table 1. Mean differences between repeated or reproduced assessments were nearly zero. Differences between repeated assessments had SD between ½ and 2/3 mm, while reproduced assessments had SDs close to 1 mm, all much comparable between estimates and measurements. Mean difference to reference was between −1.1 mm for estimates with a 4.0-Ch basket in view and 1.1 mm for measurements using a 2.5-Ch basket.

Table 2  ANCOVAs of estimated and measured stone sizes by surgeon, day, basket, color, reference size, the basket–reference interaction, and individual parabolic learning or fatigue curves across stone assessment number (No.)

Agreement with reference was further investigated by true stone size in an ANCOVA (Table 2; Fig. 4). Both estimation and measurement did not differ much from day to day, but between dark and light stones with measurement halving the 0.5 mm color bias of estimation. Surgeons differed significantly in estimation by 0.6 mm, but not in measurement. Learning curves in estimation were significantly curved, and slopes differed between surgeons, while learning curves in measurement were hardly sloped (0.026 mm/try) essentially linear and did not differ between surgeons (Fig. 5). Slope in reference size, however, was closer to the ideal value of 1 in estimation (0.91)

Estimated stone sizes:

n = 288, s = 0.774

Variable

Difference (95 % CI)

(Intercept) Surgeon (young–old) Day (day 2–day 1) Basket (3.0–4.0 Ch) (2.5–4.0 Ch) (3.0–2.5 Ch) Color (light–dark) Reference (slope) Basket × reference No. (slope) No. × surgeon No.2 (curvature) No.2 × surgeon (Residuals)

−1.08 (−1.67 to −0.49) 0.62 (0.35 to 0.89) 0.007 (−0.17 to 0.19) 1.04 (0.60 to 1.48) 0.62 (0.18 to 1.07) 0.42 (−0.27 to 1.11) 0.51 (0.33 to 0.69) 0.91 (0.85 to 0.98) 0.038 (0.014 to 0.062) 0.032 (0.018 to 0.046) −0.00092 (−0.0019 to 0.0081) 0.0018 (0.0004 to 0.0032)

Measured stone sizes:

n = 288, s = 0.727

Variable

Difference (95 % CI)

(Intercept) Surgeon (young–old) Day (day 2–day 1) Basket (3.0–4.0 Ch) (2.5–4.0 Ch) (3.0–2.5 Ch) Color (light–dark) Reference (slope) Basket × reference No. (slope) No. × surgeon No.2 (curvature) No.2 × surgeon

1.54 (0.99 to 2.09) −0.14 (−0.39 to 0.12) 0.16 (−0.01 to 0.33)

(Residuals)

0.73 (0.32 to 1.14) 1.34 (0.92 to 1.76) −0.61 (−1.26 to 0.04) 0.27 (0.10 to 0.44) 0.44 (0.38–0.50) 0.026 (0.004 to 0.048) 0.003 (−0.011 to 0.017) −0.00032 (−0.0013 to 0.0006) 0.000038 (−0.0012 to 0.0014)

R2 = 0.789 df

1 1 2

1 1 2 1 1 1 1 275

F ratio

P value

12.2190 0.0035 22.1641

20.3972 0.0058 18.4993

9.3 × 10−6* 0.9394 2.9 × 10−8*

18.3592 459.9789 8.5427 6.1020 10.7804 1.9128 3.7473 164.7396

30.6470 767.8434 7.1302 10.1861 17.9958 3.1930 6.2554

7.2 × 10−8* 1.4 × 10−81* 0.0010* 0.0016* 3.03 × 10−5* 0.0751 0.0130*

R2 = 0.647 SS

F ratio

P value

1 1 2

0.5601 1.8135 34.3829

1.0600 3.4321 32.5348

0.3041 0.0650 2.08 × 10−13*

1 1 2 1 1 1 1

5.2040 109.6092 3.5588 2.8666 0.0973 0.2293 0.0017

9.8485 207.4357 3.3676 5.4251 0.1842 0.4339 0.0032

0.0019* 1.99·10−35* 0.0359* 0.0206* 0.6681 0.5106 0.9547

275

145.3102

df

CI confidence interval, df degrees of freedom, SS sum of squares

13

SS

World J Urol

than in measurement (0.44). Differences between baskets were in the range of ½–1 mm, and the interaction with reference size was significant in both measurement and estimation. As a rule, stones were underestimated, and smaller baskets and measurements produced less bias. The larger the stone, the more did the measurements fall short of the reference (Fig. 4). The extent of the most prominent biases relative to imprecision is illustrated in plots of trueness by factor levels (Fig. 5). Clinical relevance of assessments rests with the decision to extract in toto. This was assumed to change at 3 and 6 mm. The resulting misclassifications are shown in Fig. 6.

Fig. 4  Mean estimated (blue) and measured (red) stone size by true size and basked used: 2.5 Ch (dotted), 3.0 Ch (dashed), 4.0 Ch (solid line), and line of perfect agreement (strong black line)

Fig. 5  Differences to reference for measured (above) and estimated (below) stone sizes by reference, by basket used, and by stone color with box plots. Symbols indicate baskets: +2.5 Ch, ×3.0 Ch, square

These may be summarized by statements like: Sensitivity and specificity of estimation to stones of 6 mm or more are 50 % (24/48) and 98 % (234/240), while the respective numbers are 56 % (5/9) and 84 % (73/87) for measurements using the 2.5-Ch basket.

Discussion The present accuracy study was detailed enough to quantify the respective advantages of estimation and measurement of stones using baskets. As sizes were usually given to the full millimeter, which was the unit on the measuring scale also, biases and standard or mean absolute deviations of 1 mm should certainly raise concern. The current practice of estimation produces such biases and deviations between light and dark stones and between surgeons, and such differences may change over time to that extent. The proposed practice of measuring stones using baskets eliminates these biases to a large extent as desired. What could not be bettered was the bias introduced by using a specific basket. That basket bias depended on the true stone size, which complicates things. While estimated stone sizes tended to be too low by a millimeter, measurement overestimated small stones—when the 2.5-Ch basket was used at least—and underestimated large stones. As a consequence, diagnostic accuracy was not better than accuracy of estimation. An obvious conclusion is that the scale on the baskets needs some more calibration. That may prove harder than taking more measurements. The basket bias could be a result of the mismatch of the steel spring and the nitinol

4.0 Ch, and colors indicate day. Horizontal lines indicate group means (green) and grand mean (dashed)

13



World J Urol

Fig. 7  Stress and strain curve of nitinol and stainless steel: at the beginning, steel is not flexible and shows increasing stress whereas nitinol shows more strain and less stress in the beginning and even less stress during the reverse action [8]

Fig. 6  Diagnostic value as mosaic plots of size classes by assessment (pooled estimates above, measurements by baskets below) versus reference. Absolute frequencies are printed in rectangles of proportional size colored green for correct, blue for over-, and red for underestimates

basket. The reason for this can be found in the different strain conditions due to the difference in relation to stress and strain of steel and nitinol (Fig. 7). While steel shows an increase in stress in the beginning, nitinol shows an increase in strain. The scales also differ with the size of the basket (Fig. 3), because we used the same handle with the same force of the spring with different diameters of the basket wires, so that thinner nitinol is more deformed. Furthermore, the closed position of the slider is located in different positions [4]. In the future, the baskets should be improved, maybe with a spring made of nitinol.

13

Our investigation of the estimation confirms the study of Patel et al. [9] which states that endourologists are able to intraoperatively assess accurately residual stone fragment size to guide decision making. In our study, we find a slight tendency in underestimating big stones so that for these bigger stones an improved measurement of stone size could be helpful. This is the first study which shows influence factors like the color, which significantly shows that dark stones are generally estimated smaller than bright stones. This study also shows that the experienced surgeon does not necessarily estimate stone sizes better than an unexperienced surgeon and that the thickness of the device next to the stone is a significant factor. Limitations of the study are that other clinical settings like the ureteral caliber, anatomical variations, or the use of an access sheath are not evaluated in our study.

Conclusion This is the first attempt of validation of a scale on stone baskets. It shows different results for each basket, which could be systematically improved. Until now, the estimation of the surgeons is more exact than the measurement

World J Urol

using the scale, although estimation is biased by stone color, basket size, and surgeon. Compliance with ethical standards  Conflict of interest None.

References 1. Preminger GM, Tiselius HG, Assimos DG, Alken P, Buck AC, Gallucci M, Knoll T, Lingeman JE, Nakada SY, Pearle MS, Sarica K, Türk C, Wolf JS Jr (2007) Guideline for the management of ureteral calculi. Eur Urol 52(6):1610–1631 2. Breda A, Ogunyemi O, Leppert JT, Lam JS, Schulam PG (2008) Flexible ureteroscopy and laser lithotripsy for single intrarenal stones 2 cm or greater—is this the new frontier. J Urol 179(3):981–984

3. Abdelrahim AF, Abdelmaguid A, Abuzeid H, Amin M, Mousa ES, Abdelrahim F (2008) Rigid ureteroscopy for ureteral stones: factors associated with intraoperative adverse events. J Endourol 22:277–280 4. Cordes J, Nguyen F, Pinkowski W, Jocham D (2013) Measurement of stone diameter with three sizes of automatically fixating stone baskets. Open J Urol 3:58–61 5. Cordes J, Lange B, Jocham D, Kausch I (2011) Destruction of stone extraction basket during an in vitro lithotripsy: a comparision of four lithotripters. J Endourol 25(1–4):58–61 6. Treisman A (1996) The binding problem. Curr Opin Neurobiol 6:171–178 7. Masahiko M, Shigemitsu M, Hiromi M (2010) Attribute pairbased visual recognition and memory. PLoS ONE 5:e9571 8. Stoeckel D (2001) Umformung von NiTi-Legierungen - Einen Herausforderung. Neuere Entwicklungen in der Massivumformung Siegert K (Hrsg.) 141–157 9. Patel N, Chew B, Knudsen B, Lipkin M, Wenzler D, Sur RL (2014) Accuracy of endoscopic intraoperative assessment of urologic stone size. J Endourol 28:582–586

13