Strabismus, 2014; 22(4): 158–166 ! Informa Healthcare USA, Inc. ISSN: 0927-3972 print / 1744-5132 online DOI: 10.3109/09273972.2014.972521
ORIGINAL ARTICLE
Inter-examiner variability and agreement of the alternate prism cover test (APCT) measurements of strabismus performed by 4 examiners* E. de Jongh,
1
MSc, CO
1
, C. Leach, MSc, DBO(T)2, M.J. Tjon-Fo-Sang, A. Bjerre, MSc, BSc(Hons)2
MD, PhD
1
, and
Orthoptic Department, Rotterdam Eye Hospital, the Netherlands and 2Academic Unit of Opthalmology and Orthoptics, University of Sheffield, United Kingdom
ABSTRACT Purpose: To investigate the inter-examiner variability and agreement of the alternate prism cover test (APCT) measurements of strabismus at near and distance fixation performed by 4 examiners. Methods: Forty-one participants (median age 12 years, range 3–74) with horizontal strabismus completed APCT measurements at near (1/3 m) and distance (5 m) fixation. Each participant was assessed by 4 masked experienced orthoptists on the same visit. Bland-Altman plots and inter-examiner variability (1.96xSD of the difference) were used to determine a clinical value suggestive of real change. Results: The inter-examiner variability was ±10 prism diopters (PD) at near and ±9 PD at distance fixation. The mean difference between measurements per examiner pair for near fixation ranged between 3 and 1.5 PD, with inter-examiner variability ranging between ±6.9 and ±12.5 PD, and mean difference for distance fixation between 0.8 and 1.6 PD, with inter-examiner variability ranging between ±7.5 and ±11.7 PD. Larger variability was found between some examiner pairs than others. Magnitude of the deviation, test distance, and age had no significant influence on the variation in APCT measurement. Conclusion: A variation in APCT measurement between 4 examiners of less than 10 PD for both near and distance fixation is likely to be due to inter-examiner variability. Changes of 10 PD or more are suggestive of a real change and may be employed as a management threshold. Keywords: Inter-examiner, prism, reliability, strabismus
INTRODUCTION
measurement of strabismus, such as examiner experience, examiner bias, positioning of the examiner and target, dissociative techniques, test being used (eg, APCT and simultaneous prism and cover test), neutralization endpoint employed, calibration and position of the prism in front of the eye, magnitude of deviation (eg, 20 PD vs420 PD), test distance, age, and type of strabismus.1–10 A number of studies have reported inter-examiner variability and agreement in children and adults comparing 2 examiners.1,2,8,10,11 In large eye clinics more than 2 examiners often measure the angle of deviation on a patient on separate visits. The inter-examiner variability and agreement between more than 2 examiners assessing children and
In patients with strabismus, the angle of deviation is routinely measured at clinical visits to evaluate stability. Monitoring the angle of deviation on consecutive visits allows identification of deterioration and aids in the decision whether strabismus surgery is needed. The amount of strabismus surgery performed on the extraocular muscles largely depends on the angle of deviation measured, commonly performed with the alternate prism cover test (APCT). Variation in measurements between clinical visits can occur. Several factors have been reported to play a role in the
*Presented at the European Strabismological Association, Marseille, France, September 4–6, 2013. Received 29 October 2013; Revised 29 September 2014; Accepted 30 September 2014; Published online 31 October 2014 Address for correspondence: E. de Jongh, Schiedamse Vest 180, 3011 BH, Rotterdam, The Netherlands. E-mail: [email protected]
158
The inter-examiner variability of the alternate prism and cover test 159 adult patients is however unknown. The purpose of this study was to establish this by including 4 experienced examiners and provide guiding values for clinicians as to what can be considered a true change in the size of strabismus in adults and children.
MATERIALS AND METHODS Patients with horizontal strabismus attending the Orthoptic Department of the Rotterdam Eye Hospital, Rotterdam, the Netherlands were recruited to participate in this study. Ethical approval was sought and obtained at the Medisch Ethische Toetsings Commissie (METC, Medical Ethical Committee, Rotterdam, the Netherlands). This study was conducted in accordance with the principles of the Declaration of Helsinki (October 2008), the guideline for Good Clinical Practice (CPMP/ICH/135/95), and the Medical Research Involving Human Subjects Act (WMO; 2006). Each participant or parent gave written informed consent before participating in this study. Eligibility criteria included isolated horizontal strabismic deviations (including latent, intermittent, and manifest deviations) or combined horizontal and vertical deviations (horizontal angle larger than the vertical). All participants had to wear their full refractive correction. In children aged 512 years, this was controlled by only including those who had cycloplegic refraction, within 6 months before participating in this study. Both eyes were able to fixate centrally in primary position with a visual acuity (VA) of at least 20/200 in either eye with full refractive correction. Participants with eccentric fixation or a dissociated vertical deviation were excluded. Each participant was seated at the same clinical examination station while 4 examiners rotated to ensure consistency of environmental circumstances. Each participant was assessed by 4 masked orthoptists with at least 3 years of working experience. To control for order effects, the order of examiners was determined from a counterbalancing table of possible combinations of 4 examiners. For each participant, two examiners first performed near followed by distance fixation measurements; the other two examiners performed measurements in reversed order. Examiners were masked with respect to the results of previous measurements and were not able to exchange information. An APCT instruction leaflet was given to all examiners to ensure standardization of measurements and each examiner completed a predesigned standardized case report form. Measurements were performed within 40 minutes with a 5-minute rest period between measurements to minimize the influence of the previous measurement and bias induced by fatigue. A Snellen optotype was used as standardized fixation target in each participant. Participants were !
2014 Informa Healthcare USA, Inc.
asked to fixate the smallest target they could see clearly with their worse eye. The prism bars used were the plastic Gulden prism bars, (Horizontal B-15 and Vertical B-14) calibrated for frontal plane position. All examiners were provided with a horizontal prism bar with the units 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 25, 30, 35, and 40. The vertical prism bar contained the units 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 16, 18, 20, and 25. A loose isosceles prism of 45 PD was part of the prism set. When the deviation exceeded 45 PD, such that a single prism could not be used, prisms were split between eyes. The largest prism, as close as equal as possible, was placed in front of the non-fixing eye to minimize the shift in primary position. In case of a combined horizontal and vertical deviation, simultaneously used prism bars were divided between the two eyes. Because stacking of prisms with bases in the same direction can lead to erroneous measurement, this was not allowed in this study.6 Care was taken not to tilt or rotate the prism. Deviations, based on the fixation of preference, were recorded as the value of the prism that reached the first neutral endpoint. To ensure full dissociation, the cover paddle was held as closely as possible to the patient’s eye in combination with the ‘‘long dissociation protocol’’ of Anderson et al.2 The cover paddle was initially held in front of the non-dominant eye for 5 seconds. ‘‘The eye was alternately occluded for a count of 2 seconds for nine additional excursions of the cover paddle. The right and the left eye were covered five times each, leading to a total dissociation time of 23 seconds,’’ (p. 170).2 Cooperation of participants aged 512 years was rated during each of the 4 measurements by every examiner. A 5-point coping scale was used, modified from the Distress and Approach-Avoidance score scale of Blount et al.12 The scale ranged from extreme distress/agitation/screaming (score 1) to no distress/ calm/no crying (score 5). The higher scores indicate more co-operative behavior.12
Statistical Analysis Different prism increments were used to measure smaller (2-20 PD) and larger (420 PD) horizontal angles of strabismus due to the design of the horizontal prism bar. Pairs of measurements in 2 strata were analyzed: one where the prism increment used for measurement was 2 PD (angles 20 PD) and a second where the prism increment was 5 PD (420 PD). If measurements fell into both groups, the values were analyzed in the largest increment stratum. Median and range were calculated due to the Kolmogorov-Smirnov test showing that the data was non-normally distributed. Multiple regression analysis was used to assess the relationship of variables (age, type of deviation, magnitude of the deviation, cooperation) on the variation in APCT measurements
160 De Jongh et al. between examiners. Analysis of variability and agreement between examiners comprised calculating the mean and standard deviation (SD) of the difference between measurements of all possible combinations of examiners. Pairwise, inter-examiner agreement between measurements was evaluated by means of Bland-Altman plots.13 Differences between the measurements were plotted against the mean of the two measurements. The inter-examiner variability (1.96 SD) were used to determine a clinical value likely to indicate real change. The agreement between examiners was expressed by the percentage of agreement and corresponding 95% confidence intervals (CI). P values 50.05 were considered as statistically significant and differences of at least 2 PD were considered as clinically significant.14 All reported P values were 2-tailed. The following program was used to test if a difference between two percentages was statistically significant (http://www.vassarstats.net/propdiff_ind.html).
RESULTS APCT measurements at near and distance were completed in 41 participants with horizontal strabismus (21 males and 20 females). The median age was 12 years (range 3–74 years). Twenty-five participants had an exo-deviation and 16 an eso-deviation. More than half of the number of participants had a manifest deviation (n = 22), followed by intermittent (n = 13) and latent deviations (n = 6). The median cooperation was ‘‘fine’’ (score 4, range 1–5) and corresponded with a median age of 7 years. The cooperation of all the children remained stable during the 4 APCT measurements performed by the 4 examiners at the same visit. A positive correlation was found between age in years and cooperation in participants 512 years (Pearson’s correlation coefficient r = 0.62; p = 0.01). The median distance Snellen VA in the right eyes was 6/7.5 (range 6/15–6/4) and 6/6 in the left eyes (range 6/15–6/4.6) and near VA in the right eyes was 6/9 (range 6/36–6/6) and 6/9 in the left eyes (range 6/18–6/6). The median fixation target size used was 6/6 at distance and 6/9 for near, based on the VA of the worst eye. The APCT measurements were performed by the 4 examiners on all of the 41 participants for both near and distance fixation. The median horizontal near deviation measured 20 PD (range 4–53) and 18 PD (range 0–47) at distance fixation. Table 1 shows the near and distance measurements per examiner. The 95% limits of agreement for participants measuring 20 PD were ±9.4 PD at near and ±6.1 PD at distance fixation. For deviations measuring420 PD, they were ±9.9 PD and ±10.3 PD at near and distance fixation respectively. Larger variability in APCT measurements was found in participants aged 512 years
TABLE 1. The median measurement per examiner for near and distance fixation given in PD (N = 41). Near
Examiner 1 Examiner 2 Examiner 3 Examiner 4 All 4 examiners
Distance
Median
Range
Median
Range
20.0 20.0 20.0 20.0 20.0
4–49 6–53 4–45 4–51 4–53
18.0 18.0 18.0 16.0 18.0
2–45 0–47 0–45 0–45 0–47
than 12 years for both near (±10.5 vs ±8.7 PD) and distance (±9.1 vs ±6.8 PD) fixation. Multiple regression analysis was performed to determine if variables such as age, type of deviation, magnitude of deviation, and cooperation influenced the variation in APCT measurements. 1.9% of the variation in near APCT measurements (difference in APCT measurements between examiners) was caused by these variables (R2 0.02). The standardized regression coefficients showed that none of the variables had significant influence on the variation in APCT measurements. The same variables induced 3% of the variation in APCT measurements at distance (R2 0.03), without any significant influence on the variation in APCT measurements (p50.05). Due to the lack of relationship between the above-described variables and variation in APCT measurements, only data analyses including all the participants will be reported in the following. Comparison of APCT measurements for each examiner against the average deviation of all examiners was performed. Figures 1 through 4 display the under- and overestimation of APCT measurements for each examiner for near and distance fixation. Examiners 1 and 2 showed larger differences (95% limits of agreement) for near measurements (±5.9 PD and ±8 PD) compared with examiners 3 and 4 (±5.6 PD and ±4.9 PD). Distance measurements were more alike in all examiners (range 4.2 to 5.1 PD). Besides comparing APCT measurements of each examiner against the average deviation, differences in APCT measurements between examiners were analyzed. Tables 2 and 3 show the differences in APCT measurements for each examiner pair. In Figure 5, the differences between all examiner pairs are plotted against the mean deviation. Comparison of all horizontal APCT measurements for near fixation between the examiners showed the mean difference between two measurements was 0.59 PD (95% CI = 0.55 to 0.67) with inter-examiner variability of ±10 PD (Figure 5a), and 0.21 PD (95% CI = 0.39 to 0.63) with inter-examiner variability of ±9 PD for distance fixation (Figure 5b). Greater variability in measurements was found for near compared to distance fixation, but this was not Strabismus
The inter-examiner variability of the alternate prism and cover test 161
FIGURE 1. Under- and overestimating of APCT measurements by examiner 1 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±5.9 PD for near fixation and ±4.3 PD for distance fixation. The solid line shows the mean of the differences to be 0.47 vs 0.62 PD for near and distance fixation, respectively.
statistically significant (p = 0.58). The mean difference between measurements per examiner pair for near fixation ranged between 0.8 and 1.6 PD with interexaminer variability ranging between ±7.5 and ±11.7 PD, and mean difference for distance fixation between 3 and 1.5 PD with inter-examiner variability ranging between ±6.9 and ±12.5 PD (Tables 2 and 3). The percentage of agreement based on all examiner pairs within the ranges of ±2, ±4, ±6, ±8 and ±10 PD are shown in Table 4. Agreement within 2 PD between all 4 examiners was 51% for near and 57% for distance fixation and increased to 96% for near and 98% for distance fixation within 10 PD. A difference of at least 2 PD has been suggested as clinically significant14 and therefore subsequent analysis between examiner pairs evaluated agreement within 2 PD only. The percentage agreement of obtaining the same measurement between two examiners within 2 PD ranged between 39% to 66% for near and 37% to 70% for distance fixation, respectively (Table 5). Including all 4 examiners no significant difference (p = 0.17) was found in percentage agreement within 2 PD for near against distance fixation. Most of the examiner pairs showed the same result except for examiner pair 1 vs 2, where the percentage of agreement within 2 PD was significantly larger for distance fixation than near fixation (p50.01). Distance measurements were more alike compared with near angles. On the contrary, for examiner pair 3 vs 4 near measurements were more alike compared to distance angles. !
2014 Informa Healthcare USA, Inc.
TABLE 2. Inter-examiner variability of all APCT horizontal measurements at near fixation.
Examiner 1 vs 2 Examiner 1 vs 3 Examiner 1 vs 4 Examiner 2 vs 3 Examiner 2 vs 4 Examiner 3 vs 4 All 4 examiners
Mean
SD
Inter-examiner variability (±1.96 SD)
0.2 1.5 0.6 1.6 0.8 0.8 0.6
6.0 4.7 4.3 5.8 5.5 3.8 5.1
11.7 9.3 8.4 11.3 10.7 7.5 10.0
95% limits of agreement 11.8 to 11.5 7.8 to 10.7 7.8 to 9.0 9.7 to 12.9 9.9 to 11.5 8.3 to 6.6 9.4 to 10.3
TABLE 3. Inter-examiner variability of all APCT horizontal measurements at distance fixation.
Examiner 1 vs 2 Examiner 1 vs 3 Examiner 1 vs 4 Examiner 2 vs 3 Examiner 2 vs 4 Examiner 3 vs 4 All 4 examiners
Mean
SD
Inter-examiner variability (±1.96 SD)
0.3 1.5 0.8 1.2 0.4 3.0 0.2
3.8 3.9 3.5 4.2 3.9 6.4 4.6
7.5 7.7 6.9 8.3 7.7 12.5 9.0
95% limits of agreement 7.2 to 7.8 6.2 to 9.2 6.1 to 7.7 7.1 to 9.5 7.3 to 8.2 15.4 to 9.5 9.8 to 10.2
The order of measurements did not influence the variation in APCT measurements for near and distance fixation. The inter-examiner variability of the first measurement was similar to the fourth measurement at near (±5.9 PD vs ±5.6 PD) and distance fixation (±4.8 PD vs ±4.5 PD).
162 De Jongh et al.
FIGURE 2. Under- and overestimating of APCT measurements by examiner 2 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±8 PD for near fixation and ±4.8 PD for distance fixation. The solid line shows the mean of the differences to be 0.62 vs 0.30 PD for near and distance fixation, respectively.
FIGURE 3. Under- and overestimating of APCT measurements by examiner 3 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±5.6 PD for near fixation and ±5.1 PD for distance fixation. The solid line shows the mean of the differences to be 1 vs 0.90 PD for near and distance fixation, respectively.
DISCUSSION This study investigated inter-examiner variability and agreement of APCT measurements in children and adults with horizontal strabismus performed by 4 experienced examiners. The inter-examiner variability
between APCT measurements performed by 2 examiners ranged from 6.9 to 12.5 PD, depending on the examiner pair and fixation distance. The findings show that it is important to compare more than two examiners because the APCT measurements between examiners were variable per examiner pair and even Strabismus
The inter-examiner variability of the alternate prism and cover test 163
FIGURE 4. Under- and overestimating of APCT measurements by examiner 4 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±4.9 PD for near fixation and ±4.6 PD for distance fixation. The solid line shows the mean of the differences to be 0.16 vs 0.14 PD for near and distance fixation, respectively.
FIGURE 5. Inter-examiner variability of all APCT horizontal measurements at near fixation and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±10 PD for near fixation (A) and ±9 PD for distance fixation (B). The solid line shows the mean of the differences to be 0.59 vs 0.21 PD for near and distance fixation, respectively.
more so if all 4 examiners were compared. This is of clinical relevance because in many eye clinics multiple examiners are likely to perform the APCT measurements on a patient on consecutive visits and thus will have an impact on the clinical decision, whether the strabismus is remaining stable, improving, or deteriorating. !
2014 Informa Healthcare USA, Inc.
A number of studies have evaluated inter-examiner variability between 2 examiners to establish when a difference in measurements of angle of deviation may be considered a true change.1,2,8,10,11 Holmes et al.1 reported inter-examiner variability of the APCT in adults with sixth nerve palsy and suggested a change in measurements of 10 PD or more was suggestive
164 De Jongh et al. TABLE 4. Percentage agreement based on the differences in APCT measurements between all 4 examiners within a range of ±2, ±4, ±6, ±8, and ±10 PD for near and distance fixation. Difference within (PD)
Near (%)
Distance (%)
51 59 85 91 96
57 68 87 92 98
±2 ±4 ±6 ±8 ±10
TABLE 5. Percentage agreement based on the differences in APCT measurements between all examiner pairs within a range of ±2 PD for near and distance fixation. The P values demonstrate if the difference in agreement between near and distance fixation was significant.
Examiner 1 vs 2 Examiner 1 vs 3 Examiner 1 vs 4 Examiner 2 vs 3 Examiner 2 vs 4 Examiner 3 vs 4 All 4 examiners
Near fixation
Distance fixation
P value
39% 51% 56% 41% 54% 66% 51%
70% 54% 63% 59% 61% 37% 57%
50.01 0.83 0.50 0.12 0.50 0.01 0.17
of a true change and any less may be attributable to test-retest variability. The Pediatric Eye Disease Investigator Group (PEDIG) also suggested that in case of differences within the guiding value of 10 PD, one would not be able to distinguish what component of the difference is measurement error and what component is real change.8 PEDIG investigated children aged 60 months with esotropia. Their findings suggested that a difference of 6 PD or more for esotropias measuring between 10 and 20 PD and that a difference of 12 PD for deviations measuring greater than 20 PD may be considered a true change. Hatt et al.15 examined children aged 1–13 years with intermittent exotropia. They reported threshold values for what may be considered a real change for deviations measuring 20PD or less and found values of 7PD and 3PD for near and distance fixation respectively. The present study showed slightly larger interexaminer variability for near angles compared with distance angles. Bland-Altman analysis showed a value of 10 PD was suggestive of real change at near fixation vs 9 PD for distance fixation. These values were based on the average of the measurements of 4 examiners. It should be noted that better or worse results were found dependent on different examiner pairs. Only examiner pair 1–2 showed significantly larger inter-examiner variability for near fixation compared with distance fixation (11.7 vs 7.5 PD). Examiner pair 3–4 showed significantly larger interexaminer variability for distance fixation compared
with near fixation (12.5 vs 7.5 PD) (Tables 2 and 3). The other examiner pairs showed a slightly larger inter-examiner variability for APCT measurements performed at near fixation. Accommodation and tenacious proximal fusion16 are factors that induce variation in APCT measurements, which is the reason why many studies8 found greater inter-examiner variability for near angles compared with distance angles. All subjects in the present study were wearing their full refractive correction to minimize variability in measurement due to accommodation. To control the accommodative driven vergence, a standardized accommodative fixation target and instructions to keep the target clear were given. Although the smallest target the worse eye could see was used in each subject, it is still possible that fine focusing during the APCT could have interfered with staring and not accommodating. This still does not explain why there was more interexaminer variability for measurements performed by examiner pair 3–4. In most studies, larger limits of agreement were found for near deviations compared with distance deviations and in measurements of large angles compared with small angles.1,8,10,16 The present study shows larger 95% limits of agreement at near fixation vs distance fixation for participants measuring 20 PD (±9.4 PD vs ±6.1 PD). For deviations measuring 420 PD, there was no clinically significant difference (±9.9 PD vs ±10.3 PD) at near and distance fixation, respectively. These values are based on the average of the measurements of 4 examiners and more or less test-retest inter-examiner variability is dependent on different examiner pairs. However, the variable magnitude of the deviation did not significantly influence the measurement variability between examiners. It is possible that with a larger sample size and consequently larger groups of specific types of deviation, magnitude of deviation and age group relationships may have been found. Because the increment steps in the present study were 5 PD for large angles, which is double the size of the prisms used in the PEDIG study,8 larger interexaminer variability was expected in this study. This was however not the case. Despite the larger mean deviation measured by PEDIG8 (37.2 PD ± 14.8 at near and 32.5 PD ± 14.3 at distance fixation) and the difference in prism increments, our results are comparable. It is possible the larger inter-examiner variability in large deviations reported by PEDIG was caused by the magnitude of the deviation and strength of prism, as described by Firth and Whittle,6 rather than caused by the factor prism increment. A slight tilt of a large prism can lead to measurement errors.7 The age of the subjects in the PEDIG study was younger than the present study. The present study explored whether younger children’s attention and ability to fixate well was less and Strabismus
The inter-examiner variability of the alternate prism and cover test 165 could led to greater inter-examiner variability. Larger measurement variability was seen in participants aged 512 years compared with 12 years. Although these values were larger in those 512 years, there was only a clinical difference in inter-examiner variability at distance fixation (±9.1 vs ±6.8 PD). However, standardized regression coefficients showed that age had no significant influence on the variation in APCT measurement. The median cooperation of participants aged 512 years was determined to be fine (score 4) in this study, but there was a significant positive correlation between cooperation and age in those 512 years, which confirms the impression that the older the child, the better the cooperation. Nevertheless the cooperation level of participants aged 512 years remained the same during the APCT measurements by the 4 examiners. A larger sample size is needed to confirm this finding.
SUMMARY One should be aware that more or less test-retest inter-examiner variability is dependent on different examiner pairs. A variation in APCT measurement between 4 examiners of less than 10 PD for both near and distance fixation is likely to be due to interexaminer variability. Magnitude of the deviation, test distance, and age had no significant influence on the variation in APCT measurement. Changes of 10 PD or more are suggestive of a true change and may be employed as a management threshold for defining stability, improvement, or deterioration.
ACKNOWLEDGMENTS This study was conducted at the Orthoptic Department of the Rotterdam Eye Hospital, The Netherlands.
Limitations of the Study DECLARATION OF INTEREST Using 4 examiners, it is important to control for order effects. Larger inter-examiner variability was expected during the last measurement due to fatigue and decompensation of the deviation. This study showed that the last measurement did not have the largest angle, but the type of deviation did change due to decompensation. For example, an intermittent deviation became manifest but the angle did not increase. The fact that Hatt et al.15 found less inter-examiner variability in their study investigating children with intermittent exodeviations suggests that the intermittent nature of some of the deviations did not affect the limits of agreement. This study employed a specific protocol for measurement of the deviation with a prespecified number of cover-uncovers and dissociation time. This standardized method is beyond routine clinical practice and raises the question whether the results can be generalized. Interestingly, our results produced similar differences between measurements as reported in previous studies including 2 examiners.1,8 With a less standardized protocol, a larger variation may have been found between 4 examiners. The results from this study support that the APCT is fairly reliable between examiners. It is however not possible to control all variables. It is therefore important to confirm stability of the ocular misalignment before performing strabismus surgery. PEDIG17 reported that a minimum of 3 identical measurements or 4 measurements within 5 PD would be needed to confirm stability. What should be kept in mind is that any measurement of strabismus is the sum of the true deviation and measurement error. It is not clear what component of the difference is measurement error and what component is real change.8 !
2014 Informa Healthcare USA, Inc.
The authors have no financial conflict of interest.
REFERENCES 1. Holmes JM, Leske DA, Hohberger GG. Defining real change in prism-cover test measurements. Am J Ophthalmol 2008;145:381–385. 2. Anderson HA, Manny RE, Cotter SA, et al. Effect of examiner experience and technique on the alternate cover test. Optom Vis Sci 2010;87:168–175. 3. Hrynchak PK, Herriot C, Irving EL. Comparison of alternate cover test reliability at near in non-strabismus between experienced and novice examiners. Ophthalmic Physiol Opt 2010;30:304–309. 4. Johnson R, Wynn S, Coffey B. Influences of examiner position and effective prism power on nearpoint alternate cover test. Optometry 2004;75,8:496–502. 5. Flu¨eler UR, Elhatton KM, Guyton DL. A combination horizontal/vertical prism bar. A precisely calibrated tool for use in front of one eye. Strabismus 1995;3:27–32. 6. Firth AY, Whittle JP. Clarification of the correct and incorrect use of ophthalmic prisms in the measurement of strabismus. Br Orthopt J 1994;51:15–18. 7. Thompson JT, Guyton DL. Ophthalmic prisms: Measurement errors and how to minimise them. Ophthalmology 1983;90,3:204–210. 8. Pediatric Eye Disease Investigator Group. Interobserver reliability of the prism and alternate cover test in children with esotropia. Arch Ophthalmol 2009;127:59–65. 9. Schutte S, Polling JR, van der Helm FCT, Simonsz HJ. Human error in strabismus surgery: Quantification with a sensitivity analysis. Graefes Arch Clin Exp Ophthalmol 2008; 247:399–409. 10. Rainey BB, Schroeder TL, Goss DA, Grosvenor TP. Reliability of and comparisons among three variations of the alternating cover test. Ophthalmic Physiol Opt 1998;18: 430–437. 11. Johns HA, Manny RE, Fem K, Hu YS. The intraexaminer and interexaminer repeatability of the alternate covertest
166 De Jongh et al. using different prism neutralization endpoints. Optom Vis Sci. 2004;81:939–946. 12. Blount RL, Cohen LL, Frank NC, et al. The Child-Adult Medical Procedure Interaction Scale-Revised: An assessment of validity. J Pediatr Psychol 1997;22:73–88. 13. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135–160. 14. Fogt N, Baughman BJ, Good G. The effect of experience on the detection of small eye movements. Optom Vis Sci 2000; 77:670–674.
15. Hatt SR, Leske DA, Liebermann L, et al. Variability of angle of deviation measurements in children with intermittent exotropia. J AAPOS 2012;16:120–124. 16. Yang HK, Hwang J-M. The effect of target size and accommodation on the distant angle of deviation in intermittent exotropia. Am J Ophthalmol 2011;151:907–913. 17. Melia BM, Holmes JM, Chandler DL, Christiansen SP; Pediatric Eye Investigator Group. Classifying stability of misalignment in children with esotropia using simulations. Arch Ophthalmol 2010;128:1555–1560.
Strabismus
Copyright of Strabismus (09273972) is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
ORIGINAL ARTICLE
Inter-examiner variability and agreement of the alternate prism cover test (APCT) measurements of strabismus performed by 4 examiners* E. de Jongh,
1
MSc, CO
1
, C. Leach, MSc, DBO(T)2, M.J. Tjon-Fo-Sang, A. Bjerre, MSc, BSc(Hons)2
MD, PhD
1
, and
Orthoptic Department, Rotterdam Eye Hospital, the Netherlands and 2Academic Unit of Opthalmology and Orthoptics, University of Sheffield, United Kingdom
ABSTRACT Purpose: To investigate the inter-examiner variability and agreement of the alternate prism cover test (APCT) measurements of strabismus at near and distance fixation performed by 4 examiners. Methods: Forty-one participants (median age 12 years, range 3–74) with horizontal strabismus completed APCT measurements at near (1/3 m) and distance (5 m) fixation. Each participant was assessed by 4 masked experienced orthoptists on the same visit. Bland-Altman plots and inter-examiner variability (1.96xSD of the difference) were used to determine a clinical value suggestive of real change. Results: The inter-examiner variability was ±10 prism diopters (PD) at near and ±9 PD at distance fixation. The mean difference between measurements per examiner pair for near fixation ranged between 3 and 1.5 PD, with inter-examiner variability ranging between ±6.9 and ±12.5 PD, and mean difference for distance fixation between 0.8 and 1.6 PD, with inter-examiner variability ranging between ±7.5 and ±11.7 PD. Larger variability was found between some examiner pairs than others. Magnitude of the deviation, test distance, and age had no significant influence on the variation in APCT measurement. Conclusion: A variation in APCT measurement between 4 examiners of less than 10 PD for both near and distance fixation is likely to be due to inter-examiner variability. Changes of 10 PD or more are suggestive of a real change and may be employed as a management threshold. Keywords: Inter-examiner, prism, reliability, strabismus
INTRODUCTION
measurement of strabismus, such as examiner experience, examiner bias, positioning of the examiner and target, dissociative techniques, test being used (eg, APCT and simultaneous prism and cover test), neutralization endpoint employed, calibration and position of the prism in front of the eye, magnitude of deviation (eg, 20 PD vs420 PD), test distance, age, and type of strabismus.1–10 A number of studies have reported inter-examiner variability and agreement in children and adults comparing 2 examiners.1,2,8,10,11 In large eye clinics more than 2 examiners often measure the angle of deviation on a patient on separate visits. The inter-examiner variability and agreement between more than 2 examiners assessing children and
In patients with strabismus, the angle of deviation is routinely measured at clinical visits to evaluate stability. Monitoring the angle of deviation on consecutive visits allows identification of deterioration and aids in the decision whether strabismus surgery is needed. The amount of strabismus surgery performed on the extraocular muscles largely depends on the angle of deviation measured, commonly performed with the alternate prism cover test (APCT). Variation in measurements between clinical visits can occur. Several factors have been reported to play a role in the
*Presented at the European Strabismological Association, Marseille, France, September 4–6, 2013. Received 29 October 2013; Revised 29 September 2014; Accepted 30 September 2014; Published online 31 October 2014 Address for correspondence: E. de Jongh, Schiedamse Vest 180, 3011 BH, Rotterdam, The Netherlands. E-mail: [email protected]
158
The inter-examiner variability of the alternate prism and cover test 159 adult patients is however unknown. The purpose of this study was to establish this by including 4 experienced examiners and provide guiding values for clinicians as to what can be considered a true change in the size of strabismus in adults and children.
MATERIALS AND METHODS Patients with horizontal strabismus attending the Orthoptic Department of the Rotterdam Eye Hospital, Rotterdam, the Netherlands were recruited to participate in this study. Ethical approval was sought and obtained at the Medisch Ethische Toetsings Commissie (METC, Medical Ethical Committee, Rotterdam, the Netherlands). This study was conducted in accordance with the principles of the Declaration of Helsinki (October 2008), the guideline for Good Clinical Practice (CPMP/ICH/135/95), and the Medical Research Involving Human Subjects Act (WMO; 2006). Each participant or parent gave written informed consent before participating in this study. Eligibility criteria included isolated horizontal strabismic deviations (including latent, intermittent, and manifest deviations) or combined horizontal and vertical deviations (horizontal angle larger than the vertical). All participants had to wear their full refractive correction. In children aged 512 years, this was controlled by only including those who had cycloplegic refraction, within 6 months before participating in this study. Both eyes were able to fixate centrally in primary position with a visual acuity (VA) of at least 20/200 in either eye with full refractive correction. Participants with eccentric fixation or a dissociated vertical deviation were excluded. Each participant was seated at the same clinical examination station while 4 examiners rotated to ensure consistency of environmental circumstances. Each participant was assessed by 4 masked orthoptists with at least 3 years of working experience. To control for order effects, the order of examiners was determined from a counterbalancing table of possible combinations of 4 examiners. For each participant, two examiners first performed near followed by distance fixation measurements; the other two examiners performed measurements in reversed order. Examiners were masked with respect to the results of previous measurements and were not able to exchange information. An APCT instruction leaflet was given to all examiners to ensure standardization of measurements and each examiner completed a predesigned standardized case report form. Measurements were performed within 40 minutes with a 5-minute rest period between measurements to minimize the influence of the previous measurement and bias induced by fatigue. A Snellen optotype was used as standardized fixation target in each participant. Participants were !
2014 Informa Healthcare USA, Inc.
asked to fixate the smallest target they could see clearly with their worse eye. The prism bars used were the plastic Gulden prism bars, (Horizontal B-15 and Vertical B-14) calibrated for frontal plane position. All examiners were provided with a horizontal prism bar with the units 1, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 25, 30, 35, and 40. The vertical prism bar contained the units 1, 2, 3, 4, 5, 6, 8, 10, 12, 14, 16, 18, 20, and 25. A loose isosceles prism of 45 PD was part of the prism set. When the deviation exceeded 45 PD, such that a single prism could not be used, prisms were split between eyes. The largest prism, as close as equal as possible, was placed in front of the non-fixing eye to minimize the shift in primary position. In case of a combined horizontal and vertical deviation, simultaneously used prism bars were divided between the two eyes. Because stacking of prisms with bases in the same direction can lead to erroneous measurement, this was not allowed in this study.6 Care was taken not to tilt or rotate the prism. Deviations, based on the fixation of preference, were recorded as the value of the prism that reached the first neutral endpoint. To ensure full dissociation, the cover paddle was held as closely as possible to the patient’s eye in combination with the ‘‘long dissociation protocol’’ of Anderson et al.2 The cover paddle was initially held in front of the non-dominant eye for 5 seconds. ‘‘The eye was alternately occluded for a count of 2 seconds for nine additional excursions of the cover paddle. The right and the left eye were covered five times each, leading to a total dissociation time of 23 seconds,’’ (p. 170).2 Cooperation of participants aged 512 years was rated during each of the 4 measurements by every examiner. A 5-point coping scale was used, modified from the Distress and Approach-Avoidance score scale of Blount et al.12 The scale ranged from extreme distress/agitation/screaming (score 1) to no distress/ calm/no crying (score 5). The higher scores indicate more co-operative behavior.12
Statistical Analysis Different prism increments were used to measure smaller (2-20 PD) and larger (420 PD) horizontal angles of strabismus due to the design of the horizontal prism bar. Pairs of measurements in 2 strata were analyzed: one where the prism increment used for measurement was 2 PD (angles 20 PD) and a second where the prism increment was 5 PD (420 PD). If measurements fell into both groups, the values were analyzed in the largest increment stratum. Median and range were calculated due to the Kolmogorov-Smirnov test showing that the data was non-normally distributed. Multiple regression analysis was used to assess the relationship of variables (age, type of deviation, magnitude of the deviation, cooperation) on the variation in APCT measurements
160 De Jongh et al. between examiners. Analysis of variability and agreement between examiners comprised calculating the mean and standard deviation (SD) of the difference between measurements of all possible combinations of examiners. Pairwise, inter-examiner agreement between measurements was evaluated by means of Bland-Altman plots.13 Differences between the measurements were plotted against the mean of the two measurements. The inter-examiner variability (1.96 SD) were used to determine a clinical value likely to indicate real change. The agreement between examiners was expressed by the percentage of agreement and corresponding 95% confidence intervals (CI). P values 50.05 were considered as statistically significant and differences of at least 2 PD were considered as clinically significant.14 All reported P values were 2-tailed. The following program was used to test if a difference between two percentages was statistically significant (http://www.vassarstats.net/propdiff_ind.html).
RESULTS APCT measurements at near and distance were completed in 41 participants with horizontal strabismus (21 males and 20 females). The median age was 12 years (range 3–74 years). Twenty-five participants had an exo-deviation and 16 an eso-deviation. More than half of the number of participants had a manifest deviation (n = 22), followed by intermittent (n = 13) and latent deviations (n = 6). The median cooperation was ‘‘fine’’ (score 4, range 1–5) and corresponded with a median age of 7 years. The cooperation of all the children remained stable during the 4 APCT measurements performed by the 4 examiners at the same visit. A positive correlation was found between age in years and cooperation in participants 512 years (Pearson’s correlation coefficient r = 0.62; p = 0.01). The median distance Snellen VA in the right eyes was 6/7.5 (range 6/15–6/4) and 6/6 in the left eyes (range 6/15–6/4.6) and near VA in the right eyes was 6/9 (range 6/36–6/6) and 6/9 in the left eyes (range 6/18–6/6). The median fixation target size used was 6/6 at distance and 6/9 for near, based on the VA of the worst eye. The APCT measurements were performed by the 4 examiners on all of the 41 participants for both near and distance fixation. The median horizontal near deviation measured 20 PD (range 4–53) and 18 PD (range 0–47) at distance fixation. Table 1 shows the near and distance measurements per examiner. The 95% limits of agreement for participants measuring 20 PD were ±9.4 PD at near and ±6.1 PD at distance fixation. For deviations measuring420 PD, they were ±9.9 PD and ±10.3 PD at near and distance fixation respectively. Larger variability in APCT measurements was found in participants aged 512 years
TABLE 1. The median measurement per examiner for near and distance fixation given in PD (N = 41). Near
Examiner 1 Examiner 2 Examiner 3 Examiner 4 All 4 examiners
Distance
Median
Range
Median
Range
20.0 20.0 20.0 20.0 20.0
4–49 6–53 4–45 4–51 4–53
18.0 18.0 18.0 16.0 18.0
2–45 0–47 0–45 0–45 0–47
than 12 years for both near (±10.5 vs ±8.7 PD) and distance (±9.1 vs ±6.8 PD) fixation. Multiple regression analysis was performed to determine if variables such as age, type of deviation, magnitude of deviation, and cooperation influenced the variation in APCT measurements. 1.9% of the variation in near APCT measurements (difference in APCT measurements between examiners) was caused by these variables (R2 0.02). The standardized regression coefficients showed that none of the variables had significant influence on the variation in APCT measurements. The same variables induced 3% of the variation in APCT measurements at distance (R2 0.03), without any significant influence on the variation in APCT measurements (p50.05). Due to the lack of relationship between the above-described variables and variation in APCT measurements, only data analyses including all the participants will be reported in the following. Comparison of APCT measurements for each examiner against the average deviation of all examiners was performed. Figures 1 through 4 display the under- and overestimation of APCT measurements for each examiner for near and distance fixation. Examiners 1 and 2 showed larger differences (95% limits of agreement) for near measurements (±5.9 PD and ±8 PD) compared with examiners 3 and 4 (±5.6 PD and ±4.9 PD). Distance measurements were more alike in all examiners (range 4.2 to 5.1 PD). Besides comparing APCT measurements of each examiner against the average deviation, differences in APCT measurements between examiners were analyzed. Tables 2 and 3 show the differences in APCT measurements for each examiner pair. In Figure 5, the differences between all examiner pairs are plotted against the mean deviation. Comparison of all horizontal APCT measurements for near fixation between the examiners showed the mean difference between two measurements was 0.59 PD (95% CI = 0.55 to 0.67) with inter-examiner variability of ±10 PD (Figure 5a), and 0.21 PD (95% CI = 0.39 to 0.63) with inter-examiner variability of ±9 PD for distance fixation (Figure 5b). Greater variability in measurements was found for near compared to distance fixation, but this was not Strabismus
The inter-examiner variability of the alternate prism and cover test 161
FIGURE 1. Under- and overestimating of APCT measurements by examiner 1 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±5.9 PD for near fixation and ±4.3 PD for distance fixation. The solid line shows the mean of the differences to be 0.47 vs 0.62 PD for near and distance fixation, respectively.
statistically significant (p = 0.58). The mean difference between measurements per examiner pair for near fixation ranged between 0.8 and 1.6 PD with interexaminer variability ranging between ±7.5 and ±11.7 PD, and mean difference for distance fixation between 3 and 1.5 PD with inter-examiner variability ranging between ±6.9 and ±12.5 PD (Tables 2 and 3). The percentage of agreement based on all examiner pairs within the ranges of ±2, ±4, ±6, ±8 and ±10 PD are shown in Table 4. Agreement within 2 PD between all 4 examiners was 51% for near and 57% for distance fixation and increased to 96% for near and 98% for distance fixation within 10 PD. A difference of at least 2 PD has been suggested as clinically significant14 and therefore subsequent analysis between examiner pairs evaluated agreement within 2 PD only. The percentage agreement of obtaining the same measurement between two examiners within 2 PD ranged between 39% to 66% for near and 37% to 70% for distance fixation, respectively (Table 5). Including all 4 examiners no significant difference (p = 0.17) was found in percentage agreement within 2 PD for near against distance fixation. Most of the examiner pairs showed the same result except for examiner pair 1 vs 2, where the percentage of agreement within 2 PD was significantly larger for distance fixation than near fixation (p50.01). Distance measurements were more alike compared with near angles. On the contrary, for examiner pair 3 vs 4 near measurements were more alike compared to distance angles. !
2014 Informa Healthcare USA, Inc.
TABLE 2. Inter-examiner variability of all APCT horizontal measurements at near fixation.
Examiner 1 vs 2 Examiner 1 vs 3 Examiner 1 vs 4 Examiner 2 vs 3 Examiner 2 vs 4 Examiner 3 vs 4 All 4 examiners
Mean
SD
Inter-examiner variability (±1.96 SD)
0.2 1.5 0.6 1.6 0.8 0.8 0.6
6.0 4.7 4.3 5.8 5.5 3.8 5.1
11.7 9.3 8.4 11.3 10.7 7.5 10.0
95% limits of agreement 11.8 to 11.5 7.8 to 10.7 7.8 to 9.0 9.7 to 12.9 9.9 to 11.5 8.3 to 6.6 9.4 to 10.3
TABLE 3. Inter-examiner variability of all APCT horizontal measurements at distance fixation.
Examiner 1 vs 2 Examiner 1 vs 3 Examiner 1 vs 4 Examiner 2 vs 3 Examiner 2 vs 4 Examiner 3 vs 4 All 4 examiners
Mean
SD
Inter-examiner variability (±1.96 SD)
0.3 1.5 0.8 1.2 0.4 3.0 0.2
3.8 3.9 3.5 4.2 3.9 6.4 4.6
7.5 7.7 6.9 8.3 7.7 12.5 9.0
95% limits of agreement 7.2 to 7.8 6.2 to 9.2 6.1 to 7.7 7.1 to 9.5 7.3 to 8.2 15.4 to 9.5 9.8 to 10.2
The order of measurements did not influence the variation in APCT measurements for near and distance fixation. The inter-examiner variability of the first measurement was similar to the fourth measurement at near (±5.9 PD vs ±5.6 PD) and distance fixation (±4.8 PD vs ±4.5 PD).
162 De Jongh et al.
FIGURE 2. Under- and overestimating of APCT measurements by examiner 2 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±8 PD for near fixation and ±4.8 PD for distance fixation. The solid line shows the mean of the differences to be 0.62 vs 0.30 PD for near and distance fixation, respectively.
FIGURE 3. Under- and overestimating of APCT measurements by examiner 3 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±5.6 PD for near fixation and ±5.1 PD for distance fixation. The solid line shows the mean of the differences to be 1 vs 0.90 PD for near and distance fixation, respectively.
DISCUSSION This study investigated inter-examiner variability and agreement of APCT measurements in children and adults with horizontal strabismus performed by 4 experienced examiners. The inter-examiner variability
between APCT measurements performed by 2 examiners ranged from 6.9 to 12.5 PD, depending on the examiner pair and fixation distance. The findings show that it is important to compare more than two examiners because the APCT measurements between examiners were variable per examiner pair and even Strabismus
The inter-examiner variability of the alternate prism and cover test 163
FIGURE 4. Under- and overestimating of APCT measurements by examiner 4 against mean of all examiners at near and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±4.9 PD for near fixation and ±4.6 PD for distance fixation. The solid line shows the mean of the differences to be 0.16 vs 0.14 PD for near and distance fixation, respectively.
FIGURE 5. Inter-examiner variability of all APCT horizontal measurements at near fixation and distance fixation. Bland-Altman plots showing inter-examiner variability (±1.96 standard deviation of the difference) to be ±10 PD for near fixation (A) and ±9 PD for distance fixation (B). The solid line shows the mean of the differences to be 0.59 vs 0.21 PD for near and distance fixation, respectively.
more so if all 4 examiners were compared. This is of clinical relevance because in many eye clinics multiple examiners are likely to perform the APCT measurements on a patient on consecutive visits and thus will have an impact on the clinical decision, whether the strabismus is remaining stable, improving, or deteriorating. !
2014 Informa Healthcare USA, Inc.
A number of studies have evaluated inter-examiner variability between 2 examiners to establish when a difference in measurements of angle of deviation may be considered a true change.1,2,8,10,11 Holmes et al.1 reported inter-examiner variability of the APCT in adults with sixth nerve palsy and suggested a change in measurements of 10 PD or more was suggestive
164 De Jongh et al. TABLE 4. Percentage agreement based on the differences in APCT measurements between all 4 examiners within a range of ±2, ±4, ±6, ±8, and ±10 PD for near and distance fixation. Difference within (PD)
Near (%)
Distance (%)
51 59 85 91 96
57 68 87 92 98
±2 ±4 ±6 ±8 ±10
TABLE 5. Percentage agreement based on the differences in APCT measurements between all examiner pairs within a range of ±2 PD for near and distance fixation. The P values demonstrate if the difference in agreement between near and distance fixation was significant.
Examiner 1 vs 2 Examiner 1 vs 3 Examiner 1 vs 4 Examiner 2 vs 3 Examiner 2 vs 4 Examiner 3 vs 4 All 4 examiners
Near fixation
Distance fixation
P value
39% 51% 56% 41% 54% 66% 51%
70% 54% 63% 59% 61% 37% 57%
50.01 0.83 0.50 0.12 0.50 0.01 0.17
of a true change and any less may be attributable to test-retest variability. The Pediatric Eye Disease Investigator Group (PEDIG) also suggested that in case of differences within the guiding value of 10 PD, one would not be able to distinguish what component of the difference is measurement error and what component is real change.8 PEDIG investigated children aged 60 months with esotropia. Their findings suggested that a difference of 6 PD or more for esotropias measuring between 10 and 20 PD and that a difference of 12 PD for deviations measuring greater than 20 PD may be considered a true change. Hatt et al.15 examined children aged 1–13 years with intermittent exotropia. They reported threshold values for what may be considered a real change for deviations measuring 20PD or less and found values of 7PD and 3PD for near and distance fixation respectively. The present study showed slightly larger interexaminer variability for near angles compared with distance angles. Bland-Altman analysis showed a value of 10 PD was suggestive of real change at near fixation vs 9 PD for distance fixation. These values were based on the average of the measurements of 4 examiners. It should be noted that better or worse results were found dependent on different examiner pairs. Only examiner pair 1–2 showed significantly larger inter-examiner variability for near fixation compared with distance fixation (11.7 vs 7.5 PD). Examiner pair 3–4 showed significantly larger interexaminer variability for distance fixation compared
with near fixation (12.5 vs 7.5 PD) (Tables 2 and 3). The other examiner pairs showed a slightly larger inter-examiner variability for APCT measurements performed at near fixation. Accommodation and tenacious proximal fusion16 are factors that induce variation in APCT measurements, which is the reason why many studies8 found greater inter-examiner variability for near angles compared with distance angles. All subjects in the present study were wearing their full refractive correction to minimize variability in measurement due to accommodation. To control the accommodative driven vergence, a standardized accommodative fixation target and instructions to keep the target clear were given. Although the smallest target the worse eye could see was used in each subject, it is still possible that fine focusing during the APCT could have interfered with staring and not accommodating. This still does not explain why there was more interexaminer variability for measurements performed by examiner pair 3–4. In most studies, larger limits of agreement were found for near deviations compared with distance deviations and in measurements of large angles compared with small angles.1,8,10,16 The present study shows larger 95% limits of agreement at near fixation vs distance fixation for participants measuring 20 PD (±9.4 PD vs ±6.1 PD). For deviations measuring 420 PD, there was no clinically significant difference (±9.9 PD vs ±10.3 PD) at near and distance fixation, respectively. These values are based on the average of the measurements of 4 examiners and more or less test-retest inter-examiner variability is dependent on different examiner pairs. However, the variable magnitude of the deviation did not significantly influence the measurement variability between examiners. It is possible that with a larger sample size and consequently larger groups of specific types of deviation, magnitude of deviation and age group relationships may have been found. Because the increment steps in the present study were 5 PD for large angles, which is double the size of the prisms used in the PEDIG study,8 larger interexaminer variability was expected in this study. This was however not the case. Despite the larger mean deviation measured by PEDIG8 (37.2 PD ± 14.8 at near and 32.5 PD ± 14.3 at distance fixation) and the difference in prism increments, our results are comparable. It is possible the larger inter-examiner variability in large deviations reported by PEDIG was caused by the magnitude of the deviation and strength of prism, as described by Firth and Whittle,6 rather than caused by the factor prism increment. A slight tilt of a large prism can lead to measurement errors.7 The age of the subjects in the PEDIG study was younger than the present study. The present study explored whether younger children’s attention and ability to fixate well was less and Strabismus
The inter-examiner variability of the alternate prism and cover test 165 could led to greater inter-examiner variability. Larger measurement variability was seen in participants aged 512 years compared with 12 years. Although these values were larger in those 512 years, there was only a clinical difference in inter-examiner variability at distance fixation (±9.1 vs ±6.8 PD). However, standardized regression coefficients showed that age had no significant influence on the variation in APCT measurement. The median cooperation of participants aged 512 years was determined to be fine (score 4) in this study, but there was a significant positive correlation between cooperation and age in those 512 years, which confirms the impression that the older the child, the better the cooperation. Nevertheless the cooperation level of participants aged 512 years remained the same during the APCT measurements by the 4 examiners. A larger sample size is needed to confirm this finding.
SUMMARY One should be aware that more or less test-retest inter-examiner variability is dependent on different examiner pairs. A variation in APCT measurement between 4 examiners of less than 10 PD for both near and distance fixation is likely to be due to interexaminer variability. Magnitude of the deviation, test distance, and age had no significant influence on the variation in APCT measurement. Changes of 10 PD or more are suggestive of a true change and may be employed as a management threshold for defining stability, improvement, or deterioration.
ACKNOWLEDGMENTS This study was conducted at the Orthoptic Department of the Rotterdam Eye Hospital, The Netherlands.
Limitations of the Study DECLARATION OF INTEREST Using 4 examiners, it is important to control for order effects. Larger inter-examiner variability was expected during the last measurement due to fatigue and decompensation of the deviation. This study showed that the last measurement did not have the largest angle, but the type of deviation did change due to decompensation. For example, an intermittent deviation became manifest but the angle did not increase. The fact that Hatt et al.15 found less inter-examiner variability in their study investigating children with intermittent exodeviations suggests that the intermittent nature of some of the deviations did not affect the limits of agreement. This study employed a specific protocol for measurement of the deviation with a prespecified number of cover-uncovers and dissociation time. This standardized method is beyond routine clinical practice and raises the question whether the results can be generalized. Interestingly, our results produced similar differences between measurements as reported in previous studies including 2 examiners.1,8 With a less standardized protocol, a larger variation may have been found between 4 examiners. The results from this study support that the APCT is fairly reliable between examiners. It is however not possible to control all variables. It is therefore important to confirm stability of the ocular misalignment before performing strabismus surgery. PEDIG17 reported that a minimum of 3 identical measurements or 4 measurements within 5 PD would be needed to confirm stability. What should be kept in mind is that any measurement of strabismus is the sum of the true deviation and measurement error. It is not clear what component of the difference is measurement error and what component is real change.8 !
2014 Informa Healthcare USA, Inc.
The authors have no financial conflict of interest.
REFERENCES 1. Holmes JM, Leske DA, Hohberger GG. Defining real change in prism-cover test measurements. Am J Ophthalmol 2008;145:381–385. 2. Anderson HA, Manny RE, Cotter SA, et al. Effect of examiner experience and technique on the alternate cover test. Optom Vis Sci 2010;87:168–175. 3. Hrynchak PK, Herriot C, Irving EL. Comparison of alternate cover test reliability at near in non-strabismus between experienced and novice examiners. Ophthalmic Physiol Opt 2010;30:304–309. 4. Johnson R, Wynn S, Coffey B. Influences of examiner position and effective prism power on nearpoint alternate cover test. Optometry 2004;75,8:496–502. 5. Flu¨eler UR, Elhatton KM, Guyton DL. A combination horizontal/vertical prism bar. A precisely calibrated tool for use in front of one eye. Strabismus 1995;3:27–32. 6. Firth AY, Whittle JP. Clarification of the correct and incorrect use of ophthalmic prisms in the measurement of strabismus. Br Orthopt J 1994;51:15–18. 7. Thompson JT, Guyton DL. Ophthalmic prisms: Measurement errors and how to minimise them. Ophthalmology 1983;90,3:204–210. 8. Pediatric Eye Disease Investigator Group. Interobserver reliability of the prism and alternate cover test in children with esotropia. Arch Ophthalmol 2009;127:59–65. 9. Schutte S, Polling JR, van der Helm FCT, Simonsz HJ. Human error in strabismus surgery: Quantification with a sensitivity analysis. Graefes Arch Clin Exp Ophthalmol 2008; 247:399–409. 10. Rainey BB, Schroeder TL, Goss DA, Grosvenor TP. Reliability of and comparisons among three variations of the alternating cover test. Ophthalmic Physiol Opt 1998;18: 430–437. 11. Johns HA, Manny RE, Fem K, Hu YS. The intraexaminer and interexaminer repeatability of the alternate covertest
166 De Jongh et al. using different prism neutralization endpoints. Optom Vis Sci. 2004;81:939–946. 12. Blount RL, Cohen LL, Frank NC, et al. The Child-Adult Medical Procedure Interaction Scale-Revised: An assessment of validity. J Pediatr Psychol 1997;22:73–88. 13. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135–160. 14. Fogt N, Baughman BJ, Good G. The effect of experience on the detection of small eye movements. Optom Vis Sci 2000; 77:670–674.
15. Hatt SR, Leske DA, Liebermann L, et al. Variability of angle of deviation measurements in children with intermittent exotropia. J AAPOS 2012;16:120–124. 16. Yang HK, Hwang J-M. The effect of target size and accommodation on the distant angle of deviation in intermittent exotropia. Am J Ophthalmol 2011;151:907–913. 17. Melia BM, Holmes JM, Chandler DL, Christiansen SP; Pediatric Eye Investigator Group. Classifying stability of misalignment in children with esotropia using simulations. Arch Ophthalmol 2010;128:1555–1560.
Strabismus
Copyright of Strabismus (09273972) is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
