The role of eye movements in depth from motion parallax during infancy.

Journal of Vision (2013) 13(14):15, 1–13

http://www.journalofvision.org/content/13/14/15

1

The role of eye movements in depth from motion parallax during infancy Elizabeth Nawrot Mark Nawrot

Department of Psychology, Minnesota State University Moorhead, Moorhead, MN, USA

# $

Center for Visual and Cognitive Neuroscience, Department of Psychology, North Dakota State University, ND, USA

# $

Motion parallax is a motion-based, monocular depth cue that uses an object’s relative motion and velocity as a cue to relative depth. In adults, and in monkeys, a smooth pursuit eye movement signal is used to disambiguate the depth-sign provided by these relative motion cues. The current study investigates infants’ perception of depth from motion parallax and the development of two oculomotor functions, smooth pursuit and the ocular following response (OFR) eye movements. Infants 8 to 20 weeks of age were presented with three tasks in a single session: depth from motion parallax, smooth pursuit tracking, and OFR to translation. The development of smooth pursuit was significantly related to age, as was sensitivity to motion parallax. OFR eye movements also corresponded to both age and smooth pursuit gain, with groups of infants demonstrating asymmetric function in both types of eye movements. These results suggest that the development of the eye movement system may play a crucial role in the sensitivity to depth from motion parallax in infancy. Moreover, describing the development of these oculomotor functions in relation to depth perception may aid in the understanding of certain visual dysfunctions.

Introduction Motion parallax is monocular depth cue that arises from the relative motion of objects at different distances that is created when an observer translates laterally. As the observer translates, gaze is maintained on a particular object in the scene through the activity in a number of eye movement systems (Miles, 1998). Due to the observer translation and compensatory eye movement, objects located nearer than the observer’s point of gaze have a relative movement in the opposite

direction to the observer’s movement, whereas objects located farther away have a relative movement in the same direction as the observer’s movement. Furthermore, an object’s relative velocity is also a cue to relative depth magnitude as objects nearer to the point of gaze move more slowly across the visual field than do objects more distant from the fixation point. Motion parallax and binocular stereopsis work in similar ways to provide very similar cues to relative depth. Whereas motion parallax uses retinal motion cues, with binocular stereopsis the cues come from retinal disparity. The magnitude of retinal disparity is proportional to the object’s depth from the fixation point, and disparity sign (crossed vs. uncrossed) signals opposite depths relative to fixation. However, unlike depth from retinal disparity, the retinal image motion information for depth from motion parallax is inherently depth-sign ambiguous (Farber & McConkie, 1979). Fortunately, the perception of depth from motion parallax is not depth-sign ambiguous. Recent work (Nadler, Nawrot, Angelaki, & DeAngelis, 2009; M. Nawrot, 2003; M. Nawrot & Joyce, 2006) suggests that the human and primate visual systems use a concomitant smooth pursuit eye movement signal to disambiguate the depth sign in motion parallax. Smooth pursuit eye movements are used to track a moving stimulus, or a stationary stimulus during observer movement, in order to maintain fixation, minimize blur due to motion, and maximize acuity (Krauzlis, 2004). With motion parallax, perceived depth sign relies on the direction of the smooth pursuit eye movement signal: Retinal image motion in the direction opposite the pursuit signal is perceived nearer than fixation (remember that retinal image motion and object motion in the scene are opposite). Retinal motion in the same direction is perceived farther away than fixation (M. Nawrot & Joyce, 2006).

Citation: Nawrot, E., & Nawrot, M. (2013). The role of eye movements in depth from motion parallax during infancy. Journal of Vision, 13(14):15, 1–13, http://www.journalofvision.org/content/13/14/15, doi:10.1167/13.14.15. doi: 10 .116 7 /1 3. 14 . 15

Received January 8, 2013; published December 18, 2013

Downloaded From: https://jov.arvojournals.org/ on 10/18/2018

ISSN 1534-7362 Ó 2013 ARVO


Nawrot & Nawrot

Previous research on the development of infants’ sensitivity to depth from motion parallax is limited. Two studies using real objects in a motion parallax scenario concluded that infants are sensitive to motion parallax between 3- and 6-months-of-age, based on dishabituation (von Hofsten, Kellman, & Putaansuu, 1992) and preferential reaching measures (Condry & Yonas, 2013). In a longitudinal study using computergenerated stimuli, E. Nawrot, Mayo, and Nawrot (2009) explored the onset of sensitivity to motion parallax from 8- to 29-weeks-of-age. Infants were habituated to a translating random-dot cinematogram containing a particular phase of depth corrugation specified by motion parallax. Following habituation, infants dishabituated to a depth-phase reversed test trial on average by 16-weeks-of-age. With regard to the role of pursuit eye movements in motion parallax, infant studies find that smooth pursuit gain is quite mature as early as 3-months-of-age, which is typically the lower age limit on depth-from-motion studies. Pursuit gain is calculated as the ratio of pursuit velocity to target velocity, where a value of 1.0 indicates perfectly accurate pursuit tracking. Von Hofsten and colleagues found that smooth pursuit gain improves rapidly and significantly between 2 and 5 months of age (Phillips, Finocchio, Ong, & Fuchs, 1997; Rosander & von Hofsten, 2002; von Hofsten & Rosander, 1997). In a series of longitudinal studies, von Hofsten (2004) reported that some individual infants demonstrated adult gain values by 10 weeks of age. Importantly, both esotropes and normally developing infants demonstrate a directional asymmetry when it comes to their smooth pursuit eye movements: Smooth pursuit tracking in the temporal-nasal (TN) direction is normal while nasal-temporal (NT) eye movements have a reduced velocity (Demer & von Noorden, 1988; Westall, Woodhouse, & Brown, 1989; for a review, see Braddick & Atkinson, 2011). When tested for motion parallax, adult esotropes show a depth-perception asymmetry linked to their pursuit asymmetry. That is, their sensitivity to motion parallax is normal in the TN direction (with normal TN pursuit), but elevated in the NT direction (with abnormal NT pursuit; M. Nawrot, Frankl, & Joyce, 2008; M. Nawrot, Frankl, & Stockert, 2004). These eye movement asymmetries in both normally developing infants and esotropes suggest that the eye movement systems and depth from motion parallax might provide a putative link between abnormal motion perception and abnormal stereopsis in esotropia (Norcia, 1996). That is, a failure of maturation of the eye movement systems may possibly be an early step in a developmental process, including motion perception and the perception of depth from motion, contributing to the development of esotropia (Norcia, 1996; Tychsen, Hurtig, & Scott, 1985).


2

While it is now well established that eye movements are of particular importance in depth perception from motion parallax, most previous infant studies on the perception of depth and kinetic shape perception, as well as motion parallax, have not addressed the role of either head or eye movements. To determine whether the development of smooth pursuit eye movements serves a crucial role in the perception of unambiguous depth-sign for motion parallax, we must determine when the developing smooth pursuit system has sufficiently matured and then directly measure pursuit eye movements in relation to a motion parallax task. Similar to smooth pursuit, the ocular following response (OFR) is a visually-driven, short-latency, ‘‘slow’’ eye movement that helps stabilize gaze during observer translation (Kawano, 1999; Miles, 1998; Perrinet & Masson, 2007). Also, similar to pursuit, the OFR is depth-selective, typically attempting to stabilize the retinal image of objects in the fixation plane (Busettini, Miles, Schwarz, & Carl, 1994; Masson, 2004). However, whereas pursuit eye movements constitute a voluntary, closed-loop system to maintain fixation on a target, the OFR are generated automatically as an open-loop response to a target motion. The functional similarities between these two systems suggest that OFR and the open-loop-initiator phase of pursuit might be served by related neural mechanisms. With regard to the unambiguous perception of depth from motion parallax, recent work (M. Nawrot & Stroyan, 2012) demonstrates that unambiguous depth from motion parallax can be achieved with very brief (approximately 30 ms) stimulus presentations. Moreover, a high-contrast pattern mask disrupts unambiguous perception when stimulus-onset asynchronies are shorter than about 70 ms. This suggests that OFR, the initiator-phase of pursuit, could be providing the crucial extraretinal signal necessary for unambiguous depth from motion parallax. Here, we wondered if the development of OFR in infants might show a closer link to the development of motion parallax than does the development of closed-loop pursuit of a smoothly translating target. In the current study we investigated the developmental time course for normally developing smooth pursuit and OFR eye movements in relation to depth perception from motion parallax. Using a cross section of infants from 8 to 20 weeks of age, we collected data in three experimental conditions within a single session to address the following questions: When does sensitivity to depth from motion parallax develop and what is its relationship to the development of smooth pursuit gain? Does the development of motion parallax occur at the same time as the development of high-gain pursuit or high-gain OFR eye movements? Experiment 1 examined infants’ sensitivity to motion parallax in order to replicate and extend findings from the small


Nawrot & Nawrot

3

(n ¼ 12); fussiness or sleepiness (n ¼ 5); or poor gaze or eye calibration (n ¼ 2).

Figure 1. The motion parallax stimulus.

literature that currently exists in important ways. We used stimuli and procedures that were similar to previous research, extended the age groups that have been examined, and added a partial-lag control group to examine spontaneous dishabituation effects.

Experiment 1 Method Participants For the following three experiments, the same group of infants was used, so recruitment procedures remain the same. Infant names were identified through county birth records and newspapers’ birth announcements, and parents were contacted either by letter or phone and invited to participate. Families were compensated with store gift cards and coupons to a local photography studio. Recruited infants were born within two weeks of their due date and had no birth complications by parental report. For all three experiments, infants were placed into one of four age groups: 8 weeks (mean age ¼ 64 days); 12 weeks (mean age ¼ 86 days); 16 weeks (mean age ¼ 116 days); and 20 weeks (mean age ¼ 142 days). Forty-three infants (25 female) completed the first experiment across the four age groups: three 8-weekolds; nineteen 12-week-olds; thirteen 16-week-olds; and eight 20-week-olds. An additional 41 infants (24 female) completed a partial-lag design that included posthabituation trials to examine spontaneous dishabituation effects (Bertenthal, Haith, & Campos, 1983). This group included two 8-week-olds; twenty-one 12week-olds; twelve 16-week-olds; and six 20-week-olds. Data from an additional 19 infants were dropped (11 from the experimental group, eight from the partial lag group) due to experimenter error or equipment failure


Apparatus Stimuli were generated with Presentationt software (Neurobehavioral Systems, Albany, CA) running on an Intel Core computer (Daktech, Fargo, ND) with Windows XP (Microsoft, Redmond, WA) and were presented on a 21-in. CRT (NEC, Itasca, TX) set to a resolution of 1280 · 1024 pixels at 100 Hz. The monitor was positioned on a tilt-adjustable base that allowed the monitor and eye tracker to be adjusted together. Eye position was monitored with a Tobii X120 eye-tracking system (Tobii Technology, Falls Church, VA) with the control and calibration software, Tobii Studio (Tobii), running on a separate Intel Core computer running Windows 7 (Microsoft). A powered VGA switch allowed the CRT monitor to be driven by either computer/software system, switching between Studio for calibration and Presentation for stimulus presentation. Following calibration, the Tobii eyetracking system sent eye position output data to both computers simultaneously via TCP/IP protocol. The experimental program running in Presentation recorded this eye position data and also used it to trigger stimulus events when needed. Tobii Studio, in one window, superimposed this eye position data upon an image of the dynamic stimulus being viewed by the infant, allowing the experimenter to monitor the infant’s gaze and progress of the experiment. In another window, Tobii Studio also presented a video image of the infant from a USB camera mounted just above the eye tracker. In a third window, Studio presented a graphic representation of the tracking quality and the infant’s viewing distance. A third Macintosh laptop computer was also connected to the VGA switch allowing brief cartoons to be presented to the infant before and between experiments. Stimulus The motion parallax stimulus, similar to previous research (E. Nawrot, Mayo, & Nawrot, 2009), was a random-dot cinematogram (Rogers & Graham, 1979) that appears (to adult observers) as a three-dimensional (3-D) corrugated surface, undulating in depth along the vertical axis (see Figure 1). The stimulus comprised three thousand 4.9 0 · 4.9 0 white dots (19.01 cd/m2) on a black background (0.000 cd/m2). The stimulus window subtended 16.48 · 16.48 at a viewing distance of 65 cm. Within this stimulus window, the motion parallax stimulus depicted 1.5 cycles of sinusoidal depth corrugation over the 16.48 stimulus height, giving a stimulus spatial depth frequency of 0.09 cycles/8. The additional 0.25 cycles of corrugation at the top and


Nawrot & Nawrot

4

Figure 2. A depiction of the motion parallax habituation procedure.

bottom of the stimulus helped emphasize the depth curvature of the stimulus compared to a stimulus that subtended only a single cycle of depth corrugation. The depth corrugation was created by translating dots within the stimulus window with horizontal velocities that varied vertically across the stimulus according to a sinusoidal function. Dots that translate in opposite directions appear in opposite depth relative to the fixation point. Dots with higher velocities appear to have greater depth from the zero depth found at the vertical midline of the stimulus. The peak dot velocity, within the stimulus window, was 0.918/s, while the dots representing zero depth were stationary within the stimulus window. For adult observers (and presumably infants), the unambiguous perception of near-far depth phase of the motion parallax stimulus depends on the relationship between dot movement and stimulus window translation eliciting pursuit (M. Nawrot & Joyce, 2006). For example, rightward translation of the stimulus window elicits a rightward pursuit eye movement. In such a case, dots moving rightward within the stimulus are perceived near, and leftward moving dots are perceived as far in depth. In the absence of the pursuit eye movement the stimulus is perceptually depth-sign ambiguous. Lateral head movements by the observer are not a requirement for the unambiguous perception of depth from motion parallax with these displays, and are only involved insofar as they help elicit a pursuit eye movement. In this experiment the stimulus window started at the center of the screen, and translated 13.78 side-to-side (1.5 cycles, 41.18 total stimulus extent) across the black monitor at a rate of 0.1875 Hz, or 5.48/s for a total duration of 7.6 s. This stimulus window velocity is well within the parameters used by von Hofsten (2004) to measure smooth pursuit in this infant age range. To explain stimulus window movement, every trial began with a 1/4 cycle stimulus window translation to one side of the screen, after which the stimulus moved fully across the screen both rightwards and leftwards,


concluding with a 1/4 cycle translation to the screen center. With the assumption of perfect pursuit of the translating stimulus window, the peak dot velocity of 0.918/s and the stimulus window velocity of 5.48/s give a motion/pursuit ratio of 0.17 (M. Nawrot & Stroyan, 2009). At viewing distance of 65 cm, this stimulus corresponds to the physical geometry of a corrugated stimulus having dots extending out 11 cm near and far in depth. During habituation trials, the stimulus depth phase (whether the top or bottom half appeared in near or far depth) was counterbalanced across infants (i.e., half the infants viewed habituation trials with the top half of the stimulus in near depth; half of the infants viewed habituation trials with the top half of the stimulus in far depth). Moreover, the initial direction (left or right) of stimulus window translation in each trial was randomized. This depth phase was then reversed in the test trial that following habituation. Dishabituation to this reversal in depth phase during the posthabituation trials was taken as evidence for sensitivity to depth from motion parallax (see Figure 2). A small attention-getting stimulus (AGS) was also used at the beginning of the trial. The AGS was a ‘‘happy-face’’ composed of a yellow circle, 3.58 in diameter, with two ‘‘eyes’’ and a mouth, gray in color. This stimulus was alternated with a magenta circle of the same size at 25 Hz. Procedure After obtaining parental informed consent, the infant was seated on his/her parent’s lap approximately 65 cm from the monitor face in a three-sided enclosure to minimize distraction. An experimenter viewing the infant could direct the infant’s attention to the screen, initiate or delete trials, and record looking time with a key press. Infants viewed the stimuli binocularly to minimize the fussiness of wearing an eye patch, and minimize any viewing condition changes with the subsequent two eye-tracking experiments that used


Nawrot & Nawrot

5

stimulus was again presented on the following trial, followed then by the test (depth-reversed) stimulus.

Results

Figure 3. Two panels illustrate the proportion of total looking time to the motion parallax stimulus plotted against trial for two infants: (a) a 12-week-old and (b) a 20-week-old. The shaded area indicates data from the last habituation trial and test trial. The proportion of total looking time change from the last habituation trial to the test trial is (a) 2.36 and (b) 1.68.

binocular viewing. Previous infant studies have tested motion parallax using binocular viewing (von Hofsten, Kellman, & Putaansuu, 1992). Moreover, the stimulus and procedure here are similar to those used with infant monocular viewing conditions (E. Nawrot et al., 2009). More recent work by Domini and Caudek (2011) suggests that the combination of depth cues with monocular viewing and binocular viewing with zero disparity (as in the displays used here) produces the same output from the depth cue combination process. Using the infant-control habituation paradigm (Horowitz, Paden, Bhana, & Self, 1972), the habituation criterion was defined as a 50% decline in proportion of looking from the average of the first two trials. When the infant reached that habituation criterion, the test trials were presented in which the depth phase was reversed (i.e., if the top half of the stimulus was in far depth during habituation it was now shown in near depth). For the partial-lag group, after reaching habituation criteria, the same habituation


Figure 3a and b illustrate data collected across all trials of a single experimental session for motion parallax (MP) in two different infants. For each infant, the proportion of total looking time change is plotted for each trial. Because of the infant-control habituation technique, the total number of trials is different for each infant, taking into account their initial criterion for dishabituation, which is defined as 50% of the average of the first two trials. The shaded areas of the plot indicate the last habituation and first test trial data (trials 11–12 and 13–14 for Figure 3a and b, respectively). These figures illustrate a typical pattern of decreased looking time during the habituation trials, followed by recovery of looking to the test trials. This dishabituation to the test trial (the depth-reversed stimulus) is taken as evidence for the sensitivity of depth from motion parallax. Overall, infants displayed great interest in the stimulus, taking an average of 12 trials to reach habituation criteria. An ANOVA explored whether number of trials was affected by motion parallax condition (experimental, control), the presence of dishabituation to MP (yes, no), and age group (8, 12, 16, or 20 weeks old). There were no main effects of condition, F(1, 67) ¼ 0.58, p . 0.05; no main effect of dishabituation, F(1, 67) ¼ 3.07, p . 0.05; and no main effect of age group, F(1, 67) ¼ 2.47, p . 0.05. For each infant, a proportion of total looking time (PTLT) change was calculated as the proportion of total looking time to the test trial divided by the proportion of total looking time to the last habituation trial. In this way, a proportion change greater than 1 indicates greater relative looking time and presumably dishabituation, whereas a proportion change less than 1 indicates less relative looking and therefore continued habituation. PTLT change was then used as the dependent measure in the ANOVA to explore the variables of age and partial-lag group. Figure 4 illustrates PTLT change by age for the partial-lag group. The PTLT change for this group was calculated using the values from the last habituation trial (once criterion was met) to the next trial where the stimulus remained the same. Therefore, values greater than 1 indicate spontaneous dishabituation as no change in the stimulus actually occurred. While there were indications of spontaneous dishabituation in the partial-lag group, a one-way ANOVA on PTLT change showed that these effects were not significantly related to age, F(3, 42) ¼ 0.53, p . 0.05. The effect of age was, however, marginally significant for the experimental


Nawrot & Nawrot

6

Experiment 2 Methods

Figure 4. Proportion total looking time (PTLT) change from last two habituation trials is plotted as a function of age for the partial-lag group. Values above 1 indicate spontaneous dishabituation.

group, F(3, 39) ¼ 2.4, p ¼ 0.08. In further analyses data from the experimental group (n ¼ 43) were combined with the partial-lag group (n ¼ 41). This allowed for an increase in total number of subjects (and thus statistical power), particularly in the youngest age group where there were fewer than five infants in each condition. For this analysis, we used the PTLT change from the second posthabituation trial for the partial-lag group. As with the experimental group’s first posthabituation trial, this trial presented the depth reversed stimulus for the partial-lag group. The effect of age was significant, F(3, 85) ¼ 3.94, p ¼ 0.01. Figure 5 illustrates PTLT change by age group for all infants. The approximate age at which the data intersect the PTLT change .1 (dishabituation) is estimated at 12 weeks (the dotted black line). The results from Experiment 1 replicate and extend previous research on infants’ sensitivity to depth from motion parallax, particularly the estimate of age of onset. The evidence for spontaneous dishabituation suggests that this measure must be considered cautiously when making conclusions about sensitivity to depth from motion parallax. Infants who failed to dishabituate might also have been sensitive to motion parallax, but found the test stimulus at least as interesting as the habituation stimulus. Adult studies show that the ability to discriminate depth from motion parallax relies on a smooth pursuit eye movement signal, and so we conducted Experiment 2 in order to examine the development of smooth pursuit gain in these same infants and across this age range. As in previous research, we expect to find that age is related to gain, and we might also expect to see a relationship between gain and dishabituation as an indicator of motion parallax.


Participants A subset of the infants who participated in Experiment 1 also completed Experiment 2 during the same session. Smooth pursuit tracking was elicited from 51 infants (31 female) across the four age groups: three 8-week-olds; twenty-one 12-week-olds; nineteen 16-week-olds; and nine 20-week-olds. Data from an additional 43 infants were dropped due to fussiness or sleepiness (n ¼ 19), poor calibration or eye gaze sampled (n ¼ 16); or experimenter or equipment failure (n ¼ 8). Stimulus The smooth pursuit stimulus was similar to one created by von Hofsten (2004) to elicit pursuit tracking. Infants viewed a ‘‘happy-face’’ stimulus that translated leftwards and rightwards at 108/s in 1.5 s trials. The stimulus was primarily a yellow circle, 3.58 in diameter, with two ‘‘eyes’’ and a mouth, gray in color. This stimulus appeared against a black background. Procedure The experimental set up was identical to Experiment 1 and completed within the same session with the addition of eye tracking measures. Eye movements were recorded using a Tobii systems X120 eye tracker (120 Hz sampling) following a five-point, ‘‘infant’’ calibration routine. Each infant typically viewed

Figure 5. Infants’ sensitivity to motion parallax as a function of age. Proportion total looking time (PTLT) change from last habituation trial to first test trial is plotted against age group. Values above 1 indicate dishabituation to the test trial. The solid black line is a least squares trend line fit to the data (r2 ¼ 0.11); the dotted black line approximates where the data cross the y ¼ 1 intercept.


Nawrot & Nawrot

7

Figure 6. Smooth pursuit as a function of age. Pursuit gain is calculated as eye velocity/target velocity. As values approach 1, eye-tracking velocity matches target velocity. A least-squares trend line is fit to the data (r2 ¼ 0.25).

between 5 and 20 tracking trials, until the infant reached a criterion of greater than 75% eye gaze sampled for at least one trial in each direction. Percentage of gaze sampled was calculated online for each trial and displayed to the experimenter to know when the criterion was met. Examples of two infant eye-tracking sessions can be viewed with the supplementary materials (Movies 1 and 2).

Results Smooth pursuit performance was characterized by a maximum gain value, which was calculated as horizontal eye velocity/target velocity. The goal was to assign each infant the gain value for the best, sustained interval of pursuit that they achieved. Experimenters graphed and inspected all pursuit trials in which the gaze sample .75%, and selected the longest uninterrupted segment of horizontal smooth linear pursuit to give the best estimate of the infant’s gain in the leftward and rightward direction. Figure 6 shows smooth pursuit gain as a function of age (in days) for all infants whose tracking gaze met criterion (greater than 75% gaze sampled). A gain of 1.0 indicates pursuit velocity matching stimulus velocity. Gain increased with age (i.e., von Hofsten & Rosander, 1997). The Pearson correlation between age and gain of 0.50 was significant, t ¼ 3.73, p , 0.001. Twenty percent of infants showed asymmetric pursuit, with low gain in one direction and high gain in the other. Figure 7 illustrates the performance of one infant with asymmetric pursuit. In infants with asymmetric pursuit, the average pursuit gain in the leftward and rightward directions was significantly different, t(20) ¼ 4.17, p , 0.001. Fortytwo infants who completed the motion parallax Experiment 1 also provided pursuit gain scores in Experiment 2. The difference between mean pursuit


Figure 7. Two panels illustrate asymmetric tracking for one infant (age ¼ 16 weeks). The pink line illustrates the stimulus translation, the blue line is infant gaze. Gain calculated for leftward tracking trial (a) was 0.89; gain calculated for rightward tracking trial (b) was 0.32.

gain was not significantly different for infants who dishabituated to MP (M ¼ 0.62, SD ¼ 0.21) compared to those who failed to dishabituate (M ¼ 0.64, SD ¼ 0.25), t(41) ¼ 0.24, p . 0.05. While the results of Experiment 2 are in line with previous research on the development of pursuit gain, we did not find a relationship between gain and motion parallax as we might have expected from the adult literature. The measure of dishabituation as an index of motion parallax might explain this lack of a relationship for infant subjects (see Experiment 1, Results). However, the current study attempted to isolate and measure gain for the ongoing, closed-loop aspect of smooth pursuit eye movements. However, M. Nawrot and Stroyan (2012) found that depth from motion parallax is disambiguated by very brief stimulus presentations that implicate the internal programming signal for the early, open-loop initiator phase of pursuit eye movements. We wondered whether the development of this early, initiator-phase of pursuit, often studied as the ocular following response (OFR), might be more closely tied to the development of unambig-


Nawrot & Nawrot

uous depth from motion parallax in infant observers. To examine this possibility, Experiment 3 collected data on the development of the OFR eye movement in infants across this age range.

Experiment 3

8

saccadic eye movement and the direction of OFR stimulus translation. The OFR task included eight trials of the grating translating leftwards (four trials) or rightwards (four trials). Translation direction was randomized across trials. Eye position data was recorded to file during this 400 ms period to assess whether the infant displayed OFR or not.

Methods

Results

Participants A subset of the infants who participated in Experiment 1 also completed Experiment 3 during the same session. Binocular OFR recordings were collected from 57 infants (34 female) across the age groups and averaged for the two eyes: twenty-four 12-week-olds; twenty 16-week-olds; and thirteen 20-week-olds. Data from an additional 40 infants were dropped (including all seven infants tested in the youngest age group) due to poor calibration or low eye gaze sampled (n ¼ 21); sleepiness or fussiness (n ¼ 14); or experimenter error or equipment failure (n ¼ 5).

OFR was quantified using response amplitude, the magnitude of eye movement over a fixed time interval (Perrinet & Masson, 2007). The four eye tracking recordings for leftward and rightward trials were averaged for each infant. The first 100 ms of the response was omitted, and infants were assessed as generating OFR if their average response amplitude was greater than 0.88 in the subsequent 300 ms. This response amplitude corresponds to an average OFR velocity of 2.78/s or a response gain of about 0.1. Figure 8 illustrates the ocular following response of three infants to the translating grating. On average, two patterns emerged: 28 infants demonstrated either absent (flat) or asymmetric OFR (mean age ¼ 97 days); 24 infants demonstrated symmetric OFR (mean age ¼ 114 days). The difference in age between infants with symmetric versus asymmetric responses was significant, t(38) ¼ 2.59, p , 0.05. For infants who produced both smooth pursuit gain values (Experiment 2) and OFR, the difference in pursuit gain between infants with symmetric OFR compared to those with flat OFR responses was not significantly different, t(29), ¼ 0.82, p . 0.05. Pursuit gain differences cannot be compared using the asymmetric groups from Experiment 2 as they produced significantly different leftwards and rightwards values. However, infants who demonstrated asymmetric OFR also demonstrated asymmetric smooth pursuit tracking. Of the infants with both smooth pursuit and OFR data, 77% percent (10/13) with asymmetric OFR scores also demonstrated asymmetric pursuit gain. None of the infants with either symmetric or flat OFR scores demonstrated asymmetric pursuit gains (0/30). Forty-six infants produced both OFR data and completed the motion parallax Experiment 1. Chisquare analysis found no significant correlation between OFR group (correct, asymmetric, flat) and dishabituation to motion parallax, v2(2) ¼ 1.51, p . 0.05. The results from Experiment 3 add to the literature that exists on the development of ocular following response eye movements, with symmetric responses developing on average by approximately 15 weeks of age. Interestingly, infants with a symmetric OFR did

Stimulus The OFR stimulus was a large 358 · 288 field containing a vertically-oriented square-wave grating (2.78/cycle, 100% contrast). To elicit OFR, the grating translated leftward or rightward at a velocity of 258/s (0.258 steps every 10 ms) for a period of 400 ms. In an attempt to maximize OFR magnitude, a saccadic enhancement procedure was employed (Gellman, Carl, & Miles, 1990; Takemura & Kawano, 2006).To attract the infant’s gaze to different screen locations, the ‘‘happy-face’’ attention-getting stimulus (AGS from Experiment 1) was again presented at 25 Hz. Procedure The experimental set up and eye-tracking procedure was similar to Experiment 2 and completed within the same session. Each trial began with the OFR grating stimulus drawn on the screen. Infant gaze was first drawn to a location 128 left or right of the stimulus center with the AGS. Following a 100 ms period with .70% valid eye track records with .50% within the AGS region, the stimulus immediately moved to the stimulus center. This stimulus movement maximized the likelihood of a saccade back to the stimulus center, which triggered the OFR stimulus presentation. For this, one valid eye track record within the center stimulus region caused the AGS to vanish, leaving just the stationary grating. Thirty ms later the OFR stimulus began translating leftward or rightward for 400 ms. There was no relationship between the initial



Nawrot & Nawrot

9

Figure 8. (a–f). Six panels illustrate the ocular following response eye movements from three infants: Each panel represents the responses averaged across four trials of either leftwards (a, c, e) or rightwards (b, d, f) translation. Response amplitudes of 0.88 in the direction of stimulus translation were classified as evidence of the infant having OFR in that direction. Panels (a) and (b) illustrate symmetric responding (16-week-old); panels (c) and (d) illustrate flat responding (12-week-old infant); panels (e) and (f) illustrate asymmetric responding (20-week-old).

not necessarily demonstrate more mature smooth pursuit gains. However, those who did demonstrate asymmetric OFR also demonstrated asymmetric smooth pursuit (Experiment 2). There was no apparent relationship between OFR and sensitivity to motion parallax in these infants.


Discussion The research reported here is among the few studies of depth perception from motion parallax in infancy and adds considerably to our understanding of the development of this monocular, motion-based cue to


Nawrot & Nawrot

depth. Experiment 1 finds evidence for sensitivity to depth from motion parallax by 12 weeks of age. This estimate, based on a cross-sectional design, is similar to the findings from a previous longitudinal study (E. Nawrot et al., 2009). Additional converging evidence for the sensitivity to depth from motion parallax using alternate measures come from Condry and Yonas (2013) who found that infants reach to a stimulus whose distance is conveyed via motion parallax cues by 6 months of age. Therefore, it appears that the perception of depth from motion parallax develops quite early, well before infants begin to ambulate on their own. In the current study a one might propose that infants did not dishabituate to a global change in perceived depth-phase, but to a local change in stimulus dot motion tied to the direction of stimulus window translation. As we noted before (E. Nawrot et al., 2009), this is indeed possible, but quite unlikely. Such a ‘‘low-level’’ explanation would require infants to detect and remember the combination of different movements, not just the local dot motions. This is especially difficult because throughout a trial, equal numbers of stimulus dots are moving in leftward and rightward directions (at any instant), each particular dot moves equally (over the trial duration) in leftward and rightward directions, and the stimulus window moves equally (over the trial duration) in leftward and rightward directions. Moreover, naive adult observers have difficulty keeping track of these variables. While adult observers quickly and easily report depth changes in the motion parallax stimulus, they are typically unable to describe the stimulus changes that caused the perceptual change. Therefore, we believe that any changes in infant looking time are linked to changes in perceived depth in the motion parallax stimulus, not to changes in local stimulus motion. However, the data from the partial-lag group in Experiment 1 suggest some caution in interpreting the current results as there was evidence for spontaneous dishabituation. But, since this spontaneous dishabituation was not related to age, it does not contribute a systematic bias to our understanding of how the perception of depth from motion parallax develops with age. This spontaneous dishabituation does suggest that dishabituation to a change in depth-sign or depthphase is not an ideal index of sensitivity to depth from motion parallax. The relatively high number of trials to reach habituation criterion suggests that infants found the stimulus interesting or engaging, and they habituated to it very slowly. This likely contributed to the spontaneous dishabituation. However, the large percentage of responses near the criterion point (PTLT change of 1.0) suggests that infants who failed to show large differences in looking times may have noticed the change in depth sign but did not find the test stimulus a


10

great deal more interesting than the habituation stimulus. It would be desirable to use a more meaningful measure of discrimination such as reaching Condry and Yonas (2013); however, this is typically not found reliably in infants until at least 5 months of age and so could not be used to examine the onset of depth perception from MP before this age. Future studies might use preferential looking to depth specified by MP. This measure has recently been used to examine infants’ sensitivity to depth-sign in the monocular cue of height in the picture plane (Tsuruhara, Corrow, Kanazawa, Yamaguchi, & Yonas, 2010). The results of Experiment 2, to examine smooth pursuit tracking, found a strong correlation between pursuit gain and age, also in keeping with previous literature (Rosander & von Hofsten, 2002; von Hofsten & Rosander, 1997). Combined with the results from Experiment 1, this suggests that the maturity of the smooth pursuit system may be involved in the development of sensitivity to depth from motion parallax. That is, in infancy as in adulthood, a smooth pursuit eye movement signal is necessary to disambiguate depth-sign from motion parallax (M. Nawrot, 2003; M. Nawrot & Joyce, 2006). Given the cross sectional design, we cannot directly compare every individual infant’s performance on the motion parallax task to their smooth pursuit gain as not all infants provided data in both experiments. However, average pursuit gains did not relate to performance on the MP task as we might have expected from the adult literature. Again, it may be that the measure of MP in Experiment 1 is not ideal for this comparison. In the future, it would be ideal to measure segments of smooth pursuit gain during the motion parallax task instead of using a separate tracking task to index smooth pursuit. In this way, we might be able to make predictions of sensitivity to depth from motion parallax for individual infants based on their pursuit gain in the actual task. The results of Experiment 3 indicate that infants’ smooth pursuit gain was related to their ocular following response, but we did not find a direct relationship between OFR and our measure of depth from MP. Both age and pursuit gain correlated to performance on the OFR task. Symmetric OFR occurred in infants on average two weeks later than asymmetric OFR, and we found a positive relationship between infants who demonstrated asymmetric function in both types of eye movements. Previous research also finds asymmetries in the development of infant eye movements, specifically with the monocular optokinetic response (OKR). The OFR and the OKR are believed to be quite similar, both being reflexive smooth eye movements driven by large fields of motion (Krauzlis, 2004). The velocity of the OKR is reduced in the NT direction but not the TN direction early in development


Nawrot & Nawrot

(Braddick & Atkinson, 2011; Shupert & Fuchs, 1988). The development of a symmetric OKR response presumably reflects the development of underlying cortical mechanisms (Braddick & Atkinson, 2011). However, this NT asymmetry remains in individuals who develop esotropia, which leads to a deficit of binocular stereopsis and possibly amblyopia (Demer & von Noorden, 1988; Westall & Shute, 1992). The asymmetries we report in the present experiment for both smooth pursuit and OFR represent eye movements summed across both eyes (binocular recordings) so we cannot directly examine the monocular NT or TN response. However, the presence of these smooth eye movement asymmetries and their relationship to age suggests that both smooth pursuit and OFR eye movements may provide an important developmental marker for normal depth perception. While early asymmetries may be normal, asymmetries later in infancy might be an indicator of visual dysfunction related to problems in depth perception, with motion parallax lying at the intersection. Consider that both psychophysical and neurophysiological evidence suggest a common substrate for the processing of depth from stereopsis and motion parallax. For example, M. Nawrot and Blake (1989; see also Bradley, Chang, & Anderson, 1998) have shown that stereopsis and motion parallax have early, low-level interactions that produce intercue aftereffects. Electrophysiological recordings in nonhuman primates have established that visual processing of motion in the brain takes place primarily in cortical areas medial temporal (MT) and medial superior temporal (MST) (e.g., Maunsell & Van Essen, 1983). Neurons in these areas respond selectively to visual motion with various properties, such as velocity and direction, and are also sensitive to binocular disparity, suggesting that areas MT and MST process both depth and motion. At the very least, there is evidence to suggest that motion parallax and stereopsis are ‘‘co-processed in the visual system’’ of nonhuman primates (Cao & Schiller, 2002). There is also evidence that neurons in area MT respond to information about depth from motion parallax in the absence of other depth cues (including disparity). Nadler et al. (2009) demonstrated that single cells in monkey area MT signal the depth sign from a motion parallax stimulus. If motion parallax and stereopsis are processed in interacting mechanisms, it is possible that motion parallax might develop before binocular stereopsis and perhaps even serve as the foundation of the depth processing networks in cortical area MT. The demonstration of a role for pursuit in the depth/motion selectivity in MT cells (Nadler et al., 2009) certainly suggests this is a possibility. Therefore, it is crucial that future research also directly compare a motion parallax and stereopsis version of the same stimulus to be tested


11

under identical conditions when the smooth pursuit eye movement system has matured appropriately. Future research should also investigate whether motion perception, pursuit eye movement gain, depth from motion parallax, and binocular stereopsis develop sequentially, one aiding the development of the next. In this hypothesis, the development of motion parallax serves as a developmental foundation for stereopsis in higher cortical areas, proving a functional link between the correlated eye movement, motion parallax, and stereopsis dysfunction seen in esotropia (M. Nawrot et al., 2008; M. Nawrot et al., 2004). Keywords: infant perception, motion parallax, eye movements, depth perception

Acknowledgments Portions of this research were presented at the Vision Sciences Society Meeting in Naples, Florida, May 2011, and at the International Conference on Infant Studies Meeting in Minneapolis, Minnesota, June 2012. The authors thank Jennifer Livingood, Jennifer Wenner, Kelsey Ihringer, Megan Nere, and Megan Week for help with data collection, Enrique Vasquez Alvarez and Dan Gu for programming, and Dr. Diana Divecha. This work was supported by NICHHD R15HD058179 (EN) and COBRE NIH P20 GM103505 (MN). Commercial relationships: none. Corresponding author: Elizabeth Nawrot. Email: [email protected]. Address: Department of Psychology, Minnesota State University Moorhead, Moorhead, MN, USA.

References Bertenthal, B. I., Haith, M. M., & Campos, J. J. (1983). The partial-lag design: A method for controlling spontaneous regression in the infant-control habituation paradigm. Infant Behavior & Development, 6(2), 331–338. Braddick, O., & Atkinson, J. (2011). Development of human visual function. Vision Research, 51(13), 1588–1609. Bradley, D. C., Chang, G. C., & Andersen, R. A. (1998). Encoding of three-dimensional structurefrom-motion by primate area MT neurons. Nature, 392, 714–717. Busettini, C., Miles, F. A., Schwarz, U., & Carl, J. R. (1994). Human ocular responses to translations of the observer and of the scene: Dependence on


Nawrot & Nawrot

viewing distance. Experimental Brain Research, 100, 484–494. Cao, A., & Schiller, P. H. (2002). Behavioral assessment of motion parallax and stereopsis as depth cues in rhesus monkeys. Vision Research, 42, 1953– 1961. Condry, K., & Yonas, A. (2013). Six-month-old infants use motion parallax to direct reaching in depth. Infant Behavior & Development, 36(2), 238–244. Demer, J. L., & von Noorden, G. K. (1988). Optokinetic asymmetry in esotropia. Journal of Pediatric Ophthalmology and Strabismus, 25, 286– 292. Domini, F., & Caudek, C. (2011). Combining image signals before three-dimensional reconstruction: The intrinsic constraint model of cue integration. In J. Trommershäuser, K. P. Körding, & M. S. Landy (Eds.), Sensory cue integration (pp. 120–143). New York: Oxford University Press. Farber, J. M., & McConkie, A. B. (1979). Optical motions as information for unsigned depth. Journal of Experimental Psychology: Human Perception & Performance, 5, 494–500. Gellman, R. S., Carl, J. R., & Miles, F. A. (1990). Short latency ocular-following responses in man. Visual Neuroscience, 5(2), 107–122. Horowitz, F. D., Paden, L., Bhana, K., & Self, P. (1972). An infant controlled procedure for studying infant visual fixations. Developmental Psychology, 7, 90. Kawano, K. (1999). Ocular tracking: Behavior and neurophysiology. Current Opinion in Neurobiology, 9, 467–473. Krauzlis, R. J. (2004). Recasting the smooth pursuit eye movement system. Journal of Neurophysiology, 91, 591–603. Masson, G. S. (2004). From 1D to 2D via 3D: Dynamics of surface motion segmentation for ocular tracking in primates. Journal of Physiology, 98(1–3), 35–52. Maunsell, J. H., & Van Essen, D. C. (1983). Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. Journal of Neurophysiology, 49(5), 1127–1147. Miles, F. A. (1998). The neural processing of 3-D visual information: Evidence from eye movements. European Journal of Neuroscience, 10, 811–822. Nadler, J. W., Nawrot, M., Angelaki, D. E., & DeAngelis, G. C. (2009). MT neurons combine visual motion with a smooth eye movement signal


12

to code depth sign from motion parallax. Neuron, 63, 523–532. Nawrot, E., Mayo, S. L., & Nawrot, M. (2009). The development of depth perception from motion parallax in infancy. Attention, Perception, & Psychophysics, 71(1), 194–199. Nawrot, M. (2003). Eye movements provide the extraretinal signal required for the perception of depth from motion parallax. Vision Research, 43, 1553– 1562. Nawrot, M., & Blake, R. (1989). Neural integration of information specifying structure from stereopsis and motion. Science, 244, 716–718. Nawrot, M., Frankl, M., & Joyce, L. (2008). Concordant eye movement and motion parallax asymmetries in esotropia. Vision Research, 48(6), 799–808. Nawrot, M., Frankl, M., & Stockert, C. (2004). Elevated motion parallax thresholds are related to eye movement anomalies in strabismus. Journal of Vision, 4(8):202, http://www.journalofvision.org/ content/4/8/202, doi:10.1167/4.8.202. [Abstract] Nawrot, M., & Joyce, L. (2006). The pursuit theory of motion parallax. Vision Research, 28, 4709–4725. Nawrot, M., & Stroyan, K. (2009). The motion/pursuit law for visual depth perception from motion parallax. Vision Research, 49(15), 1969–1978. Nawrot, M., & Stroyan, K. (2012). Integration time for the perception of depth from motion parallax. Vision Research, 59, 64–71. Norcia, A. M. (1996). Abnormal motion processing and binocularity: Infantile esotropia as a model system for effects of early interruptions of binocularity. Eye, 10, 259–265. Perrinet, L. U., & Masson, G. S. (2007). Modeling spatial integration in the ocular following response using a probabilistic framework. Journal of Physiology, 101(1–3), 46–55. Phillips, J. O., Finocchio, D. V., Ong, L., & Fuchs, A. F. (1997). Smooth pursuit in 1- to 4-month-old human infants. Vision Research, 37, 3009–3020. Rogers, B., & Graham, M. (1979). Motion parallax as an independent cue for depth perception. Perception, 8, 125–134. Rosander, K., & von Hofsten, C. (2002). Development of gaze tracking of small and large objects. Experimental Brain Research, 146, 257–264. Shupert, C., & Fuchs, A. F. (1988). Development of conjugate human eye movements. Vision Research, 28(5), 585–596. Takemura, A., & Kawano, K. (2006). Neuronal responses in MST reflect the post-saccadic enhancement of short-latency ocular following re-


Nawrot & Nawrot

sponses. Experimental Brain Research, 173(1), 174– 179. Tsuruhara, A., Corrow, S., Kanazawa, S., Yamaguchi, M. K., & Yonas, A. (2010). Infants’ perception of depth from a pictorial cue: Comparing monocular and binocular preferential-looking. Perception, 39, 68. Tychsen, L., Hurtig, R. R., & Scott, W. E. (1985). Pursuit is impaired but the vestibulo-ocular reflex is normal in infantile strabismus. Archives of Ophthalmology, 103(4), 536. von Hofsten, C. (2004). An action perspective on motor development. Trends in Cognitive Sciences, 8, 266– 272.


13

von Hofsten, C., Kellman, P., & Putaansuu, J. (1992). Young infants’ sensitivity to motion parallax. Infant Behavior & Development, 15, 245–264. von Hofsten, C., & Rosander, K. (1997). Development of smooth pursuit tracking in young infants. Vision Research, 37, 1799–1810. Westall, C. A., & Shute, R. H. (1992). OKN asymmetries in orthoptic patients: Contributing factors and effect of treatment. Behavioral Brain Research, 49, 77–84. Westall, C. A., Woodhouse, J. M., & Brown, V. A. (1989). OKN asymmetries and binocular function in amblyopia. Ophthalmic & Physiological Optics, 9(3), 269–276.

The neural basis of depth perception from motion parallax.

Motion parallax thresholds for unambiguous depth perception.

Eye movements during motion after-effect.

The effects of aging on the perception of depth from motion parallax.

In pursuit of perspective: does vertical perspective disambiguate depth from motion parallax?

Motion parallax as an independent cue for depth perception.

Judging object motion during smooth pursuit eye movements: the role of optic flow.

Correction to 'The neural basis of depth perception from motion parallax'.

V1 neurons respond differently to object motion versus motion from eye movements.

The perception of heading during eye movements.

Vergence eye movements are not essential for stereoscopic depth.

Eye Movements During Action Observation.

Information transmission during eye movements.

Eye movements and the motion aftereffect: alternatives to the induced motion hypothesis.

The role of eye movements in a contour detection task.

Illusory position shift induced by motion within a moving envelope during smooth-pursuit eye movements.

Binocular motion rivalry in macaque monkeys: eye dominance and tracking eye movements.

Muscle tension during unrestrained human eye movements.

Eye movements during reading: case reports.

The Role of Dopamine in Anticipatory Pursuit Eye Movements: Insights from Genetic Polymorphisms in Healthy Adults.

A Pursuit Theory Account for the Perception of Common Motion in Motion Parallax.

Contrast sensitivity during saccadic eye movements.

Evidence for a role of corrective eye movements during gaze fixation in saccade planning.

Eye movements during the viewing of Necker cubes.