0042-6989/92 WOO + 0.00 Copyright 0 1992 Pergamon Prcsa Ltd
Vision Res. Vol. 32, No. 11, pp. 2177-2186, 1992 Printed in Great Britain. All rights -cd
Extrapolation M. PAVEL,*
of Linear Motion
H. CUNNINGHAM,?
V. STONE?
Received 7 February 1991; in revised form 21 November 1991
We iavestigated observers’ ability to extrapolate a linear trajectory of a moving point, in order to dw hsw eiTectively the visual system can combine orientation and position information for mo~~Pli.ObsenerssPwaprobedotmoviagPloagastrslisbtliaetowuda~tiolruytuget dot. The probe dot exhguiskd before reaching the targeh aad the observers’ task was to judge whether aa extrapolation of the trajectory of the probe would pass to the left or right of the target. Performance wasmeasued as a function of probe velocity, leagtb of tbe visible trajectory, and location of the target. Tbe empirical resultg indicated that over a raage of conditl~ performance on this task is qualitatively simllar to, but somewhat less acfzwate tbm, that on an eoas task witb static stimuli. A four-compoaent model ls preseated to account for tbe reaulta The model specilk an accurate extraction of probe motion parameters, extrapolatioo of the metion by aa ideal observer, and limitatiolls on the input to tbeae proceMS in the form of visual field spatial inhomogeneity and temporal decay of position information.
INTBODUCTION
BACKGROUND
Successful performance of many perceptual and motor tasks requires not only a sufbcient internal representation of the current state of the world, but also the ability to predict future states. Otherwise, inherent delays in information processing and response generation would result in responses that are unstable or too slow. For example, accurate tracking of moving objects with eye movements often requires prediction and extrapolation over time intervals of 200-500 msec (see Pavel, 1990). That humans perform well on many everyday visual and motor tasks indicates their ability to predict and to extrapolate. The goal of the present study was to determine how accurately people can perceive and extrapolate linear trajectories (i.e. the path in space taken by moving objects) by having observers perform the task that is illustrated in Fig. 1. We assessed observers’ performance for probes moving along uniform linear paths confined to a two-dimensional plane perpendicular to the observers’ line of sight. On each trial the observer is shown, on a display screen, a luminous dot (the probe) that is moving along a specific path (the trajectory) toward a luminous dot that is stationary (the target). The probe disappears at a specified distance prior to reaching the target. The observer is asked to indicate whether the probe would pass to the left or to the right of the target if it continued on its present course.
Prior experiments (Westheimer & McKee, 1975) have shown that, for briefly exposed vernier acuity stimuli moving with velocity less than three degrees per second, vernier threshold is approximately the same as for stationary stimuli. Observers in these experiments judged the relative position of two features within a moving object. Observers can perform such a task by analyzing the moving stimulus and need not combine location and motion information. Considering the spatial aspects alone, the extrapolation task that we used is directly comparable to the one used by Salomon (1947) and later by Bouma and Andriessen (1968) to investigate perception of the orientation of line segments. Salomon (1947) projected images of lines of various lengths on a wall, with a single point at a set distance (gap) from the end of the line. Observers were required to adjust the position of the single point to coincide with the extension of the line. Using three different line lengths, a constant absolute line orientation, and different gap sixes, Salomon obtained a linear relationship between the gap size and the standard deviation of the adjustments, with a slope ranging from l/90 to l/40. Bouma and Andriessen (1968) used Salomon’s method in a study of the effect of the absolute line orientation on the extrapolation adjustments. They found that the accuracy of the adjustments varied with the absolute orientation (horizontal and vertical lines were adjusted more accurately than were oblique ones), but the overall error was generally low, typically less than 2”. These studies suggest that observers extrapolate static lines to within 1” of angular change in line orientation.
*Department of Psychology and Center for Neural Science, New York University, 6 Washington Place, Rm 959, New York, NY 10003, U.S.A. tDepartment of Psychology, Stanford University, Stanford, CA 94305, U.S.A. 2177
2178
M. PAWS. et al.
Procedure
PROBE
Initial position
-c
1-@OtO~
length s
Visible traiectov
~
FIGURE 1. A diagram of the prediction task. The probe, shown in its initial position, is moving along the visible trajectory indicated by the solid line. The probe disappears at the end of the solid line. The distance between the target and the trajectory is designated by d.
At the beginning of each trial, a probe point appeared at a random location in the upper half of the display area. After 2OOmsec, the probe began to move along a straight trajectory with uniform velocity. As the probe began to move, a stationary target appeared in the lower half of the display. The location of the target was randomly varied over several degrees with respect to the true extrapolation of the trajectory. The moving probe was displayed for a variable period of time, and then disappeared. The observer’s task was to indicate whether the extrapolated trajectory would have passed to the left or to the right of the target. We call the visible portion of the trajectory the visible path, and the distance between the final position of the visible path and the target the gnp. Approximately 500 msec after the observer’s response, the probe appeared in a starting location for the next trial. After each response, observers were given feedback about the correctness of their responses. Apparatus
The static extrapolation task can be used as a reference to evaluate the effects of motion. To perform this extra~lation task with a moving probe, the observer must use information about current and prior locations of the probe. For some conditions, such as with very slow motion, the extrapolation task should be more diEcult because information about prior locations might be lost. For conditions under which observers are found to perform less accurately with moving stimuli than they do with static ones, the difference can be used to estimate the uncertainty (i.e. noise) due to the inability to retain and integrate information over time.* By varying the velocity and length of the visible trajectories independently, we can assess the noise due to spatial un~~ainty and to temporal integration. To quantify the contribution of these sources of uncertainty, we developed a model incorporating both spatial and temporal effects. We performed three experiments; the first measured extrapoIation performance as a function of distance, the second evaluated the effects of motion arrears, and the final assessed the performance on a similar, but static, test. EXPEIl#MlMT
1
The two stimuli in this experiment, the target and the probe, were luminous dats approx. 1 min of arc in diameter, presented on a CRT display which subtended a diameter of approximately 10”. The luminance of the stimuli exceeded thresholds of visibi~ty by two log units at the highest speeds. The stimulus position was updated at 5-msec intervals. *An equivalent interpretation can be given in terms of the direction sensitivity motion detectors.
The stimuli were presented on a CRT screen of a vector graphics processor (HP-1345) with a P31 phosphor. The display, controlled by an II?M/XT ~~~ornputer, was mounted in one channel of a modified two-channel tachistoscope enclosure, so that no extraneous contours were visible. The other-channel of the tachistoscope was used to generate a background field consisting of a uniform central disk, at approximately 50cd/mz, with edges approximating Gaussian blur. The background &mination bid the borders of the CRT screen, and masked the slow component of -the phosphor persistence. The observer’s head was partially stabilized by a chin rest and viewing goggles. The observer viewed the stimuli binocularly from a distance of 6Ocm. Observers
Three observers with normal vision participated in this experiment. Two practiced observers were authors; the third was a naive observer who was paid for her participation. Conditions
The orientation and the starting position of the trajectory were varied randomly from trial to trial to eliminate any absolute position cues. The orientation was varied over an angular range of f 10” around the vertica1 meridian, and the horizontal starting position was jittered in the range of f0.5”. The length of the visible trajectory was 46.9 min of arc. Combined with a velocity of 2346 n&/see, this resulted in a 2OOmsec period during which the probe was visible. The gap was varied randomly over a block of 300 trials, with the gap size ranging from 47 min to 6” of arc. The horizontal location of the target relative to the true extrapolated trajectory was controlled by randomly interleaved staircases. There
EXTRAPOLATION
OF LINEAR MOTION
2179
f
0
100
200
300
GAP SIZE [mln]
400
0
100
200
300
400
GAP SIZE [min]
0
200
100
300
-4 400
GAP SIZE [min]
FIGURE 2. Just noticeable differences (JNDs) expressed in minutes of visual angle for three observers, plotted against gap size: (a) observer HC; (b) observer JG; (c) observer MP. The length of the visible trajectory was 46.88 min and the probe was moving at 3.91”/sec. The duration of the visible portion of the trajectory was 2OOmsec.
were two up-down staircases for each gap condition in each block. One staircase was designed to present stimuli for which the probability of a right-of-the-target response was in the neighborhood of the 0.7 level (41/2); the other staircase for the same condition presented stimuli near the 0.3 level. The overall probability of right-of-the-target judgments for the combination of the two staircases was l/2. We used this symmetric arrangement to minimize the probability of getting stable sequences when the observers responded randomly. The performance in each condition was determined using at least 200 trials for each observer. Results
We summarized observers’ responses by empirical psychometric functions representing the proportion of right-of-the-target judgments as a function of the perpendicular distance d between the actual extrapolated trajectory and the target illustrated in Fig. 1. We computed a maximum likelihood fit of the logistic function for each empirical psychophysical function. The logistic function of the form f(x) = l/[l + E-(~+~‘)] with two parameters, a, b > 0, is empirically indistinguishable from a cumulative Gaussian distribution function. We evaluated the goodness of fit of the psychometric function using a x2 test. The data in all conditions and for all observers were well approximated by the logistic function and none of the fits could be rejected at P = 0.05. The just noticeable difference (JND) was defined as the difference between the 50 and 75% points on the best fitting logistic function, which would correspond to d’ = 0.67 in the case of the normal approximation. The points of subjective equality, i.e. d,, such that P{right-of-the-target} = 0.5, were very small (~20% of the JNDs). Moreover we could not find any systematic variation of the dso with the experimental conditions. The standard deviation of the JND estimates was less than 10%. Therefore, we concluded that the JNDs represent an adequate summary of the data. Figure 2 shows the JND for the three observers as a function of gap size. The most obvious
features of these data are that the JNDs increase with gap size, and that this increase is well approximated by a straight line. We tested the linearity of the data by computing a linear regression for each observer. The resulting regression lines are shown in Fig. 2. Although there appears to be a nonlinear component in the results of observer HC, the departure from linearity is small in comparison to the variance accounted for by the linear term. The slopes for the regression lines, as well as the proportion of explained variance expressed by the squared correlation coefficients, r*, are shown in Table 1. The linear dependence of the JND on gap size is consistent with Salomon’s (1947) results obtained in a static extrapolation task using the method of adjustment. This linear dependence suggests that the noise affecting the extrapolation task represents constant orientation uncertainty because the peformance expressed as the ratio of the JNDs to the gap size is independent of the gap size g. The ratio JND/g = tan(A4), where A4 represents the JND in the angle or the orientation of the trajectory. The estimates of the angular JNDs are included in the second row of Table 1. Although these data show similar behavior, the angular threshold values are higher than those obtained by Salomon (1947) and later by Bouma and Andriessen (1968) in static extrapolation tasks. Our hypothesis is that the decrement in performance occurred because in the present experiment, in addition to the probe motion, both orientation and position of the trajectory were varied randomly. In Experiment 2 we measured the effects of the length of the visible trajectory and the velocity of the probe. TABLE 1
Slope Angle r2
HC
Observer MP
GJ
0.0644 3.68 0.934
0.0438 2.5 0.971
0.0875 5.0 0.999
2180
M. PAVEL el al EXPERIMENT
2
The data of Salomon (1947) indicate that the precision of extrapolation adjustments improves with line length. Other researchers have demonstrated that the discrimination of the orientation of static lines in the fovea improves with the length of short lines (< 1”) and is independent of the stimulus length for longer lines (e.g. Paradiso & Camey, 1988; Andrews, Butcher & Buckley, 1972; Watt, 1987). In this experiment, we investigated the effect of the length and velocity of the visible trajectory on extrapolation judgments. The stimuli, apparatus, procedure, and observers were identical to those in Experiment 1. Conditions
The gap was kept constant at 3.1”. We used a mixed list design, in which variable path lengths and probe velocities were ordered randomly within a block of 300 trials. Two independent variables characterized the motion of the probe: (1) the visible path length, ranging from 11.5 to 70 min of arc, and (2) the probe velocity, ranging from 1 to 7.7”/sec. The duration of the visible path depended on the path length and on the velocity; it ranged from 43.3 msec to 1.7 sec. Results
Figure 3 illustrates JND as a function of the length of the visible trajectory for one observer. It shows that JND decreases with path length up to a critical value of approximately 30 min of arc. For longer visible trajectories, the performance reaches an asymptotic level. In the trajectory extrapolation task, the lack of further improvement could be due to either spatial limitations or temporal limitations on the ability to integrate information. The data in Fig. 3 suggest that the limitation is spatial, because the path length cutoff is about the same
30
for all velocities. These data are comparable to the results of static orientation judgments reported by Watt (1987). Although Watt’s task and stimuli were different from ours, the dependency of response accuracy on line length was similar to that reported here. In particular, Watt’s data reached asymptotic values for line lengths between 0.5 and 2”, depending on the exposure duration. Although the results of the trajectory extrapolation task considered thus far are qualitatively similar to those obtained with stationary stimuli, there is one important difference between the two tasks: the dynamic task involves an additional parameter-the velocity of the probe. To examine the effects of probe velocity, we plotted the observed JNDs as a function of probe velocity (Fig. 4). These JND functions indicate that extrapolation is more difficult with slower probes than with faster ones. Performance improves with increasing velocity but reaches an asymptotic level for velocities exceeding 3S”/sec. Asymptotic behavior was expected for very high velocities, because a rapidly moving point is perceived as a stationary line, but our data show that an asymptote was reached for velocities much slower than those required for the percept to become stationary. The form of these JND functions indicates the existence of certain limitations on human ability to retain and use prior positions of the probe. We shall discuss the effects of both temporal and spatial limitations when we describe the quantitative model. The results of Experiment 1 suggest that, in some respects, an observer’s performance on the dynamic extrapolation task is similar to that observed previously on a static extrapolation task. This fact is consistent with the possibility that performance on both tasks is limited by the same mechanism. We performed Experiment 3 to measure human extrapolation performance on a static task under conditions comparable to those in Experiments 1 and 2.
HC
25
20
-
-
2 E
0
58
mdsec mirdsec
0
77
0
116
minhec
+
230
midset
0
346
min/sec
A
460
min/sec
,5-
4 IO
-
5-
0 I
0
I
I
I
I
1
20
40
60
80
100
Path length lminl FIGURE 3. JNDs for one observer, as funcoions of the visible path length for six velocities. The gap was constant at 47 min of arc, and the parameter is the velocity of the probe.
2181
EXTRAPOLATION OF LINEAR MOTION 25 HC r
0
0
0
11.7
min
0
23.4
min
+
35.2
min
0
46.9
min
A
70.3
min
I
I
I
I
200
400
600
000
Velocity
Cmin/secl
FIGURE 4. JNDs for one observer as functions of probe velocity. The gap was constant at 47 min of arc, and the parameter is the length of the visible path.
EXPERIMENT
3
The procedure, apparatus, and stimuli in this experiment were identical to those in Experiment 1, except that the moving probe was replaced by a stationary line. In fact, because a stationary line is indistinguishable from a probe moving with a very high velocity, we expected that the performance on the static task would represent the upper limit of performance on the dynamic task. Observers Two observers participated in this experiment; was an author; the other was a naive volunteer.
one
Conditions In each condition, the line had the same length as the corresponding visible trajectory of the moving probe in Experiments 1 and 2, and was presented for the same
oI 0
100
200
300
400
GAP SIZE [mlnl
FIGURE 5. JNDs from Experiment 2 with stationary lines, for two observers, plotted against the gap size.
length of time that the motion was visible. The observers were asked to indicate on which side of the stationary target an extrapolation of the line would pass. Results The results of this experiment are shown in Fig. 5. As expected from prior empirical results (Salomon, 1947), these data show the same linear dependency of JNDs on gap size as did the corresponding data from Experiment 1. However, performance with a moving probe is somewhat worse than that on the static task. This decrement in peformance can probably be attributed to the effects of motion. The similarities between the results of Experiments 1 and 3 were used as a basis for a model in which both tasks are performed by the same mechanisms except for additional uncertainty due to the motion.
MODEL
In this section we describe a model of how an observer might perform the trajectory estimation and extrapolation task. The purpose of this model is to relate an observer’s performance on the extrapolation task to his internal representation of the stimulus. The model is based on the assumption that the image of the stimulus is contaminated by intrinsic perceptual spatial noise, and that the noisy signal is then processed by an ideal observer. The observed human performance, therefore, can be characterized by the properties of this intrinsic noise which may depend on time. The assumptions concerning the representation are similar to those proposed by Andrews and his colleagues (Andrews, 1967; Andrews, Butcher & Buckley, 1972) to characterize perception of orientation of contours. A similar approach was used later to characterize detection of curvature (Watt & Andrews, 1982) and detection of deviations from straight lines (Watt, Ward Br Casco, 1987). One
2182
M. PAVEL et al.
9 .I I
Actual Trajectory
1 I 1 I I i I
Estimated Trajectory
FIGURE 6. A diagram illustrating the model of the extrapolation process. The actual trajectory (solid line), the sampled noisy repmentation (dots), and the beat fitting straight line trajectory (dashed line).
distinction between the prior and the current work is that, for the extrapolation task, orientation information must be combined with position information. In addition, in our model, the observer is assumed to compute his decision only from the information available in the stimulus without the prior knowledge of the orientation of the line. The model is based on the assumption that a continuous trawory is represented by a set of discrete samples (e.g. responses of receptor units). The spatial uncertainty in the trajectory samples and in the location of the target is due to spatial Gaussian noise. We assumed that the observer knows a priori that the trajectory is a straight line, and that he determines his responses by minimizing distance errors. Thus, the sampled physical trajectory is given by Xi = a + flyi, where a and #Iare constants. The corresponding perceived trajectory is ui = xi + eXi, vi = yi + eYi, where eXi and eui are independent random variables with zero means and variances 0: and a%, respectively. Using this noisy percept of the trajectory, the observer must estimate the physical trajectory by finding the straight line that minimizes the sum of the perpendicular distances* between each point and the line, as illustrated in Fig. 6. We assumed that the observer computes the linear trajectory from the perceived sampled trajectory using a least squares minimization process such as principal component analysis. Such a process would fmd that line for which the orthoaxial (i.e. perpendicular to the line) errors are minimixed. Note that this computation can not, in general, be wormed by a linear regression. *The perpendicular distance between a line and a point is given by &(_)
= (ari + 4s + CY n2+b2 .
The variability of the observer’s estimated line would then determine how well he or she can perform on the extrapolation task. The variability of the principal components for a trajectory in a general position is difficult to compute in closed form. For the stimuli in our experiment, however, the variability of the first principal component can be well approximated by the variability of a regression line. In that case, the expected orthoaxial errors are approximately equal to the deviation along the perceived horizontal axis (a). This approximation is valid because the empirically observed errors are small (i.e. JNDs on the order of a few degrees), and the physical trajectories were nearly vertical. The observer determines the perceived trajectory tii = oi + fluj used for the extrapolation task by finding the intercept oi and the slope b using the standard linear regression formulae. The variability of the extrapolated trajectory and consequently the observers’ performance is determined by the variability of oi and B. If the origin of the coordinate system is placed at the center of the visible trajectory, then the variances of the line parameters are given by var(8) = ;,
var(fi) = n
CC
iz,("i-
(v))2
where (v ) is the average value of Vi. The model generates a response by computing the distance 2 between the perceived target location us and the estimate of the nearest point on the trajectory a = oi + /3.$ - U8
(1)
where v8 and uB are the perceived coordinates of the target. The probability of the response right-of-target, which is generated whenever d > 0, is the theoretical psychometric function Pr{right-of-target) = Pr{a > 0) = @[d/a(a)], where d is the actual distance, @ is a curnultive normal distribution and o(a) = ,/var@) is the standard deviation of d obtained from equation (1). Evaluation of observers’ performance, JND = 0.6750(a) requires estimation of the variance of d var(d) = at
,$(vi- s) the variance of d is determined mostly by the variability of the slope
1
126s ‘j2 JND z,kga,, s3 [
EXTRAPOLATION
and, consequently, the JND is approximately proportional to the gap size. The slope of the JND function is proportional to the spatial uncertainty represented by 6,. It decreases with the length of the visible trajectory, and increases with the distance between adjacent samples. Therefore, for a given level of performance, equation (3) specifies a tradeoff between the spatial density of sampling and the uncertainty associated with each sample. The contribution of these two parameters is, therefore, not distinguishable using the results of the present experiments. The linear relationship between the JND and the gap size also can be characterized by the just noticeable differences A4 in the orientation of the probe trajectory given by
OF LINEAR MOTION
2183
(a) Trajectory
Length
= 9.6 min
25.
14 4 min
20.
21 0 min 41 8 min
200
400
Gap Size
600
800
1000
1200
[mid
(b)
This model is based on the assumption that all perceived homogeneously sampled locations are available simultaneously. We evaluated this static homogenous model under those stimulus conditions for which we had human data. Figure 7 illustrates the predicted JNDs as functions of gap size for two values of the visible trajectory length. These JND functions account well for the data obtained in our experiments and for those obtained by Salomon (1947), reproduced in Fig. 8(a). The homogeneous static model, however, has several shortcomings. First, any regression line must pass through the center of the points which, in this case, is in the center of the visible trajectory. Therefore, this model predicts that performance for small gaps will be relatively poor, and will depend on the length of the trajectory, as indicated in Fig. 7. This prediction contradicts our experimental results, shown in Fig. 2, as well as those of Salomon, shown in Fig. 8(a). Another discrepancy between the data and the model is illustrated in Fig. 8(b), in which we compare the theoretical slopes to the results of Salomon (1947) shown in Fig. 8(a). The slopes of the theoretical JNDs exhibit greater dependency on the visible trajectory (or line) Trajectory
/
Length
= 30 min
r
100
200
Gap Size
[minl
300
400
FIGURE 7. Theoretical JND as a function of the gap size for two different lengths of the visible trajectory (30min and 60 min). The origin is at zero gap, and the length of the visible trajectories is shown as a negative gap on the horizontal axis.
0.04
- Mode’
Salomon
100
200
Line Length
(1947)
300
400
[min]
FIGURE 8. Results obtained by Salomon (1947) on the static extrapolation task. (a) Standard deviation of the adjustments averaged over 10 observers for line lengths 0.16”, 0.24”, 3.54”, and 6.97”. (b) Slopes of the functions in (a) as a function of the line length. The solid line represents the standard deviations generated by the model.
length than do those of human observers. For humans, performance on an extrapolation task improves with short lines or visible trajectories, but reaches an asymptotic level or improves only slightly for longer ones. This observation suggests that human observers can use only a limited portion of the trajectory in making their estimations. For the model, the improvement is not limited, and is proportional to l/~~‘~. Therefore, the static model cannot account for any velocity and duration data. These considerations motivated our introduction of spatial and temporal dependencies into the model. Our experimental results suggest that the human visual system has limited capability to integrate over space and time. To account for these limitations, we extended the static model to include both (1) spatial inhomogeneity in retinal sampling and spatial judgments, and (2) memory or temporal persistence. We modified the previously uniform spatial uncertainty cr,, to depend both on the location of each sample and on the time interval from stimulus offset. There is considerable experimental evidence demonstrating that spatial acuity decreases with increasing distance from the fovea (eccentricity) for a variety of
2184
M. PAVEL ef al.
spatial tasks (e.g. Westheimer & McKee, 1975; Levi et al., 1985). This spatial inhomogeneity can be interpreted as an increase in spatial uncertainty with increasing eccentricity. The dependency on eccentricity is typically approximated by I/( 1 + cr), where r is the eccentricity of a point defined as the distance from the fovea, and c is an experimentally determined positive constant (see, e.g. Levi et al., 1985). This formulation was used by Paradiso (1988) to model the effects of eccentricity and line length on orientation judgments. We incorporated spatial inhomogeneity in our model by modifying the variability of the perceived location of each sample point so that the variance increased with eccentricity in accordance with @z(r) = crz(1 + cr), where (T#is the standard deviation of the noise representing spatial uncertainty, and o, is the standard deviation in the fovea. An observer’s memory for the location of previously displayed information is of limited temporal extent. This memory might, in part, rely on visible persistence, which lasts between a few tens to few hundreds of msees (Farrell et al., 1990), depending on stimulus characteristics. A higher-level memory also may be involved in the extrapolation judgments, but the data suggest that much of the precise information is lost after a few hundred msec. We incorporated this temporal limitation into the model by assuming that the uncertainty of spatiaf information increases exponentially from the time of stimulus offset to according to exp( - t - t,,/T), with a fixed time constant z. Assuming that the spatial and temporal effects are mutually inde~ndent, the variance of the spatial noise is aE(r,t) = cri(
1 + cr)e”“.
(5)
We obtained a complete model by substituting this expression for the standard deviation of spatial noise in equation (2). This ~p~~n~tion is similar to that of the static model, except for the dependency on time and location. The corresponding regression or principalcomponent analysis must be replaced by the corresponding weighted version, where the weights are the reciprocal of the variance. The dependence of the weights on time and space is illustrated in Fig. 9. We compared the predictions made by our complete model to the observed results in two steps. First, we considered the static version without spatial inhomogeneity to account for the effects of gap size. Subsequently, we examined the effects of temporal Iimitation and spatiaf i~omogeneity. Because of spatial inhomogeneity, the performance of the model might depend on the direction of gaze, i.e. by the point of fixation at the time of the judgment. Additional analyses and simulation of the model revealed that the predicted performance on the trajectory extrapolation task does not depend significantly on the point of fixation. The independence obtains because there is a tradeoff between the accuracy of the perception of the trajectory and the perception of the target. If the fixation point is assumed to be located at the target, the
FIGURE
9. Spatial and temporal representation weighting function.
of the temporal
location of the target would be perceived with a high accuracy but the trajectory would have more variability. When the fixation point is on the trajectory, then the location of the target becomes uncertain. Because of the lack of expected effect of eye fixation, we assumed that the observers made their judgments while looking at the last point of the trajectory. This assumption is partially supported by the high accuracy of observers’ judgments of small gaps. Using this assumption, we compared the complete model’s predictions to the observed data; one such comparison is shown in Fig. 10. The dashed line shows the original model’s prediction; the solid line shows the modified model’s predictions. The modified model provides a good appro~mation to the data of observer HC, which are plotted as diamonds. To verify the predictions of the model that human extrapolation performance should be relatively iadependent of the fixation point, we carried out a set of informal experiments in which the observers performed the trajectory extra~lation task and were instructed to use one of three different fixation points: (1) the end of trajectory, (2) the midpoint and, (3) the target. Our results were consistent with our model’s predictions; the .IND did not vary consistently with the fixation instructions. The reason for not reporting the detailed results is that we were not able to monitor the observers’ eye movements, so as to confirm their compliance with the instructions. CONCLUSIONS
We studied the accuracy with which the visual system can extract simultaneously information about probe location and direction of motion, as well as the ability to combine position and velocity information over space and time. The extrapolation task that we developed required the visual system to combine these two components with a clear objective: to minimize errors between the predicted and the actual extrapolated trajectory. Although trajectory extrapolation is a simple task, it shares aspects with many other real-life tasks (e.g. perception of depth from
2185
EXTRAPOLATION OF LINEAR MOTION 30
-
25
-
20
-
0
HC 0
.c bs
min/sec
115
\ 15
\
2 )\ 10 ‘Y,\* 5
O \
-
-*-.
01 0
-*-
---
I
I
I
20
40
60
Path
--80
100
length Cminl
FIGURE 10. Comparison of the two versions of the regression model. The dashed line represents predictions of the simple regression model, and the solid line represents the linear regression approximation modified by temporal and spatial weighting. Diamond’s denote data from observer HC.
motion, manual control, etc.) that require encoding of location of a trajectory as well as of the direction of motion. Our data indicate that observers are able to perform an extrapolation task with high accuracy. The JNDs obtained on both the dynamic and the static extrapolation tasks increased linearly with the gap size in a similar manner to that obtained with static stimuli, and we developed a model for integration of motion and location information under the assumption that the only source of uncertainty arises in the coding of the retinal location. In this model, the direction of motion was derived from the location of static samples, and the integration was performed over space. Of course, equivalent information could have been derived from a model based on motion detectors. We developed the model to account for several aspects of the observed data: Perception of straight line: We assumed that the visual system can extract the parameters of a straight line by regression or principal-components analyses. Extrapolation: The visual system is capable of making efficient use of the estimated straight line. Spatial extent limitation: The efficient use of information is limited to a relatively small spatial extent. This limitation was modeled in terms of spatial inhomogeneity of the visual system, such that spatial uncertainty increases with eccentricity. Temporal extent limitation: Limited memory for motion and location information was accounted for by visible persistence. While the ability to integrate over time may be due to more complex
memory mechanisms, a simple exponential increase in uncertainty with time was sufficient to account for our empirical data. We have characterized the limits of performance in terms of postulated intrinsic noise, making no assumptions concerning the biological origins of this noise. However, since we were able to describe the behavior in terms of least-squares analyses (i.e. principal component or regression), there are many biologically plausible process models that would result in the same performance. For example, there are several recursive filtering schemes (Pavel, 1990) that approximate the optimal least-squares analysis. To discriminate among the various possibilities we would have to develop experiments that would require judgments of trajectories more complex than are those of straight lines.
REFERENCES Andrews, D. P. (1967). Perception of contour integration in central fovea. Part II: Spatial integration. Vision Research, 7, 999-1013. Andrews, D. P., Butcher, A. K. & Buckley, B. R. (1972). Acuities for spatial arrangement in line figures: Human and ideal observers compared. Vision Research, 13, 599-620. Bouma, H. & Andriessen, J. J. (1968). Perceived orientation of isolated line segments. Vision Research, 8, 493-507. Levi, D. M., Klein, S. A. & Aitsebaomo, P. K. (1985). Vernier acuity, crowding and cortical magnification. Vision Research, 25, 963-977. Paradiso, M. A. & Camey, T. (1988). Orientation discrimination as a function of stimulus eccentricity and size: Nasal/temporal retinal asymmetry. Vision Resewch, 28, 867-875. Pavel, M. (1990). Predictive control of eye movement. In Kowler, E. (Ed.), Eye movements and their role in visual and cognitive processes (pp. 71-114). Amsterdam: Elsevier. Salomon, A. D. (1947). Visual field factors in the perception of direction. American Journal of Psychology, 60, 6$-88. Ulhnan, S. U. (1979). The interpretation of uisuul mofion. Cambridge, Mass.: MIT Press.
2186
M. PAVEL er 01.
Watt, R. J. (1987). Scanning from coarse to fine spatial scales in the human visual system after the onset of a stimulus. Journal C$ the
of retinal image-motion. Journul of the Optical Sociery oJ ,4mericu, 65, 847-850.
Optical Society of America, 4, 200&2021.
Watt, R. J. & Andrews, D. P. (1982). Contour curvature analysis: Hyperacuities in the discrimination of detailed shape. Vision Research, 22, 449-460.
Watt, R. J., Ward, R. M. & Casco, C. (1987). The detection of deviation from straight lines. Vision Research, 27, 1659-1678. Westheimer, G. & McKee, S. P. (1975). Visual acuity in the presence
Acknowledgements-This
work was supported by grants to Stanford University from the Air Force OfTice for Scientific Research, AFOSR-84-03-08, and NASA, NCC 2-269. We thank Dr E. Kowler and L. Dupre for useful discussions and their comments on this manuscript.
Vision Res. Vol. 32, No. 11, pp. 2177-2186, 1992 Printed in Great Britain. All rights -cd
Extrapolation M. PAVEL,*
of Linear Motion
H. CUNNINGHAM,?
V. STONE?
Received 7 February 1991; in revised form 21 November 1991
We iavestigated observers’ ability to extrapolate a linear trajectory of a moving point, in order to dw hsw eiTectively the visual system can combine orientation and position information for mo~~Pli.ObsenerssPwaprobedotmoviagPloagastrslisbtliaetowuda~tiolruytuget dot. The probe dot exhguiskd before reaching the targeh aad the observers’ task was to judge whether aa extrapolation of the trajectory of the probe would pass to the left or right of the target. Performance wasmeasued as a function of probe velocity, leagtb of tbe visible trajectory, and location of the target. Tbe empirical resultg indicated that over a raage of conditl~ performance on this task is qualitatively simllar to, but somewhat less acfzwate tbm, that on an eoas task witb static stimuli. A four-compoaent model ls preseated to account for tbe reaulta The model specilk an accurate extraction of probe motion parameters, extrapolatioo of the metion by aa ideal observer, and limitatiolls on the input to tbeae proceMS in the form of visual field spatial inhomogeneity and temporal decay of position information.
INTBODUCTION
BACKGROUND
Successful performance of many perceptual and motor tasks requires not only a sufbcient internal representation of the current state of the world, but also the ability to predict future states. Otherwise, inherent delays in information processing and response generation would result in responses that are unstable or too slow. For example, accurate tracking of moving objects with eye movements often requires prediction and extrapolation over time intervals of 200-500 msec (see Pavel, 1990). That humans perform well on many everyday visual and motor tasks indicates their ability to predict and to extrapolate. The goal of the present study was to determine how accurately people can perceive and extrapolate linear trajectories (i.e. the path in space taken by moving objects) by having observers perform the task that is illustrated in Fig. 1. We assessed observers’ performance for probes moving along uniform linear paths confined to a two-dimensional plane perpendicular to the observers’ line of sight. On each trial the observer is shown, on a display screen, a luminous dot (the probe) that is moving along a specific path (the trajectory) toward a luminous dot that is stationary (the target). The probe disappears at a specified distance prior to reaching the target. The observer is asked to indicate whether the probe would pass to the left or to the right of the target if it continued on its present course.
Prior experiments (Westheimer & McKee, 1975) have shown that, for briefly exposed vernier acuity stimuli moving with velocity less than three degrees per second, vernier threshold is approximately the same as for stationary stimuli. Observers in these experiments judged the relative position of two features within a moving object. Observers can perform such a task by analyzing the moving stimulus and need not combine location and motion information. Considering the spatial aspects alone, the extrapolation task that we used is directly comparable to the one used by Salomon (1947) and later by Bouma and Andriessen (1968) to investigate perception of the orientation of line segments. Salomon (1947) projected images of lines of various lengths on a wall, with a single point at a set distance (gap) from the end of the line. Observers were required to adjust the position of the single point to coincide with the extension of the line. Using three different line lengths, a constant absolute line orientation, and different gap sixes, Salomon obtained a linear relationship between the gap size and the standard deviation of the adjustments, with a slope ranging from l/90 to l/40. Bouma and Andriessen (1968) used Salomon’s method in a study of the effect of the absolute line orientation on the extrapolation adjustments. They found that the accuracy of the adjustments varied with the absolute orientation (horizontal and vertical lines were adjusted more accurately than were oblique ones), but the overall error was generally low, typically less than 2”. These studies suggest that observers extrapolate static lines to within 1” of angular change in line orientation.
*Department of Psychology and Center for Neural Science, New York University, 6 Washington Place, Rm 959, New York, NY 10003, U.S.A. tDepartment of Psychology, Stanford University, Stanford, CA 94305, U.S.A. 2177
2178
M. PAWS. et al.
Procedure
PROBE
Initial position
-c
1-@OtO~
length s
Visible traiectov
~
FIGURE 1. A diagram of the prediction task. The probe, shown in its initial position, is moving along the visible trajectory indicated by the solid line. The probe disappears at the end of the solid line. The distance between the target and the trajectory is designated by d.
At the beginning of each trial, a probe point appeared at a random location in the upper half of the display area. After 2OOmsec, the probe began to move along a straight trajectory with uniform velocity. As the probe began to move, a stationary target appeared in the lower half of the display. The location of the target was randomly varied over several degrees with respect to the true extrapolation of the trajectory. The moving probe was displayed for a variable period of time, and then disappeared. The observer’s task was to indicate whether the extrapolated trajectory would have passed to the left or to the right of the target. We call the visible portion of the trajectory the visible path, and the distance between the final position of the visible path and the target the gnp. Approximately 500 msec after the observer’s response, the probe appeared in a starting location for the next trial. After each response, observers were given feedback about the correctness of their responses. Apparatus
The static extrapolation task can be used as a reference to evaluate the effects of motion. To perform this extra~lation task with a moving probe, the observer must use information about current and prior locations of the probe. For some conditions, such as with very slow motion, the extrapolation task should be more diEcult because information about prior locations might be lost. For conditions under which observers are found to perform less accurately with moving stimuli than they do with static ones, the difference can be used to estimate the uncertainty (i.e. noise) due to the inability to retain and integrate information over time.* By varying the velocity and length of the visible trajectories independently, we can assess the noise due to spatial un~~ainty and to temporal integration. To quantify the contribution of these sources of uncertainty, we developed a model incorporating both spatial and temporal effects. We performed three experiments; the first measured extrapoIation performance as a function of distance, the second evaluated the effects of motion arrears, and the final assessed the performance on a similar, but static, test. EXPEIl#MlMT
1
The two stimuli in this experiment, the target and the probe, were luminous dats approx. 1 min of arc in diameter, presented on a CRT display which subtended a diameter of approximately 10”. The luminance of the stimuli exceeded thresholds of visibi~ty by two log units at the highest speeds. The stimulus position was updated at 5-msec intervals. *An equivalent interpretation can be given in terms of the direction sensitivity motion detectors.
The stimuli were presented on a CRT screen of a vector graphics processor (HP-1345) with a P31 phosphor. The display, controlled by an II?M/XT ~~~ornputer, was mounted in one channel of a modified two-channel tachistoscope enclosure, so that no extraneous contours were visible. The other-channel of the tachistoscope was used to generate a background field consisting of a uniform central disk, at approximately 50cd/mz, with edges approximating Gaussian blur. The background &mination bid the borders of the CRT screen, and masked the slow component of -the phosphor persistence. The observer’s head was partially stabilized by a chin rest and viewing goggles. The observer viewed the stimuli binocularly from a distance of 6Ocm. Observers
Three observers with normal vision participated in this experiment. Two practiced observers were authors; the third was a naive observer who was paid for her participation. Conditions
The orientation and the starting position of the trajectory were varied randomly from trial to trial to eliminate any absolute position cues. The orientation was varied over an angular range of f 10” around the vertica1 meridian, and the horizontal starting position was jittered in the range of f0.5”. The length of the visible trajectory was 46.9 min of arc. Combined with a velocity of 2346 n&/see, this resulted in a 2OOmsec period during which the probe was visible. The gap was varied randomly over a block of 300 trials, with the gap size ranging from 47 min to 6” of arc. The horizontal location of the target relative to the true extrapolated trajectory was controlled by randomly interleaved staircases. There
EXTRAPOLATION
OF LINEAR MOTION
2179
f
0
100
200
300
GAP SIZE [mln]
400
0
100
200
300
400
GAP SIZE [min]
0
200
100
300
-4 400
GAP SIZE [min]
FIGURE 2. Just noticeable differences (JNDs) expressed in minutes of visual angle for three observers, plotted against gap size: (a) observer HC; (b) observer JG; (c) observer MP. The length of the visible trajectory was 46.88 min and the probe was moving at 3.91”/sec. The duration of the visible portion of the trajectory was 2OOmsec.
were two up-down staircases for each gap condition in each block. One staircase was designed to present stimuli for which the probability of a right-of-the-target response was in the neighborhood of the 0.7 level (41/2); the other staircase for the same condition presented stimuli near the 0.3 level. The overall probability of right-of-the-target judgments for the combination of the two staircases was l/2. We used this symmetric arrangement to minimize the probability of getting stable sequences when the observers responded randomly. The performance in each condition was determined using at least 200 trials for each observer. Results
We summarized observers’ responses by empirical psychometric functions representing the proportion of right-of-the-target judgments as a function of the perpendicular distance d between the actual extrapolated trajectory and the target illustrated in Fig. 1. We computed a maximum likelihood fit of the logistic function for each empirical psychophysical function. The logistic function of the form f(x) = l/[l + E-(~+~‘)] with two parameters, a, b > 0, is empirically indistinguishable from a cumulative Gaussian distribution function. We evaluated the goodness of fit of the psychometric function using a x2 test. The data in all conditions and for all observers were well approximated by the logistic function and none of the fits could be rejected at P = 0.05. The just noticeable difference (JND) was defined as the difference between the 50 and 75% points on the best fitting logistic function, which would correspond to d’ = 0.67 in the case of the normal approximation. The points of subjective equality, i.e. d,, such that P{right-of-the-target} = 0.5, were very small (~20% of the JNDs). Moreover we could not find any systematic variation of the dso with the experimental conditions. The standard deviation of the JND estimates was less than 10%. Therefore, we concluded that the JNDs represent an adequate summary of the data. Figure 2 shows the JND for the three observers as a function of gap size. The most obvious
features of these data are that the JNDs increase with gap size, and that this increase is well approximated by a straight line. We tested the linearity of the data by computing a linear regression for each observer. The resulting regression lines are shown in Fig. 2. Although there appears to be a nonlinear component in the results of observer HC, the departure from linearity is small in comparison to the variance accounted for by the linear term. The slopes for the regression lines, as well as the proportion of explained variance expressed by the squared correlation coefficients, r*, are shown in Table 1. The linear dependence of the JND on gap size is consistent with Salomon’s (1947) results obtained in a static extrapolation task using the method of adjustment. This linear dependence suggests that the noise affecting the extrapolation task represents constant orientation uncertainty because the peformance expressed as the ratio of the JNDs to the gap size is independent of the gap size g. The ratio JND/g = tan(A4), where A4 represents the JND in the angle or the orientation of the trajectory. The estimates of the angular JNDs are included in the second row of Table 1. Although these data show similar behavior, the angular threshold values are higher than those obtained by Salomon (1947) and later by Bouma and Andriessen (1968) in static extrapolation tasks. Our hypothesis is that the decrement in performance occurred because in the present experiment, in addition to the probe motion, both orientation and position of the trajectory were varied randomly. In Experiment 2 we measured the effects of the length of the visible trajectory and the velocity of the probe. TABLE 1
Slope Angle r2
HC
Observer MP
GJ
0.0644 3.68 0.934
0.0438 2.5 0.971
0.0875 5.0 0.999
2180
M. PAVEL el al EXPERIMENT
2
The data of Salomon (1947) indicate that the precision of extrapolation adjustments improves with line length. Other researchers have demonstrated that the discrimination of the orientation of static lines in the fovea improves with the length of short lines (< 1”) and is independent of the stimulus length for longer lines (e.g. Paradiso & Camey, 1988; Andrews, Butcher & Buckley, 1972; Watt, 1987). In this experiment, we investigated the effect of the length and velocity of the visible trajectory on extrapolation judgments. The stimuli, apparatus, procedure, and observers were identical to those in Experiment 1. Conditions
The gap was kept constant at 3.1”. We used a mixed list design, in which variable path lengths and probe velocities were ordered randomly within a block of 300 trials. Two independent variables characterized the motion of the probe: (1) the visible path length, ranging from 11.5 to 70 min of arc, and (2) the probe velocity, ranging from 1 to 7.7”/sec. The duration of the visible path depended on the path length and on the velocity; it ranged from 43.3 msec to 1.7 sec. Results
Figure 3 illustrates JND as a function of the length of the visible trajectory for one observer. It shows that JND decreases with path length up to a critical value of approximately 30 min of arc. For longer visible trajectories, the performance reaches an asymptotic level. In the trajectory extrapolation task, the lack of further improvement could be due to either spatial limitations or temporal limitations on the ability to integrate information. The data in Fig. 3 suggest that the limitation is spatial, because the path length cutoff is about the same
30
for all velocities. These data are comparable to the results of static orientation judgments reported by Watt (1987). Although Watt’s task and stimuli were different from ours, the dependency of response accuracy on line length was similar to that reported here. In particular, Watt’s data reached asymptotic values for line lengths between 0.5 and 2”, depending on the exposure duration. Although the results of the trajectory extrapolation task considered thus far are qualitatively similar to those obtained with stationary stimuli, there is one important difference between the two tasks: the dynamic task involves an additional parameter-the velocity of the probe. To examine the effects of probe velocity, we plotted the observed JNDs as a function of probe velocity (Fig. 4). These JND functions indicate that extrapolation is more difficult with slower probes than with faster ones. Performance improves with increasing velocity but reaches an asymptotic level for velocities exceeding 3S”/sec. Asymptotic behavior was expected for very high velocities, because a rapidly moving point is perceived as a stationary line, but our data show that an asymptote was reached for velocities much slower than those required for the percept to become stationary. The form of these JND functions indicates the existence of certain limitations on human ability to retain and use prior positions of the probe. We shall discuss the effects of both temporal and spatial limitations when we describe the quantitative model. The results of Experiment 1 suggest that, in some respects, an observer’s performance on the dynamic extrapolation task is similar to that observed previously on a static extrapolation task. This fact is consistent with the possibility that performance on both tasks is limited by the same mechanism. We performed Experiment 3 to measure human extrapolation performance on a static task under conditions comparable to those in Experiments 1 and 2.
HC
25
20
-
-
2 E
0
58
mdsec mirdsec
0
77
0
116
minhec
+
230
midset
0
346
min/sec
A
460
min/sec
,5-
4 IO
-
5-
0 I
0
I
I
I
I
1
20
40
60
80
100
Path length lminl FIGURE 3. JNDs for one observer, as funcoions of the visible path length for six velocities. The gap was constant at 47 min of arc, and the parameter is the velocity of the probe.
2181
EXTRAPOLATION OF LINEAR MOTION 25 HC r
0
0
0
11.7
min
0
23.4
min
+
35.2
min
0
46.9
min
A
70.3
min
I
I
I
I
200
400
600
000
Velocity
Cmin/secl
FIGURE 4. JNDs for one observer as functions of probe velocity. The gap was constant at 47 min of arc, and the parameter is the length of the visible path.
EXPERIMENT
3
The procedure, apparatus, and stimuli in this experiment were identical to those in Experiment 1, except that the moving probe was replaced by a stationary line. In fact, because a stationary line is indistinguishable from a probe moving with a very high velocity, we expected that the performance on the static task would represent the upper limit of performance on the dynamic task. Observers Two observers participated in this experiment; was an author; the other was a naive volunteer.
one
Conditions In each condition, the line had the same length as the corresponding visible trajectory of the moving probe in Experiments 1 and 2, and was presented for the same
oI 0
100
200
300
400
GAP SIZE [mlnl
FIGURE 5. JNDs from Experiment 2 with stationary lines, for two observers, plotted against the gap size.
length of time that the motion was visible. The observers were asked to indicate on which side of the stationary target an extrapolation of the line would pass. Results The results of this experiment are shown in Fig. 5. As expected from prior empirical results (Salomon, 1947), these data show the same linear dependency of JNDs on gap size as did the corresponding data from Experiment 1. However, performance with a moving probe is somewhat worse than that on the static task. This decrement in peformance can probably be attributed to the effects of motion. The similarities between the results of Experiments 1 and 3 were used as a basis for a model in which both tasks are performed by the same mechanisms except for additional uncertainty due to the motion.
MODEL
In this section we describe a model of how an observer might perform the trajectory estimation and extrapolation task. The purpose of this model is to relate an observer’s performance on the extrapolation task to his internal representation of the stimulus. The model is based on the assumption that the image of the stimulus is contaminated by intrinsic perceptual spatial noise, and that the noisy signal is then processed by an ideal observer. The observed human performance, therefore, can be characterized by the properties of this intrinsic noise which may depend on time. The assumptions concerning the representation are similar to those proposed by Andrews and his colleagues (Andrews, 1967; Andrews, Butcher & Buckley, 1972) to characterize perception of orientation of contours. A similar approach was used later to characterize detection of curvature (Watt & Andrews, 1982) and detection of deviations from straight lines (Watt, Ward Br Casco, 1987). One
2182
M. PAVEL et al.
9 .I I
Actual Trajectory
1 I 1 I I i I
Estimated Trajectory
FIGURE 6. A diagram illustrating the model of the extrapolation process. The actual trajectory (solid line), the sampled noisy repmentation (dots), and the beat fitting straight line trajectory (dashed line).
distinction between the prior and the current work is that, for the extrapolation task, orientation information must be combined with position information. In addition, in our model, the observer is assumed to compute his decision only from the information available in the stimulus without the prior knowledge of the orientation of the line. The model is based on the assumption that a continuous trawory is represented by a set of discrete samples (e.g. responses of receptor units). The spatial uncertainty in the trajectory samples and in the location of the target is due to spatial Gaussian noise. We assumed that the observer knows a priori that the trajectory is a straight line, and that he determines his responses by minimizing distance errors. Thus, the sampled physical trajectory is given by Xi = a + flyi, where a and #Iare constants. The corresponding perceived trajectory is ui = xi + eXi, vi = yi + eYi, where eXi and eui are independent random variables with zero means and variances 0: and a%, respectively. Using this noisy percept of the trajectory, the observer must estimate the physical trajectory by finding the straight line that minimizes the sum of the perpendicular distances* between each point and the line, as illustrated in Fig. 6. We assumed that the observer computes the linear trajectory from the perceived sampled trajectory using a least squares minimization process such as principal component analysis. Such a process would fmd that line for which the orthoaxial (i.e. perpendicular to the line) errors are minimixed. Note that this computation can not, in general, be wormed by a linear regression. *The perpendicular distance between a line and a point is given by &(_)
= (ari + 4s + CY n2+b2 .
The variability of the observer’s estimated line would then determine how well he or she can perform on the extrapolation task. The variability of the principal components for a trajectory in a general position is difficult to compute in closed form. For the stimuli in our experiment, however, the variability of the first principal component can be well approximated by the variability of a regression line. In that case, the expected orthoaxial errors are approximately equal to the deviation along the perceived horizontal axis (a). This approximation is valid because the empirically observed errors are small (i.e. JNDs on the order of a few degrees), and the physical trajectories were nearly vertical. The observer determines the perceived trajectory tii = oi + fluj used for the extrapolation task by finding the intercept oi and the slope b using the standard linear regression formulae. The variability of the extrapolated trajectory and consequently the observers’ performance is determined by the variability of oi and B. If the origin of the coordinate system is placed at the center of the visible trajectory, then the variances of the line parameters are given by var(8) = ;,
var(fi) = n
CC
iz,("i-
(v))2
where (v ) is the average value of Vi. The model generates a response by computing the distance 2 between the perceived target location us and the estimate of the nearest point on the trajectory a = oi + /3.$ - U8
(1)
where v8 and uB are the perceived coordinates of the target. The probability of the response right-of-target, which is generated whenever d > 0, is the theoretical psychometric function Pr{right-of-target) = Pr{a > 0) = @[d/a(a)], where d is the actual distance, @ is a curnultive normal distribution and o(a) = ,/var@) is the standard deviation of d obtained from equation (1). Evaluation of observers’ performance, JND = 0.6750(a) requires estimation of the variance of d var(d) = at
,$(vi- s) the variance of d is determined mostly by the variability of the slope
1
126s ‘j2 JND z,kga,, s3 [
EXTRAPOLATION
and, consequently, the JND is approximately proportional to the gap size. The slope of the JND function is proportional to the spatial uncertainty represented by 6,. It decreases with the length of the visible trajectory, and increases with the distance between adjacent samples. Therefore, for a given level of performance, equation (3) specifies a tradeoff between the spatial density of sampling and the uncertainty associated with each sample. The contribution of these two parameters is, therefore, not distinguishable using the results of the present experiments. The linear relationship between the JND and the gap size also can be characterized by the just noticeable differences A4 in the orientation of the probe trajectory given by
OF LINEAR MOTION
2183
(a) Trajectory
Length
= 9.6 min
25.
14 4 min
20.
21 0 min 41 8 min
200
400
Gap Size
600
800
1000
1200
[mid
(b)
This model is based on the assumption that all perceived homogeneously sampled locations are available simultaneously. We evaluated this static homogenous model under those stimulus conditions for which we had human data. Figure 7 illustrates the predicted JNDs as functions of gap size for two values of the visible trajectory length. These JND functions account well for the data obtained in our experiments and for those obtained by Salomon (1947), reproduced in Fig. 8(a). The homogeneous static model, however, has several shortcomings. First, any regression line must pass through the center of the points which, in this case, is in the center of the visible trajectory. Therefore, this model predicts that performance for small gaps will be relatively poor, and will depend on the length of the trajectory, as indicated in Fig. 7. This prediction contradicts our experimental results, shown in Fig. 2, as well as those of Salomon, shown in Fig. 8(a). Another discrepancy between the data and the model is illustrated in Fig. 8(b), in which we compare the theoretical slopes to the results of Salomon (1947) shown in Fig. 8(a). The slopes of the theoretical JNDs exhibit greater dependency on the visible trajectory (or line) Trajectory
/
Length
= 30 min
r
100
200
Gap Size
[minl
300
400
FIGURE 7. Theoretical JND as a function of the gap size for two different lengths of the visible trajectory (30min and 60 min). The origin is at zero gap, and the length of the visible trajectories is shown as a negative gap on the horizontal axis.
0.04
- Mode’
Salomon
100
200
Line Length
(1947)
300
400
[min]
FIGURE 8. Results obtained by Salomon (1947) on the static extrapolation task. (a) Standard deviation of the adjustments averaged over 10 observers for line lengths 0.16”, 0.24”, 3.54”, and 6.97”. (b) Slopes of the functions in (a) as a function of the line length. The solid line represents the standard deviations generated by the model.
length than do those of human observers. For humans, performance on an extrapolation task improves with short lines or visible trajectories, but reaches an asymptotic level or improves only slightly for longer ones. This observation suggests that human observers can use only a limited portion of the trajectory in making their estimations. For the model, the improvement is not limited, and is proportional to l/~~‘~. Therefore, the static model cannot account for any velocity and duration data. These considerations motivated our introduction of spatial and temporal dependencies into the model. Our experimental results suggest that the human visual system has limited capability to integrate over space and time. To account for these limitations, we extended the static model to include both (1) spatial inhomogeneity in retinal sampling and spatial judgments, and (2) memory or temporal persistence. We modified the previously uniform spatial uncertainty cr,, to depend both on the location of each sample and on the time interval from stimulus offset. There is considerable experimental evidence demonstrating that spatial acuity decreases with increasing distance from the fovea (eccentricity) for a variety of
2184
M. PAVEL ef al.
spatial tasks (e.g. Westheimer & McKee, 1975; Levi et al., 1985). This spatial inhomogeneity can be interpreted as an increase in spatial uncertainty with increasing eccentricity. The dependency on eccentricity is typically approximated by I/( 1 + cr), where r is the eccentricity of a point defined as the distance from the fovea, and c is an experimentally determined positive constant (see, e.g. Levi et al., 1985). This formulation was used by Paradiso (1988) to model the effects of eccentricity and line length on orientation judgments. We incorporated spatial inhomogeneity in our model by modifying the variability of the perceived location of each sample point so that the variance increased with eccentricity in accordance with @z(r) = crz(1 + cr), where (T#is the standard deviation of the noise representing spatial uncertainty, and o, is the standard deviation in the fovea. An observer’s memory for the location of previously displayed information is of limited temporal extent. This memory might, in part, rely on visible persistence, which lasts between a few tens to few hundreds of msees (Farrell et al., 1990), depending on stimulus characteristics. A higher-level memory also may be involved in the extrapolation judgments, but the data suggest that much of the precise information is lost after a few hundred msec. We incorporated this temporal limitation into the model by assuming that the uncertainty of spatiaf information increases exponentially from the time of stimulus offset to according to exp( - t - t,,/T), with a fixed time constant z. Assuming that the spatial and temporal effects are mutually inde~ndent, the variance of the spatial noise is aE(r,t) = cri(
1 + cr)e”“.
(5)
We obtained a complete model by substituting this expression for the standard deviation of spatial noise in equation (2). This ~p~~n~tion is similar to that of the static model, except for the dependency on time and location. The corresponding regression or principalcomponent analysis must be replaced by the corresponding weighted version, where the weights are the reciprocal of the variance. The dependence of the weights on time and space is illustrated in Fig. 9. We compared the predictions made by our complete model to the observed results in two steps. First, we considered the static version without spatial inhomogeneity to account for the effects of gap size. Subsequently, we examined the effects of temporal Iimitation and spatiaf i~omogeneity. Because of spatial inhomogeneity, the performance of the model might depend on the direction of gaze, i.e. by the point of fixation at the time of the judgment. Additional analyses and simulation of the model revealed that the predicted performance on the trajectory extrapolation task does not depend significantly on the point of fixation. The independence obtains because there is a tradeoff between the accuracy of the perception of the trajectory and the perception of the target. If the fixation point is assumed to be located at the target, the
FIGURE
9. Spatial and temporal representation weighting function.
of the temporal
location of the target would be perceived with a high accuracy but the trajectory would have more variability. When the fixation point is on the trajectory, then the location of the target becomes uncertain. Because of the lack of expected effect of eye fixation, we assumed that the observers made their judgments while looking at the last point of the trajectory. This assumption is partially supported by the high accuracy of observers’ judgments of small gaps. Using this assumption, we compared the complete model’s predictions to the observed data; one such comparison is shown in Fig. 10. The dashed line shows the original model’s prediction; the solid line shows the modified model’s predictions. The modified model provides a good appro~mation to the data of observer HC, which are plotted as diamonds. To verify the predictions of the model that human extrapolation performance should be relatively iadependent of the fixation point, we carried out a set of informal experiments in which the observers performed the trajectory extra~lation task and were instructed to use one of three different fixation points: (1) the end of trajectory, (2) the midpoint and, (3) the target. Our results were consistent with our model’s predictions; the .IND did not vary consistently with the fixation instructions. The reason for not reporting the detailed results is that we were not able to monitor the observers’ eye movements, so as to confirm their compliance with the instructions. CONCLUSIONS
We studied the accuracy with which the visual system can extract simultaneously information about probe location and direction of motion, as well as the ability to combine position and velocity information over space and time. The extrapolation task that we developed required the visual system to combine these two components with a clear objective: to minimize errors between the predicted and the actual extrapolated trajectory. Although trajectory extrapolation is a simple task, it shares aspects with many other real-life tasks (e.g. perception of depth from
2185
EXTRAPOLATION OF LINEAR MOTION 30
-
25
-
20
-
0
HC 0
.c bs
min/sec
115
\ 15
\
2 )\ 10 ‘Y,\* 5
O \
-
-*-.
01 0
-*-
---
I
I
I
20
40
60
Path
--80
100
length Cminl
FIGURE 10. Comparison of the two versions of the regression model. The dashed line represents predictions of the simple regression model, and the solid line represents the linear regression approximation modified by temporal and spatial weighting. Diamond’s denote data from observer HC.
motion, manual control, etc.) that require encoding of location of a trajectory as well as of the direction of motion. Our data indicate that observers are able to perform an extrapolation task with high accuracy. The JNDs obtained on both the dynamic and the static extrapolation tasks increased linearly with the gap size in a similar manner to that obtained with static stimuli, and we developed a model for integration of motion and location information under the assumption that the only source of uncertainty arises in the coding of the retinal location. In this model, the direction of motion was derived from the location of static samples, and the integration was performed over space. Of course, equivalent information could have been derived from a model based on motion detectors. We developed the model to account for several aspects of the observed data: Perception of straight line: We assumed that the visual system can extract the parameters of a straight line by regression or principal-components analyses. Extrapolation: The visual system is capable of making efficient use of the estimated straight line. Spatial extent limitation: The efficient use of information is limited to a relatively small spatial extent. This limitation was modeled in terms of spatial inhomogeneity of the visual system, such that spatial uncertainty increases with eccentricity. Temporal extent limitation: Limited memory for motion and location information was accounted for by visible persistence. While the ability to integrate over time may be due to more complex
memory mechanisms, a simple exponential increase in uncertainty with time was sufficient to account for our empirical data. We have characterized the limits of performance in terms of postulated intrinsic noise, making no assumptions concerning the biological origins of this noise. However, since we were able to describe the behavior in terms of least-squares analyses (i.e. principal component or regression), there are many biologically plausible process models that would result in the same performance. For example, there are several recursive filtering schemes (Pavel, 1990) that approximate the optimal least-squares analysis. To discriminate among the various possibilities we would have to develop experiments that would require judgments of trajectories more complex than are those of straight lines.
REFERENCES Andrews, D. P. (1967). Perception of contour integration in central fovea. Part II: Spatial integration. Vision Research, 7, 999-1013. Andrews, D. P., Butcher, A. K. & Buckley, B. R. (1972). Acuities for spatial arrangement in line figures: Human and ideal observers compared. Vision Research, 13, 599-620. Bouma, H. & Andriessen, J. J. (1968). Perceived orientation of isolated line segments. Vision Research, 8, 493-507. Levi, D. M., Klein, S. A. & Aitsebaomo, P. K. (1985). Vernier acuity, crowding and cortical magnification. Vision Research, 25, 963-977. Paradiso, M. A. & Camey, T. (1988). Orientation discrimination as a function of stimulus eccentricity and size: Nasal/temporal retinal asymmetry. Vision Resewch, 28, 867-875. Pavel, M. (1990). Predictive control of eye movement. In Kowler, E. (Ed.), Eye movements and their role in visual and cognitive processes (pp. 71-114). Amsterdam: Elsevier. Salomon, A. D. (1947). Visual field factors in the perception of direction. American Journal of Psychology, 60, 6$-88. Ulhnan, S. U. (1979). The interpretation of uisuul mofion. Cambridge, Mass.: MIT Press.
2186
M. PAVEL er 01.
Watt, R. J. (1987). Scanning from coarse to fine spatial scales in the human visual system after the onset of a stimulus. Journal C$ the
of retinal image-motion. Journul of the Optical Sociery oJ ,4mericu, 65, 847-850.
Optical Society of America, 4, 200&2021.
Watt, R. J. & Andrews, D. P. (1982). Contour curvature analysis: Hyperacuities in the discrimination of detailed shape. Vision Research, 22, 449-460.
Watt, R. J., Ward, R. M. & Casco, C. (1987). The detection of deviation from straight lines. Vision Research, 27, 1659-1678. Westheimer, G. & McKee, S. P. (1975). Visual acuity in the presence
Acknowledgements-This
work was supported by grants to Stanford University from the Air Force OfTice for Scientific Research, AFOSR-84-03-08, and NASA, NCC 2-269. We thank Dr E. Kowler and L. Dupre for useful discussions and their comments on this manuscript.
