A new technique for generating disordered point-light animations for the study of biological motion perception.

Journal of Vision (2015) 15(11):13, 1–18

1

A new technique for generating disordered point-light animations for the study of biological motion perception Department of Psychology, Duksung Women’s University, Seoul, Korea

# $

Eunice L. Jung

Department of Brain and Cognitive Sciences, Seoul National University, Seoul, Korea

$

Sang-Hun Lee

Department of Brain and Cognitive Sciences, Seoul National University, Seoul, Korea

# $

Department of Brain and Cognitive Sciences, Seoul National University, Seoul, Korea Vanderbilt Vision Research Center and Department of Psychology, Vanderbilt University, Nashville, TN, USA

# $

Jejoong Kim

Randolph Blake

Studies of biological motion perception often use stimuli depicting human actions portrayed via point-light (PL) displays. Typically, counterpart, or control, stimuli for PL biological motion are created by spatially scrambling motion trajectories of individual PL dots. Depending on the purpose of the study, however, this procedure may be inappropriate as a foil for genuine PL animations, because spatial scrambling not only disrupts coherent motion activity but also eliminates pair-wise motion relationships among dots and, unless corrected, alters the spatial spread of PL dot motions. We introduce a new technique for producing perturbed PL animations, called pair-wise shuffled motion, that preserves the elementary features of biological motion in spatial and motion energy domains and only disrupts the specific sense of global, coherent perception of biological motion. First we describe the procedure for creating pair-wise shuffled motion sequences. Next we compare unperturbed PL animations to pair-wise shuffled motion, to spatially scrambled motion, and to spatially constrained scrambled motion in terms of spatial distributions of the dots, spatiotemporal amplitude spectra derived from Fourier analysis of those sequences, and space-time motion energy associated with those perturbed animations. We then show that the results from those analyses generalize to a large family of PL animations, including the widely used PL walker. Finally we present results from a two-interval forced-choice biological-motion discrimination experiment comparing the robustness of scrambled and pair-wise shuffled motions as foil stimuli. Results from

these comparisons suggest that pair-wise shuffled motion offers advantages as a foil stimulus compared to foils using the conventional scrambling technique. Pairwise shuffled motion provides an additional, effective control display for evaluating PL biological motion perception in future psychophysical, computational, and imaging studies that focus on mechanisms of processing spatiotemporal information signifying biological motion within PL displays.

Introduction Forty years ago the study of perception of biological kinematics was revolutionized by Johansson’s introduction of the point-light (PL) animation technique for portraying biological motion (Johansson, 1973). With this technique, small lights were attached strategically to the limbs and torso of a human being who was then filmed while engaging in different activities.1 The outcome was striking: animations comprising just the moving dots of light preserve sufficient information that an observer viewing those animations can readily recognize the action being portrayed and, in some instances, the gender, mood, and even identity of the actor (see review by Blake & Shiffrar, 2007). The significance of Johansson’s invention of the PL technique is evidenced by the more than 3,000 references to his original article, not to mention the many references

Citation: Kim, J., Jung, E. L., Lee, S.-H., & Blake, R. (2015). A new technique for generating disordered point-light animations for the study of biological motion perception. Journal of Vision, 15(11):13, 1–18, doi:10.1167/15.11.13. doi: 10 .116 7 /1 5. 11 .1 3

Received October 24, 2014; published August 25, 2015

ISSN 1534-7362 Ó 2015 ARVO


Kim, Jung, Lee, & Blake

to other articles that have developed refined versions of his technique (e.g., Cutting, 1978). Not surprisingly, PL animations have been enthusiastically adopted by vision scientists interested in identifying the neural bases of perception of biological motion based on neuropsychological evidence from people with brain damage (Pavlova, Lutzenberger, Sokolov, Birbaumer, & Krägeloh-Mann, 2007; Saygin, 2007; Schenk & Zihl, 1997; Vaina, Lemay, Bienfang, Choi, & Nakayama, 1990) and neuroimaging results from typical individuals (Bonda, Petrides, Ostry, & Evans, 1996; Grossman, 2005; Grossman et al., 2000; Howard, Brammer, Wright, Woodruff, Bullmore, & Zeki, 1996; Puce & Perret, 2003). The roster of studies using PL animations in psychophysical and brain-imaging studies continues to grow briskly (Pavlova, 2012). In both psychophysical and neuroimaging studies using PL animations, it is valuable to include PL animations in which kinematic information is perturbed by disrupting the spatiotemporal coherence of the dot motions. Perturbation can be as simple as turning the sequence upside down (Pavlova & Sokolov, 2003; Sumi, 1984), playing it backward (Klin, Lin, Gorrindo, Ramsay, & Jones, 2009), or randomly shuffling the locations of the dot motions (e.g., Grossman et al., 2000). Perturbed PL sequences have been used as foil targets in psychophysical studies using categorization (e.g., Blake, Turner, Smoski, Pozdol, & Stone, 2003) and forced-choice testing (Kim et al., 2008; Kim, Park, & Blake, 2011), and the inclusion of perturbed sequences has been used for identifying regions of interest in fMRI studies (e.g., Saygin, Wilson, Hagler, Bates, & Sereno, 2004) and as comparison stimuli in event-related potential studies (Krakowski et al., 2011). Dot motions derived from perturbed sequences have also been used to create noise masks that degrade perception of PL animations (e.g., Grossman, Blake, & Kim, 2004). In this article we introduce a novel technique for perturbing PL animation sequences while preserving pair-wise hierarchical motions contained in the original PL sequences. Rather than the often-used term scrambled motion, we refer to these as pair-wise shuffled motion (PSM) sequences, in recognition of the selective destruction of relations among local segments but preservation of pair-wise relations within those segments. From the outset we wish to stress that PSM sequences are readily distinguishable from conventionally scrambled sequences, for PSMs—unlike scrambled sequences—bear some resemblance to their unperturbed counterparts that portray normal biological motion (in this article, we use the abbreviation BM when referring to unperturbed PL sequences). It should also be kept in mind that none of the scrambling techniques—i.e., neither the scrambled varieties nor those produced using PSM—is effective when applied

2

to noncyclic motions (e.g., throwing) that are portrayed for longer than one action cycle; for extended-duration sequences, only cyclic activities (e.g., walking) should be used. The next three sections of this article are organized in the following way. First, we describe two variants of the conventional method for producing perturbed PL animations, both based on spatial scrambling of the dots comprising BM animations. That sets the stage for introducing the new PSM procedure for producing perturbed sequences that more closely match normal sequences in terms of pair-wise dot motions. Next, after describing how PSM sequences are created, we create 80 separate versions of PSM for each of four BM sequences sampled from a 25-item library of PL animations portraying human activities (Ahlström, Blake, & Ahlström, 1997). As well, we create 80 perturbed versions of those four PL sequences using two different spatial-scrambling algorithms. Then we present results comparing different indices of motion energy contained in the four BM sequences to those same indices derived from each of the three sets of perturbed animations. Finally, we present results from a psychophysical experiment to validate the utility of PL sequences produced using the PSM procedure.

Creation of PSM Previously used perturbation techniques In previous work, several different but related perturbation techniques have been employed, all aimed at destroying the perception of recognizable biological motion. The simplest strategy is to disrupt the spatial relations of dots within a given unperturbed BM animation by spatially randomizing the initial starting locations of the dots comprising that animation, with the only constraint being that the starting positions of the repositioned dots fall within the same virtual area as the starting positions defined by the normal BM counterpart. This procedure, which we designate by the abbreviation SM, has been widely used in research on biological-motion perception (Chandrasekaran, Turner, Bulthoff, & Thornton, 2010; Chang & Troje, 2008; Garcia & Grossman, 2008; Grossman, 2005; Ikeda, Blake, & Watanabe, 2005; Saygin et al., 2004). The result is an animation of SM that contains little hint of its BM origins. Movie 1BM shows an unperturbed action (underhand ball toss) portrayed by a BM sequence, and Movie 1SM shows the version produced by randomizing the starting positions of the dots—i.e., SM. It is very easy to distinguish which one is more nearly biological in appearance. Figure 1A and B shows the actual trajectories of individual dots



Movie 1. (BM) PL animation of an actor executing an underhand toss of a ball (left sequence). (SM) Spatially scrambled version of the underhand-toss sequence with each dot’s starting position randomly reassigned within the frame of the animation, such that individual dots undergo the same motions as the ones shown in the unscrambled version (middle sequence). (SMsc) Spatially scrambled version of the underhand-toss sequence with each dot’s starting position randomly reassigned within the frame of the animation, adjusted to ensure that no dot moves outside of the area in which BM dots move (right sequence)

comprising the unperturbed and randomized-position animations, respectively (successive dot positions over time are shown in different colors, blue being the dot positions in Frame 1 of the animation and red being the terminal positions in Frame 20). Here it can be seen that the trajectories of individual dots are maintained over time but the starting locations are disordered in the scrambled version. In other words, this scrambling technique leaves the motion paths of individual dots unaltered, meaning that a scrambled sequence contains the same individual motion vectors as the unperturbed PL display from which it was made—what is destroyed are the coherent spatial relations among dots and, therefore, the appearance of hierarchical, pendular motions characteristic of unperturbed PL animations. But the SM procedure described does have a drawback: While the dot starting positions fall within the same area as the starting positions for the related BM sequence, some portions of a scrambled dot’s trajectory may fall outside the virtual region defined by the boundary of the dot movements comprising the corresponding BM sequence from which the scrambled version was generated. As a result, the SM dot motions can be spread over a larger area than are the dots comprising the BM sequence. To preclude the possibility of this difference between BM and SM, one can create spatially scrambled PL sequences in which the motion paths of the randomly repositioned dots are prohibited to move beyond the boundaries defined by a given BM sequence (Gilaie-Dotan, Saygin, Lorenzi, Rees, & Behrmann, 2015; Jastorff & Orban, 2009); we refer to such animations as spatially constrained SM, abbreviated SMsc. Movie 1SMsc illustrates an SMsc version of the underhand toss shown in Movie 1BM, and Figure 1C shows the motion trajectory plot for this

3

Figure 1. Motion trajectory plots of PL animations, where dot colors change from cool (blue) to hot (red) over the duration of the animation. Animations showing these sequences are shown in Movie 1. A. Dot positions over time of an unperturbed PL animation of an actor executing an underhand ball toss (BM). B. Scrambled version of BM, where each dot’s starting position is randomly repositioned within the boundary defined by the most extreme dot positions comprising the BM sequence (SM); once relocated to those positions, the SM dots undergo the same motions as they would in BM. C. Scrambled version of BM where the starting positions of the dots are randomly shuffled, with the constraint that no dot will have motion trajectories spreading beyond the boundary of dot motions contained in the BM sequence (SMSC). Several schemes are possible for achieving this constraint; we implemented it by repositioning any dot that strayed into the forbidden zone so that it was shifted parallel to its trajectory by an amount sufficient to bring it back into the neutral zone.

SMsc animation. Later in this article we will show results that systematically compare SM and SMsc. It is also possible to produce graded degrees of spatial scrambling of individual dots, using a statistical procedure that relocates each dot some percentage of the distance between its ordinary, unperturbed location and a new, spatially randomized location. This technique is useful for measuring acuity for discriminating small perturbations in normal PL sequences, but it is not designed to destroy perception of biological motion. Readers interested in this technique and the kinds of results obtained using it are referred to Kim et al. (2011). Another technique for degrading BM sequences is to shuffle the starting locations of all dots in the initial frame of the animation, meaning that each dot will begin its unique motion at a start location within the dot array normally occupied by another dot. With this variant of spatial scrambling, the overall motion excursion of a given dot originating at its new body location may carry that dot outside the boundary of the BM sequence from which it was generated; that can, of course, be corrected as was done for SMsc. What cannot be fixed, however, is that the initial frame of a shuffled sequence is perforce identical to the first frame of the unperturbed sequence from which it was derived.



This could pose a problem depending on the task being performed by an observer. For instance, if the task were to entail identifying the direction (left vs. right) in which a BM figure is walking, the first frame of the shuffled sequence could give away the answer. This unwanted cue does not arise, of course, with SM and SMsc sequences. Scrambling of BM dots can also be achieved in the time domain by disrupting the temporal phase relations of the dots without disturbing their spatial locations (Annaz, Campbell, Coleman, Milne, & Swettenham, 2012; Jackson & Blake, 2010; Krakowski et al., 2011). In its simplest form, this technique entails randomly selecting each dot’s starting position from all possible positions within its motion trajectory (as defined by that dot’s changing locations within successive animation frames). Thus with temporal scrambling, each dot undergoes the same movement within the same locations as the dots in the BM counterpart, only now the hierarchical, pendular motions are perturbed because the individual dot movements are being sampled from different time points of the unfolding activity. But a drawback to temporal shuffling is that it inevitably introduces unnatural, abrupt jumps in a dot’s position for actions where the depicted motion is acyclic, such as throwing; for this reason, temporal shuffling is only effective for cyclic actions. In the remainder of this article we will focus primarily on SM and SMsc and the comparison of those scrambling techniques to PSM.

Rationale for development of a more refined perturbation technique In normal PL sequences, limbs and torso are defined by coherent spatial relations of the positions of individual dots over time: Movements of the arms and legs, for example, are orchestrated by hierarchical motions specifiable by the phase relations among neighboring dots (Johansson, 1973). The perturbation techniques already described involve repositioning individual dots within the display area, thereby destroying the spatial relations among dots normally constrained to move in a linked fashion: the synchronized dot movements defining arm and leg trajectories are replaced by individual dot motions that specify nothing about the hierarchical spatial relations of dot motions in the unperturbed counterpart. As we will show in a later section of this article, however, these perturbation maneuvers also introduce additional motion information not contained in unperturbed, BM versions of those PL animations. At the same time, spatial scrambling, whether SM or SMsc, may be too extreme in terms of the information removed by scrambling. Depending on an investigator’s aim in a

4

study, therefore, these consequences of scrambling could be problematic. Consider, for example, the often used two-interval forced-choice (2IFC) task in which PL sequences and their spatially scrambled counterparts appear embedded in noise; the observer’s task is to indicate which one of two intervals contains the PL sequence. One could successfully perform this task just by identifying which interval gives an impression of pendular motions among neighboring dots, without regard to the overall, global organization of those pendular motions that specify the action. Or consider a brain-imaging study that employs a contrast between activations produced by normal BM sequences and activations produced by scrambled sequences. This contrast would not necessarily reveal only those cortical sites responsive to biological motion per se but also voxels sensitive to hierarchical, pair-wise motions that, on their own, may be insufficient to signify genuine biological motion. To put it in other words, these scrambling techniques, depending on the question being asked, could strip too much motion information from an unperturbed BM sequence, leaving them potentially discriminable from BM animations based on motion information other than the visual impression of natural biological motion or based on artifactual motion signals unwittingly introduced by the scrambling procedure.

Description of PSM technique To minimize these potential confounding consequences that can arise when individual dots’ positions are scrambled or shuffled, we have developed a modified form of perturbation that preserves pair-wise dot motions defining the local pendular movements associated with individual limbs. This perturbation technique accomplishes two things: It minimizes differences in overall motion content between a perturbed sequence and its BM counterpart and at the same time selectively disrupts the global spatial relations among those pairwise motions.2 In the following paragraphs we detail the steps involved in implementing an algorithm that most closely satisfied our criteria for successful perturbation: (a) preservation of pair-wise hierarchical motions, (b) reduced motion artifacts, and (c) no impression of natural biological motion. Figure 2 illustrates the steps comprising the algorithm that generates PSM sequences. As an example, we have selected the first frame from a PL animation depicting one half cycle of movement of an actor performing a jumping-jack movement (sometimes called a star jump or a side-straddle hop)—because this action is symmetrical and the actor is front facing, it is easy to visualize how the algorithm works. The key difference in the procedure used to produce PSM



5

Figure 2. Schematic of steps involved in implementing the PSM algorithm. Details are given in the text, and example animations are provided in Movies 2 and 3.

sequences pertains to the repositioning of dots within the first frame of a BM animation. Specifically, pairs of dots—not individual dots—are repositioned within the initial frame; in subsequent frames, all dot motions correspond to the trajectories they would have executed in the original sequence, only now they are portrayed using pairs of linked dots that have been relocated within the array. The following four steps explain how that new, initial animation frame is generated.3

Step 1. Each dot defining the PL figure to be perturbed is assigned an identification (ID) number; for convenience, these ID assignments are made such that neighboring dots of each arm and each leg have successive ID numbers, although this is not necessary. Figure 2A shows an example of these ID assignments for the first frame of the jumping-jack sequence, with the actor’s arms at the sides of the torso and the feet relatively close together. Step 2. Next, each dot is repositioned to the location corresponding to the centroid of that dot’s complete trajectory over the n-frame animation sequence. In Figure 2B, the colored dots show the new, adjusted locations of the jumping-jack dots in Frame 1 of that animation. Also shown in Figure 2B, as gray dots, are the original positions of the dots in Frame 1 of the animation (Figure 2A), and it is useful to keep in mind that those dots still have their respective ID numbers. Step 3. Dots in this centroid-defined trajectory array are grouped into pairs based on their structural relations within the body/limb hierarchy. Specifically, pairs are formed between head and hip (Dots 1 and 8, outlined white in Figure 2B), right and left shoulder (2 and 5, orange pair), left elbow and wrist (6 and 7, purple pair), right elbow and wrist (3 and 4, blue pair), left knee and ankle (11 and 12, green pair), and right knee and ankle (9 and 10, red pair). For

emphasis, those pairings are indicated by the dotted ellipses in Figure 2B. (The use of color in this figure is for illustrative purposes only—in all resulting animations, all dots were monochrome.) Step 4. As shown schematically in Figure 2C, the dot pairs defined in Step 3 are now spatially interchanged (shuffled), with specific constraints imposed: (a) The two upper body pairs (hip/head and left shoulder/right shoulder) are not swapped with each other (i.e., each upper body part is relocated to one of the limb-pair locations); (b) pairs of left-arm dots and right-arm dots are not swapped with each other (i.e., left-arm dots can be moved into any body parts except right arm); (c) pairs of left-leg dots and right-leg dots are not swapped with each other (i.e., left leg can be moved to hip/head or shoulders or left/right arm locations, but not to right leg); and (d) when two pairs of dots swap locations, the dot with the higher ID number within one pair takes the location of the dot with the lower ID number in the other pair. For example, when the left arm and leg are swapped (as in the example shown in Figure 2), the knee (ID 11) and wrist (ID 7) dots are swapped, as are the ankle (ID 12) and elbow (ID 6) dots. The maximum number of unique arrays that can be produced from these swapping rules, while finite, is quite large (n ¼ 80), and thus in an experiment it is possible to utilize many different samples of PSM without repetition. For illustrative purposes, Figure 2 shows one of those 80 unique versions of the PSM algorithm, one where the head/torso dots (white) swap positions with right-arm dots (blue), shoulder dots (orange) swap positions with right-leg dots, and leftarm dots swap positions with left-leg dots.

A key maneuver involved in Step 4 is the following: Each swap of a dot within the mean trajectory array is accompanied by an equivalent repositioning of the corresponding dot from the original, Frame 1 array (gray



Movie 2. Animation illustrating the successive steps in creating the first frame of one version of PSM of the jumping-jack sequence.

dots in Figure 2B, C). In other words, each dot within the original Frame 1 remains yoked to its corresponding dot within the mean trajectory array. To illustrate this feature of the algorithm, look at the green dot designating the left ankle in Figure 2B (i.e., the dot designated by ID 12). The gray dot just beneath and to the left of Dot 12 is the left-ankle dot in its original position within Frame 1. In this particular shuffle sequence, green Dot 12 gets repositioned to the left elbow, as can be seen in Figure 2C—in consequence, the yoked left-ankle dot from Frame 1 is now repositioned just below its counterpart green dot. Purple Dot 6 that was positioned at the left elbow in the mean trajectory array is repositioned to the left-ankle location that was occupied by green Dot 12 before the swap, and purple Dot 6’s counterpart gray dot moves comparably to its new position. Once all dots have been repositioned, the dots from the mean trajectory array are removed, leaving only the dots from the Frame 1 array, now repositioned so as to preserve pair-wise motions but to remove the impression of a body figure that was achieved by using the mean position array for pair-wise shuffling. This now constitutes Frame 1 of the PSM animation of the jumping-jack action, and in all subsequent frames of this PSM sequence, dots undergo exactly the same motions that they execute in the original BM version, only now originating from these shuffled locations. Movie 2 illustrates the sequence of steps comprising the algorithm just described, using the jumping-jack action shown in Figure 2. Movie 3 shows three different exemplars of the PSM sequence applied to the jumpingjack action, with each accompanied by an animation tracing the motion trajectory of the dots comprising that PSM version. (Just to reiterate, this algorithm for shuffling dot pairs can produce 80 distinct versions of PSM for a given BM action.)

6

Movie 3. Three examples of PSM generated from the jumpingjack sequence; each was produced by applying the algorithm described in the text and illustrated in Movie 2. Also shown in each animation is the unfolding trajectory plot associated with that particular PSM sequence (with time represented by the transition from blue dots to red dots). These three examples were randomly sampled from the 80 possible variants that can be produced using the PSM algorithm.

To see how the PSM procedure affects the appearance of the underhand-toss animation (Figure 1), compare Movie 4 (created using the PSM algorithm) to Movie 1SM and SMsc. These comparisons reveal the relational information that is preserved by the PSM algorithm but destroyed by the two SM procedures. This is because the PSM algorithm ensures that the hierarchical motion structure defined by the pair-wise dot trajectories is preserved when the sequence is animated, albeit at locations within the global configuration that are inappropriate for a given limb. There are, of course, different swapping configurations that could be envisioned, but we settled on this strategy based on a previous analysis of features critical for perception of biological motion (Thurman, Giese, & Grossman, 2010) and on rating judgments obtained from volunteers in our laboratory who viewed PSM sequences employing different pair-wise dot-swapping configurations. In the next section, we use quantitative techniques to characterize the spatiotemporal information content in

Movie 4. Example of a PSM version of the underhand-toss animation. Compare this movie to Movie 1SM and SMsc.



BM, PSM, SM, and SMsc sequences, the aim being to learn how those various perturbation procedures differ from one another and how each compares with BM.

Motion information comprising PL, SM, and PSM sequences We have examined BM, PSM, SM, and SMsc animations from three complementary perspectives: (a) variation in the 2-D spatial positions of dots over successive frames, (b) 2-D and 3-D Fourier amplitude spectra of the animations, and (c) motion energy within the animations computed using a standard motionenergy model (Lee & Lee, 2012). As we step through the results from these analyses, we will use a single BM action sequence and the derived PSM, SM, and SMsc versions to explain the analytic technique and illustrate the results. Then we will show results from application of that technique to four different PL sequences that have been used in earlier published work from our laboratory. Those four were selected because the variety of activities they portray—underhand toss, climbing, kicking, and throwing—provide a diverse sample of the kinematics found in PL animations. Moreover, in an earlier study, those four were comparable in terms of their detectability when tested using a noise-masking procedure and a larger set of PL animations (Jung, Zadbood, Lee, Tomarken, & Blake, 2013), suggesting to us that it would be reasonable to pool results from those four animations when drawing conclusions about their spatiotemporal information content.

Spatial-domain analysis We began by performing a frame-by-frame analysis of the static 2-D images comprising BM, SM, SMsc, and PSM animations (20 frames/animation) to learn in what respects they differ. Figure 3 provides examples of the dot motions comprising those four types animations for the underhand-toss sequence. The four scatter plots in Figure 3A show the locations, in x/y coordinates, of the individual dots over time for the BM sequence and for the perturbed versions of that sequence created using SM, SMsc, and PSM. Each plot shows results for 80 independent samples created for each animation. For the BM sequence, dots move in exactly the same way in each sample, which is why the distribution plot is unvarying over repetitions. But for SM, SMsc, and PSM sequences, variability will exist over the 80 different samples, thereby creating the cloud of dots. The dotted lines in each plot denote the boundaries of motion within the BM sequence, allowing us to visualize the extent to which dot

7

movements in the perturbed sequences tend to conform to those boundaries. The graph in Figure 3B plots the height against the width of the virtual rectangle defining the outer boundaries formed by dots from each of 80 samples created by the PSM, SM, and SMsc sequences of the underhand-toss BM animation. The single gray dot located at the intersection of the vertical and horizontal lines defines the area of the virtual rectangle surrounding that BM sequence, and its dimensions are the same from sample to sample, of course. The cloud of red dots shows the height/width values for 80 unique samples created using the PSM algorithm; the clouds of dark-blue and light dots show those values for 80 independent samples using the SM and the SMsc procedures, respectively. The data in Figure 3A and B confirm that dot motions associated with BM, SMsc, and PSM exhibit less spatial spread than do the movements of the SM dots, and this disparity in spatial spread is seen in both the horizontal and the vertical dimensions. This comes as no surprise, because with the SM procedure a dot’s starting position and subsequent velocity (speed and direction) are unrelated. As a consequence, the arc of motion created by a given dot (e.g., the right-wrist dot) can get repositioned to a location that causes it to move higher within the frame than would any dot in the unperturbed version of this animation. These kinds of unconstrained motions guarantee that dots in some SM sequences will stray outside the boundaries of motion associated with BM and SMsc sequences. The constraints implemented in positioning dot pairs for PSM and SMsc sequences minimize the likelihood of producing motions beyond the area encompassed by dot motions associated with the BM sequence. Next, we calculated for this PL sequence the pair-wise interdot distances for all possible pairs of dots in each of the 20 animation frames; those results appear in the four histograms shown in Figure 3C for each of the four animation types. The blue line repeated in each plot is the interdot distance distribution associated with the BM sequence, and the shaded gray area is the actual distribution for each animation type. We see that the distributions for SM and SMsc only coarsely resemble that for BM, whereas the PSM distribution more closely matches the BM distribution. We quantified the goodness of fit between BM distribution and each of the other three distributions using the Hellinger distance index, which expresses the similarity between two probability distributions (Kailath, 1967). The three curves in Figure 3D summarize the results from those analyses, with each curve showing the distribution-ofsimilarity index compiled from the 80 samples associated with PSM (red curve), SM (dark blue curve), or SMsc (light blue curve). The Hellinger test confirms the visual impression given by inspection of the distributions in Figure 3C: PSM more closely matches BM than do



8

Figure 3. Frame-by-frame analysis of changes in dot positions over time, for the PL activity depicting an underhand ball toss. (A) Distributions of dot positions over frames. Vertical and horizontal positions of individual dots are plotted for each of the 20 frames of the animation. For BM, those positions are the same upon repeated animations, but for the other three categories of animations we created 80 versions of each, and the dot clouds summarize dot positions for each of the 20 frames for each of those 80 repetitions. The dashed black lines repeated in each of the four plots indicate the horizontal and vertical boundaries of dot locations within the BM sequence. (B) Scatter plots showing the width and height of the virtual rectangle within which dots from a given PL animation fall during a 20-frame sequence of the underhand-toss animation. The single gray dot at the intersection of the dotted horizontal and vertical lines specifies the dimensions of the area within which dots fall during the BM sequence—i.e., the coordinates of the dotted lines shown in (A). The red, dark-blue, and light-blue circles correspond to the area samples for the PSM, SM, and SMsc sequences, respectively. (C) Histograms of interdot distances. The interdot distance was computed for each of all the possible pairs of dots within each of the 20 motion frames of the four PL types of the underhand-toss animation. The blue curve repeated in all four panels corresponds to the interdistance distribution for the BM sequence, and the shaded gray areas within each plot correspond to the actual distributions for BM, SM, SMsc, and PSM. (D) Plot of similarity (quantified as the Hellinger distance) between the distribution of interdot distances for BM and for those distributions associated with PSM (red curve), SM (dark-blue curve), and SMsc (light-blue curve). Smaller Hellinger distances indicate better correspondence between BM and the other distribution.

either SM or SMsc. This analysis suggests that the two spatial-scrambling techniques blur the unique relational properties comprising BM sequences that are reflected in the multiple peaks seen in the BM distribution. The PSM technique does a much better job of preserving those relational properties, which is not surprising given that the technique is based on pair-wise scrambling. In summary, these analyses of the dot locations over animation frames show that PSM, compared to SM and SMsc, more closely resembles BM in terms of spatial spread of dot locations and in terms of interdot distances over frames. These features of the dot-distance analyses are not peculiar to this animation, for they are seen in the results obtained from analyses of the other three exemplars as well. How do these findings translate into the motion domain? The following two sections attempt to get at that question more directly.

Fourier-domain analysis For the Fourier-domain analysis, we used the FFT routine in MATLAB (MathWorks, Natick, MA) to

compute amplitude spectra separately for individual 2D static frames in a given motion sample, and we then averaged those individual spectra across frames. We computed these average spectra for the four BM sequences along with spectra from 80 repeated, independent samples of SM, SMsc, and PSM sequences derived from each of those four animations. Figure 4A shows an example of the amplitude spectrum obtained by averaging the spectra from the 20 frames comprising the underhand-ball-toss sequence (BM) and the average amplitude spectra from the all frames produced by 80 independent repeated samples of SM, SMsc, and PSM for this same sequence. Following convention, the color temperature represents the amplitude of spectral energy over spatial frequency (moving radially from the origin) and orientation (designated by angle). Visual inspection of these spectra readily shows that the SM and SMsc sequences contain overall more spectral energy than do the PSM sequences or the BM sequence. In addition, the variance spectra in Figure 4B reveal that the acrosssamples variance in spectral amplitude among SM and



9

Figure 4. Summary results from 2-D Fourier analysis of individual 2-D frame images comprising BM, SM, SMsc, and PSM sequences derived from the underhand-toss action. (A) Average amplitude spectrum computed over the 20 frames of the BM sequence and average amplitude spectra based on 80 independent repetitions of the SM, SMsc, and PSM versions of the underhand-toss sequence, averaged over individual frames. (B) Across-samples variances of spectral energy for the 80 repetitions 3 20 frames 3 4 PL actions of SM, SMsc, and PSM. All three plots have been normalized to the same relative energy scale. (C) Falloff in energy amplitude as a function of spatial frequency, averaged over all four PL actions. The curves were created by sampling the spatial-frequency axis with a sliding Gaussian bin of 0.25r. The four colored curves represent spectra for BM (black), PSM (red), SM (dark blue), and SMsc (light blue). The inset magnifies the plot in the region of the spatial-frequency spectrum where there are notable departures from monotonicity in the BM and PSM spectra. These signature irregularities are conspicuous in the individual spectra for each action, too.

SMsc sequences derived from all four animations is considerably larger than the variance among PSM sequences for those animations. We also derived plots of spectral amplitude for limited bands of spatial frequency, each differing in center frequency, for the underhand-toss animations. Figure 4C plots the amplitudes of those band-pass-limited spectra for SM, SMsc, PSM, and BM sequences, all of which show the falloff in power with spatial frequency so often seen in other complex images (e.g., Geisler, 2008). Note, though, that both the BM and PSM sequences contain a pronounced spike in energy within a narrow range of spatial frequencies, a spike not seen in either the SM or the SMsc spectra. We surmise that this arises from the similarity of interdot distances between neighboring joints defined by pairs of dots designating the arms and the legs. The spikes in those two spectra, in other words, underscore the salience of the linkage between pair-wise dots, a linkage that is destroyed when spatially scrambled sequences are generated. Figure 5 shows 2-D spectral energy analyses that summarize the similarities between SM and BM, between SMsc and BM, and between PSM and BM, averaged across all four animations. The left-hand panel

was created by generating 80 samples of each of the four animations using the SM procedure (i.e., random relocation of starting dots), taking the average spectrum for each of the four samples, and then subtracting from each of those averages the spectrum associated with the BM version of that sample. Those four resulting difference spectra were themselves then averaged to produce the 2-D. The same steps were repeated using samples created by the SMsc algorithm and by the PSM algorithm, and those average spectra difference plots are shown in the middle and right-hand panels, respectively. Comparison of those three population difference spectra clearly shows that the BM sequences, on average, are more similar in spectral energy to the sequences produced using the PSM algorithm than they are to sequences produced using the SM or the SMsc procedures (i.e., the subtraction leaves more residual energy in the difference spectra for SM and for SMsc compared to the PSM difference spectrum). Those difference maps reveal that, in terms of 2-D spectral energy, sequences produced by the two scrambling procedures—SM and SMsc—primarily differ in terms of their low spatial content compared to sequences produced by PSM (also evident in Figure 4C).



10

Figure 5. Differences in amplitude spectrum of individual 2-D frame images between different motion types on 4 actions 3 80 samples. Each heat map shows the mean absolute difference in amplitude between SM and BM, SMsc and BM, and PSM and BM. The color temperature represents the normalized (divided by the maximum amplitude) spectral energy. BM is more different from SM and SMsc than it is from PSM, particularly at low spatial frequencies.

The analyses just described summarized energy measures within the 2-D spectra by averaging spectra derived for successive frames. We also derived 3-D spectral energy maps by creating space-time plots of the changing positions of dots over the course of an animation (Figure 6A) and then deriving the Fourier amplitude spectrum of the pattern of luminance changes occurring within that x/y/t space (Figure 6B). This procedure was repeated for each of the four BM versions, and also for 80 samples of each of the four actions modified using the SM and SMsc scrambling procedure and 80 samples of each of the four actions modified using the PSM algorithm. To derive an index expressing the similarity between the spectrum for a given BM sequence and the sample spectra for that sequence modified by SM, SMsc, and BM, we did the following: For a given PL activity, we calculated the absolute difference between each of the 80 SM 3-D spectra for a given action and the 3-D BM spectra for that action, then summed those absolute differences and divided the total by the total BM spectral amplitude measured over all four actions. The exact same procedure was followed to derive the dissimilarity index for the SMsc sequences and for the PSM sequences produced by a given PL activity:

SMi ¼

i X

ðjampSMi ampBMijÞ=ðR ampBMÞ

ð1Þ

1;80

PSMi ¼

i X

ðjampPSMi ampBMijÞ=ðR ampBMÞ: ð2Þ

1;80

This yielded four pairs of index values—i.e., values for BM/SM, BM/SMsc, and BM/PSM—for each of the four action sequences. It is important to note that smaller index values denote greater similarity. Those pairs of index values are plotted in Figure 6C and D, where each different colored symbol represents a given action from the set of four total. Here it can be seen

that different animations generate different degrees of similarity between the BM and the perturbed versions, but clearly the similarity between BM and PSM is overall greater (i.e., the differences in 3-D energy spectra are smaller) than the similarity between SM and BM or SMsc and BM; nearly all of the PSM–BM index values lie below the unity line. In summary, Fourier analyses of BM, SM, SMsc, and PSM sequences indicate that BM and PSM are more similar than are BM and SM or SMsc, thus confirming what visual inspection implied when we began exploring different means for perturbing BM sequences.

Motion analyses of a wider range of PL actions We also computed several indices of motion energy for SM, PSM, and BM PL sequences from a library of 25 PL animations depicting a wide range of activities (Ahlström et al., 1997). That set of animations is diverse in terms of actor viewpoint, number of dots used to create PL figures, overall net motion, and detectability of animations presented in noise (Jung et al., 2013). Would the similarity of PSM to BM, compared to SM and BM, found in the previous sections also emerge from this highly varied set of animations? The answer is yes, as the following two motion analyses reveal. For one analysis, we computed the absolute difference in 2-D spectral energy between SM and BM and between PSM and BM, and the results are shown in, respectively, Figure 7A and B. These difference maps are averages based on 56 samples of each of the 25 PL action sequences. Among these diverse BM sequences, we see that PSM more closely resembles BM in terms of 2-D energy content than does the SM version of those animations. Figure 7C shows the subtraction of the average spectrum for SM from the average spectrum for PSM, revealing that the difference between the two is most prominent at low spatial frequency.



11

Figure 6. Summary of procedure and results from 3-D Fourier analysis of sequences depicting BM, SM, SMsc, and PSM. (A) Space-time pattern of the jumping-jack biological-motion sequence. The frame-to-frame sets of 12 dots are ordered in time and tagged by color temperature (early to late: blue to red). The thin lines showing the skeletal structure relating individual dots were not present in the sequence of frames defining the animation. (B) Fourier amplitude spectrum of the luminance space-time pattern depicted in (A), represented by the color temperature of voxels in the three-dimensional spatiotemporal-frequency domain. (C) Scatter plot showing dissimilarity indices for SM relative to BM (vertical axis) and SMsc relative to BM (horizontal axis) for each of the four actions (denoted by the four different colors of circles, 80 samples/animation type); the graph on the right plots the mean and standard error of the values for each of the four actions. (D) Both panels follow the same format as (C), except that now the vertical axis plots the dissimilarity index for PSM relative to BM.

We also calculated the dissimilarity indices for SM versus BM and for PSM versus BM (as defined in Equations 1 and 2), and the results for each of the 25 animations are plotted in Figure 7D (each colored symbol denotes a given PL action). We again see that the similarity between BM and PSM is overall greater (i.e., the differences in 3-D energy spectra are smaller) than the similarity between SM and BM; this is indicated by the fact that nearly all of the PSM–BM index values lie below the unity line in Figure 7D. Next we capitalized on the statistical power afforded by this large sample of PL actions to derive modelbased estimates of motion energy for BM, SM, SMsc, and PSM animations; details of that model are given by Lee and Lee (2012). In brief, the model comprises motion-energy filters that sample the 3-D spatiotemporal frequency plane at 288 different translational motion velocities (16 equally spaced directions 3 18 speeds covering 08/s–908/s, equally spaced in linear speed steps of 58/s). In the model these filters correspond to speed/direction planes whose tilt in zenith angle and in azimuth angle specify different directions and speeds, respectively.

To compute motion-energy values, the steps shown schematically in Figure 8 were implemented. We started with the 3-D Fourier energy spectra from space/ time plots of the successive frames of BM, SM, SMsc, and PSM animations (i.e., the spectra derived in the previous analysis). A given spectrum was then sampled by each of the 288 filter planes to produce 2-D motionenergy maps like the one shown in the heat map in Figure 8. The schematic shows two example filters tuned to different velocities, with their contributions to the motion-energy map shown by the dark arrows. We then calculated the difference in motion energy between SM and BM and the difference between PSM and BM by subtracting their respective motion-energy maps for a given PL sequence. Panels A and B in Figure 9 show examples of this procedure applied to one specific animation (underhand toss). Notice that at slow speeds (corresponding to low temporal frequencies), the SM sequence (Figure 9A) contains a noticeable splash of energy in excess of that found in the corresponding BM sequence. This is not nearly so conspicuous in Figure 9B, which shows the difference between PSM and BM. The same pattern of results is observed in the average of the population energy maps



12

Figure 7. Comparison of 2-D and 3-D motion energy in SM, PSM, and BM animations, derived from 25 PL actions 3 56 samples/action. (A) Mean absolute difference in 2-D amplitude between SM and BM. The color temperature represents the normalized (divided by the maximum amplitude) spectral energy. (B) Mean absolute difference in 2-D amplitude between PSM and BM. (C) Subtraction of (B) from (A), indicating that BM is more different from SM than from PSM, particularly at lower spatial frequencies. (D) Scatter plot summarizing differences in 3-D spectra of SM and PSM relative to BM. The 3-D spectra for BM, SM, and PSM versions of each of the 25 activities were generated in the manner illustrated in Figure 6. Dissimilarity indices for SM compared to BM and for PSM compared to BM were then derived using the formulae described in the text and plotted in this scatter plot. Each colored dot designates one of 25 different animation activities, with vertical and horizontal lines indicating standard errors. Smaller values of the similarity index denote greater similarity between BM and its companion SM or PSM.

associated with all 25 animations, as shown in Figure 9C and D. We conclude that the main source of this conspicuous difference between spatially scrambled animations and BM can be traced to the random repositioning of individual dots in the scrambled sequences, for this contamination of the motion-energy spectrum is minimized when the PSM algorithm is used.

Motion analyses of the canonical walker Probably the single most widely used PL animation in research on biological-motion perception is the socalled PL walker, an example of which is shown in Movie 5BM. This familiar figure is sometimes shown from a side view and other times as if walking forward or backward or at an oblique angle. Given the popularity of this PL animation, we felt it worthwhile to create and analyze SM, SMsc, and PSM versions; the animations can be seen in the other three sequences animated in Movie 5, and the motion-analysis results are summarized in Figure 10. Figure 10A shows histograms of the interdot distances for BM and for each of the three perturbation variants, using the same format as described in Figure 3C. Thus the solid curve

repeated in each panel is the actual distribution for the BM version of the walker, and the gray areas are the distributions for the three scrambling techniques (SM, SMsc, and PSM). Figure 10B shows the 2-D Fourier spectra derived from the original version of the PL walker shown in Movie 5 (BM) along with average spectra derived from 80 independent replications of each of three perturbation techniques—SM, SMsc, and PSM—using the same format as described in Figure 4A. Here it can be seen that the spectra for SM and SMsc resemble one another and are both distinctively different from the spectra for BM and for PSM, which are highly similar to one another. In general, application of these three different techniques for generating perturbed PL walker sequences yields the same patterns of similarities and differences documented for other activities.

2IFC discrimination performance comparing PSM and SM The pronounced differences between spatially scrambled and PSM sequences naturally leads to the following question: How do these two forms of



Figure 8. Steps involved in deriving the motion-energy content in a given animation sequence. The 3-D Fourier energy spectrum for a given animation is derived in the manner summarized in Figure 6. That spectrum is then sampled by motion-energy filters, two of which are illustrated; the filters are visualized in the Fourier domain. The 3-D orientation of a given filter plain specifies the speed and direction of a translational motion to which the filter is sensitive in a 2-D image domain. The two example filters shown in the two panels detect different translational motions specified by two different pairs of orientation and speed. The color-coded 2-D plot shows the motion energy in an SM version of the underhand-toss animation, with energy expressed in 288 voxels each corresponding to a different motion-energy filter. The two example filters are sensitive to leftward translational motion at 458/s (left-hand filter) and to motion upward and slightly to the right at 158/s (right-hand filter).

perturbation stack up against one another in terms of their impact on the relative difficulty of a discrimination task involving PL sequences presented in noise? Recall that SM largely destroys pairwise motion relations among dots on neighboring parts of the body (e.g., the pendular action of the wrist relative to the motion of the elbow). Those relationships are retained to a large degree in animations created using the PSM algorithm, so we would predict that normal PL animations should be more difficult to discriminate from PSM sequences relative to SM sequences. To test this prediction, we performed the following psychophysical experiment. We used a 2IFC task together with a noise-masking paradigm to measure thresholds for discriminating normal PL animations from perturbed ones. On each trial, the observer viewed two successive animations

13

Figure 9. Difference maps of motion energy in pairs of animations. (A–B) For underhand-toss PL action, signed difference in translational motion energy for SM and BM (A) and for PSM and BM (B). (C–D) Population (25 stimuli 3 56 samples) averages of translational motion-energy differences between SM and BM (C) and between PSM and BM (D).

comprised of black dots moving against a white background; each animation lasted 1 s, and the two animations forming a trial were separated by 0.5 s. Both intervals contained a PL sequence composed of approximately 12 dots embedded within an array of noise dots. (We say approximately because a dot will sometimes disappear briefly owing to occlusion.) In one interval, the sequence portrayed an unperturbed target PL animation depicting one of four possible activities, and in the other interval the nontarget sequence was a perturbed version of the activity shown in the target interval; the interval containing the unperturbed sequence varied randomly over trials. To discourage observers from monitoring just one small area of the display, the position of the target and nontarget relative to the fixation point were varied randomly over trials

Movie 5. PL sequence of a leftward-facing person walking in place, along with examples of perturbed animation sequences produced to create SM, SMsc, and PSM versions (shown from left to right, respectively). (Animation courtesy of Frank Pollick, with modification to eliminate lateral motion.)



14

Figure 10. (A) Histograms of interdot distances measured for each of all the possible pairs of dots within each of the motion frames of the four PL types of the PL walker shown in Movie 5. The blue curve repeated in all four panels corresponds to the interdistance distribution for the BM sequence, and the shaded gray areas within each plot correspond to the actual distributions of interdot distances for BM, SM, SMsc, and PSM. (B) 2-D Fourier spectra derived from the original version of the PL walker along with average spectra derived from 80 independent replications of each of three perturbation techniques: SM, SMsc, and PSM. The PL walker animation used in these analyses was created using a high-precision motion-capture system that generated PL sequences capable of being animated at 60 Hz. We are grateful to the creator of this animation, Frank Pollick, for sharing it with us.

within limits. On each trial the target could be one of four activities: climbing, drop-kicking, overhead throwing, or underhand tossing. These four were selected based on their similarity in difficulty of discriminability as determined in an earlier study using the same 2IFC noise-masking procedure (Jung et al., 2013). Both sequences, target and nontarget, were embedded within a field of dynamic noise dots whose number was varied over trials according to QUEST (Watson & Pelli, 1983), an adaptive procedure that determined the threshold level of noise producing 75% correct performance on this 2IFC task. A total of six repetitions of this QUEST procedure were administered, with three sets of thresholds estimated using PSM as the nontarget and three estimated using SM as the nontarget. The noise dots in both intervals of any given trial were generated online, prior to each trial, by replicating the motion trajectories of the dots comprising the PL figure, with independent noise samples created for the target and nontarget intervals. Each of the six QUEST estimates was based on 50 trials. Fourteen young adults who were unaware of the purpose of the experiment participated, and all received practice on the task before formal data collection. As found in a previous study from our lab (Jung et al., 2013), performance on this noise-masking task varied substantially among individuals. Note, however, that thresholds measured using PSM and using SM nontargets are highly correlated across observers (r ¼

0.91, p , 0.0001), as is readily obvious from the scatter plot in Figure 11. Each filled symbol in the scatter plot shows pairs of threshold estimates from a given observer (in units of number of noise dots), one obtained using SM as the nontarget and the other obtained using PSM as the nontarget. The open symbol shows the across-subjects averages and associated standard errors. Of relevance for our question, the staircase trials utilizing SM as the nontarget foil required more noise dots to achieve a criterion level of performance than did staircase trials utilizing PSM, a difference that is statistically significant, t(13) ¼ 5.75, p , 0.00001. Observers did tend to show overall improvement on this task over blocks of trials, a practice effect that has been documented elsewhere (Grossman et al., 2004). But the difference between PSM and SM did not disappear, and instead was evident during practice blocks and over the three pairs of successive SM and PSM blocks of trials comprising the actual experiment. So these results confirm the prediction based on the motion-energy measures described in the previous section (e.g., recall Figure 7C): in terms of motion energy, PSM is more similar to BM than is SM, and for that reason fewer BM noise dots are needed to interfere with discrimination of BM compared to SM noise dots.



Figure 11. Results from the psychophysical noise-masking experiment. The scatter plot summarizes masking-noise levels (expressed as number of noise dots at threshold) required to produce a criterion level of performance on a 2IFC discrimination task where one interval contained BM and the other interval contained either SM or PSM (this factor varied between blocks of trials, not within a block). Each filled symbol shows for a given observer the thresholds obtained using SM (x-axis) and using PSM (y-axis). The open symbol shows the average thresholds over the 14 observers, with the horizontal and vertical lines denoting the standard error of the mean. The correlation between SM and PSM noise levels is highly significant (r ¼ 0.91, p , 0.00001), as is the difference between performance measures associated with the two different forms of noise, t(13) ¼ 5.75, p , 0.00001.

Discussion In this article, we introduced a novel technique for generating perturbed motion sequences from PL biological-motion animations, a technique that preserves second order spatial relationships among dot pairs comprising normal (i.e., unperturbed) PL biological-motion sequences and closely matches the motion-energy content of those unperturbed sequences. We foresee that PSM might be an ideal foil display in studies that are interested explicitly in global perception of biological motion, not just second order spatial relations among pairs of dots, and that need foil displays that do not inject additional motion information into the displays. These could include (a) brainimaging studies that use PL animations to identify sources of neural responses evoked explicitly by biological motion and not just pair-wise, structured motion and (b) psychophysical studies of biologicalmotion perception in humans or animals where even subtle differences in motion information (e.g., motion energy) between target and foil could support successful discrimination without necessarily isolating biological motion. In a similar vein, it could be informative to

15

use PSM when testing computational models of PL action perception. To give just one example, an essential component in the model devised by Giese and Poggio (2003) is the so-called motion pattern neurons, which are activated by combinations of learned patterns of optic flow defining different PL actions. One piece of evidence cited in support of that model is its robustness to the presence of SM noise when required to detect a PL walker embedded within that noise. How would the model perform when confronted with PSM? Moreover, in future work, it could be revealing to examine the correlation between indices of spatiotemporal information content and discriminability for individual PL animations that vary in their salience as assessed, for example, by susceptibility to noise masking. In closing, we want to underscore that our message is not that PSM should replace other perturbation techniques, nor that other techniques are irrevocably flawed. We agree with a comment offered by a reviewer of this article, namely that there is no single, correct way to generate scrambled biological motion. The key is to identify what information one wants to retain in the perturbed sequences and what information one wants to eliminate. In this spirit, PSM can be added to the arsenal of biological animation techniques and used when appropriate. Moreover, we hope that our approach to the evaluation of information contained in PL sequences may be of general use to others in future investigations of biologicalmotion perception.

Conclusion We demonstrated a novel technique for producing perturbed animations (referred to as pair-wise shuffled motion) via a stimulus that maintains the characteristic features and quantitative properties related to the spatial domain, amplitude spectra, and motion energies similar to those of the coherent PL BM display, yet through only removing the sense of BM perception. Our analyses yielded robust results across various types of actions within a library of BM sequences, and psychophysical results show that PSM provides an effective foil for BM in a discrimination experiment where observers are simply indicating which of two intervals contains BM. We suggest that this new technique for PSM be considered for use in future research aimed at clarifying genuine BM processing and its perceptual and neural underpinnings. Keywords: biological motion, masking, motion energy, pair-wise shuffled motion, point-light animation



Acknowledgments The authors are grateful to Frank Pollick for sharing with us his library of PL animations created using a refined motion-capture system and for his permission to use the PL walker in our motion analyses. This work was supported by the Brain Research Program (20062005108); the Mid-career Researcher Program (NRF2013R1A2A2A03017022) through the National Research Foundation of Korea grant funded by the Ministry of Science, ICT, and Future Planning; and by an NIH Center Core Grant (P30-EY008126). This project began while JK was a postdoctoral fellow at Seoul National University, supported by funds from the listed NRF grant. Commercial relationships: none. Corresponding author: Randolph Blake. Email: [email protected]. Address: Vanderbilt Vision Research Center and Department of Psychology, Vanderbilt University, Nashville, TN, USA.

Footnotes 1

This idea of distilling kinematic information from full-body animations can actually be dated back to the ´ technique developed by Etienne-Jules Marey, dubbed chronophotography, which produced a single photographic representation in stick-figure format of the movements of an organism through space over time. Johansson’s technique has the advantage of eliminating the form information carried by a stick figure, although it is arguable whether residual form information is also contained in animations produced using the PL technique (Beintema & Lappe, 2002; Gilaie-Dotan, Bentin, Harel, Rees, & Saygin, 2011; Thurman & Grossman, 2008). 2 Following submission of the original version of this article in fall 2014, we learned that Cusack, Williams, and Neri (2015), in their study of biological motion in individuals with autism, included a form of PL animation perturbation that they dubbed temporal limb scrambling, whereby pairs of dots defining each limb were shuffled in terms of their starting phases. Our procedure, in contrast, disrupts only the spatial positions of pairs of limbs. 3 Software written in MATLAB for implementing this PSM procedure is available for download at http:// www.psy.vanderbilt.edu/faculty/blake/PSM/PSM.zip.

References Ahlström, V., Blake, R., & Ahlström, U. (1997).

16

Perception of biological motion. Perception, 26(12), 1539–1548. Annaz, D., Campbell, R., Coleman, M., Milne, E., & Swettenham, J. (2012). Young children with autism spectrum disorder do not preferentially attend to biological motion. Journal of Autism and Developmental Disorders, 42(3), 401–408. Beintema, J. A., & Lappe, M. (2002). Perception of biological motion without local image motion. Proceedings of the National Academy of Sciences, USA, 99(6), 5661–5663. Blake, R., & Shiffrar, M. (2007). Perception of human motion. Annual Review of Psychology, 58, 47–73. Blake, R., Turner, L. M., Smoski, M. J., Pozdol, S. L., & Stone, W. L. (2003). Visual recognition of biological motion is impaired in children with autism. Psychological Science, 14(2), 151–157. Bonda, E., Petrides, M., Ostry, D., & Evans, A. (1996). Specific involvement of human parietal systems and the amygdala in the perception of biological motion. The Journal of Neuroscience, 16(11), 3737– 3744. Chandrasekaran, C., Turner, L., Bulthoff, H. H., & Thornton, I. M. (2010). Attentional networks and biological motion. Psihologija, 43(1), 5–20. Chang, D. H. F., & Troje, N. F. (2008). Perception of animacy and direction from local biological motion signals. Journal of Vision, 8(5):3, 1–10, doi:10.1167/ 8.5.3. [PubMed] [Article] Cusack, J. P., Williams, J. H. G., & Neri, P. (2015). Action perception is intact in autism spectrum disorder. The Journal of Neuroscience, 35(5), 1849– 1857. Cutting, J. E. (1978). A program to generate synthetic walkers as dynamic point-light displays. Behavior Research Methods & Instrumentation, 10(1), 91–94. Garcia, J. O., & Grossman, E. D. (2008). Necessary but not sufficient: Motion perception is required for perceiving biological motion. Vision Research, 48(9), 1144–1149. Geisler, W. S. (2008). Visual perception and the statistical properties of natural scenes. Annual Review of Psychology, 59, 167–192, doi:10.1146/ annurev.psych.58.110405.085632. Giese, M. A., & Poggio, T. (2003). Neural mechanisms for the recognition of biological movements. Nature Neuroscience,4,179–192. Gilaie-Dotan, S., Bentin, S., Harel, M., Rees, G., & Saygin, A. P. (2011). Normal form from biological motion despite impaired ventral stream function. Neuropsychologia, 49, 1033–1043. Gilaie-Dotan, S., Saygin, A. P., Lorenzi, L. J., Rees,



G., & Behrmann, M. (2015). Ventral aspect of the visual form pathway is not critical for the perception of biological motion. Proceedings of the National Academy of Sciences, USA, 112(4), E361– E370. Grossman, E. D. (2005). Evidence for a network of brain areas involved in perception of biological motion. In M. Grosjean, G. Knoblich, M. Shiffrar, & I. M. Thornton (Eds.), Human body perception from the inside out (pp. 361–384). Oxford, UK: Oxford University Press. Grossman, E. D., Blake, R., & Kim, C.-Y. (2004). Learning to see biological motion: Brain activity parallels behavior. Journal of Cognitive Neuroscience, 16(9), 1669–1679. Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan, V., Neighbor, G., & Blake, R. (2000). Brain areas involved in perception of biological motion. Journal of Cognitive Neuroscience, 12(5), 711–720. Howard, R. J., Brammer, M., Wright, I., Woodruff, P. W., Bullmore, E. T., & Zeki, S. (1996). A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain. Current Biology, 6, 1015–1019. Ikeda, H., Blake, R., & Watanabe, K. (2005). Eccentric perception of biological motion is unscalably poor. Vision Research, 45(15), 1935–1943. Jackson, S., & Blake, R. (2010). Neural integration of information specifying human structure from form, motion, and depth. The Journal of Neuroscience, 30(3), 838–848. Jastorff, J., & Orban, G.A. (2009). Human functional magnetic resonance imaging reveals separation and integration of shape and motion cues in biological motion processing. The Journal of Neuroscience, 29(22), 7315–7329. Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception & Psychophysics, 14, 195–204. Jung, E. L., Zadbood, A., Lee, S.-H., Tomarken, A. J., & Blake, R. (2013). Individual differences in the perception of biological motion and fragmented figures are not correlated. Frontiers in Psychology, 4, 795, doi:10.3389/fpsyg.2013.00795. Kailath, T., (2010). The divergence and Bhattacharyya distance measures in signal selection. IEEE Transactions on Communication Technology, 15, 52–60. Kim, J., Blake, R., Park, S., Shin, Y.-W., Kang, D.-H., & Kwon, J. S. (2008). Selective impairment in visual perception of biological motion in obsessivecompulsive disorder. Depression and Anxiety, 25(7), E15–E25.

17

Kim, J., Park, S., & Blake, R. (2011). Perception of biological motion in schizophrenia and healthy individuals: A behavioral and fMRI study. PLoS ONE, 6(5), e19971, doi:10.1371/journal.pone. 0019971. Klin, A., Lin, D. J., Gorrindo, P., Ramsay, G., & Jones, W. (2009). Two-year-olds with autism orient to non-social contingencies rather than biological motion. Nature, 459, 257–261, doi:10.1038/ nature07868. Krakowski, A. I., Ross, L. A., Snyder, A. C., Sehatpour, P., Kelly, S. P., & Foxe, J. J. (2011). The neurophysiology of human biological motion processing: A high-density electrical mapping study. NeuroImage, 56, 373–383, doi:10.1016/j. neuroimage.2011.01.058. Lee, H. A., & Lee, S.-H. (2012). Hierarchy of directiontuned motion adaptation in human visual cortex. Journal of Neurophysiology, 107, 2163–2184, doi:10. 1152/jn.00923.2010. Pavlova, M. A. (2012). Biological motion processing as a hallmark of social cognition. Cerebral Cortex, 22, 981–995, doi:10.1093/cercor/bhr156. Pavlova, M. A., Lutzenberger, W., Sokolov, A. N., Birbaumer, N., & Krägeloh-Mann, I. (2007). Oscillatory MEG response to human locomotion is modulated by periventricular lesions. NeuroImage, 35(3), 1256–1263, doi:10.1016/j.neuroimage.2007. 01.030. Pavlova, M., & Sokolov, A. (2003). Prior knowledge about display inversion in biological motion perception. Perception, 32(8), 937–946. Puce, A., & Perret, D. (2003). Electrophysiology and brain imaging of biological motion. Philosophical Transactions of the Royal Society B, 358, 435–445. Saygin, A. P. (2007). Superior temporal and premotor brain areas necessary for biological motion perception. Brain, 130, 2452–2461. Saygin, A. P., Wilson, S. M., Hagler, D. J., Jr., Bates, E., & Sereno, M. I. (2004). Point-light biological motion perception activates human premotor cortex. The Journal of Neuroscience, 24(27), 6181– 6188. Schenk, T., & Zihl, J. (1997). Visual motion perception after brain damage: II. Deficits in form-frommotion perception. Neuropsychologia, 35(9), 1299– 1310. Sumi, S. (1984). Upside-down presentation of the Johansson moving light-spot pattern. Perception, 13, 283–286. Thurman, S. M., Giese, M. A., & Grossman, E. D. (2010). Perceptual and computational analysis of



critical features for biological motion. Journal of Vision, 10(12):15, 1–14, doi:10.1167/10.12.15. [PubMed] [Article] Thurman, S. M., & Grossman, E. D. (2008). Temporal ‘‘bubbles’’ reveal key features for point-light biological motion perception. Journal of Vision, 8(3):28, 1–11, doi:10.1167/8.3.28. [PubMed] [Article]

18

Vaina, L. M., Lemay, M., Bienfang, D. C., Choi, A. Y., & Nakayama, K. (1990). Intact ‘‘biological motion’’ and ‘‘structure from motion’’ perception in a patient with impaired motion mechanisms: A case study. Visual Neuroscience, 5(4), 353–369. Watson, A. B., & Pelli, D. G. (1983). QUEST: A Bayesian adaptive psychometric method. Perception & Psychophysics, 33(2), 113–120.

A Novel Technique for Generating and Observing Chemiluminescence in a Biological Setting.

Local form interference in biological motion perception.

The neurophysiology of biological motion perception in schizophrenia.

A new technique for generating pathogenic barcodes in breast cancer susceptibility analysis.

The Continuous Motion Technique for a New Generation of Scanning Systems.

A method for describing the motion of biological systems.

Motion aftereffects: evidence for parallel processing in motion perception.

A Pursuit Theory Account for the Perception of Common Motion in Motion Parallax.

Motion parallax thresholds for unambiguous depth perception.

Minimal conditions for the perception of rotary motion.

Form information is necessary for the perception of motion.

A touchscreen based global motion perception task for mice.

A new technique for minilaparotomy.

A new technique for urethrography.

Model for the computation of self-motion in biological systems.

The Perception of Auditory Motion.

The perception of visual motion.

Neural population models for perception of motion in depth.

Consequences of spatial sampling for human motion perception.

A Possible Role for End-Stopped V1 Neurons in the Perception of Motion: A Computational Model.

Individual differences in the perception of biological motion and fragmented figures are not correlated.

Accuracy and Tuning of Flow Parsing for Visual Perception of Object Motion During Self-Motion.

Two carriers for motion perception: color and luminance.

Visual Acceleration Perception for Simple and Complex Motion Patterns.