Vision Res. Vol. 31, No. 6, pp. 951-966, 1991 Printed in Great Britain. All rights reserved

0042-6989/91 $3.00 + 0.00 Copyright © 1991 Pergamon Press plc

COLOR

IN VISUAL

SEARCH

MICHAELD'ZMta~ Department of Cognitive Sciences and IRU in Mathematical Behavioral Sciences, University of California, Irvine, CA 92717, U.S.A.

(Received 4 April 1990; in revised form 1 October 1990) Abstract--Colored targets pop out of displays under conditions in which the standard red-green, yellow-blue and black-white mechanisms cannot directly mediate detection. Experimental evidence suggests that observers possess chromatic detection mechanisms tuned to intermediate hues such as orange as well as to hues characterizing the standard color-opponent mechanisms and that these mechanisms, as a group, form a fine-grained representation of hue within the central visual field. Spatially-parallel search is mediated by a single such mechanism that is spectrally sensitive to the target chromaticity but insensitive to the distractor chromaticities; different mechanisms are used to detect a single target in a way that depends on distractor chromaticities. Color

Detection

Search

Attention

INTRODUCTION

Over a century ago, Hering noted that the four hues red, green, yellow and blue are fundamental in the sense that they cannot be described as mixtures of two or more hues (Hering, 1878/1964). Crucial to this observation is the existence of corresponding lights that have a unique hue appearance. Lights that appear unique yellow, for instance, do not appear reddish, greenish or bluish. Intermediate hues, on the other hand, may be describd as appropriate mixtures of the fundamental hues (e.g. orange is a combination of red and yellow). The unique hues, furthermore, exist in complementary pairs (red/green and yellow/blue) that do not readily coexist at any one point in space and time (but see Crane & Piantanida, 1983). Hering explained these phenomena by postulating four chromatic mechanisms that come to be opposed in pairs, so that each of the two resulting opponent mechanisms is activated in opposite directions by lights from the two spectral regions to which it is most sensitive. It was recognized in the previous century that opponent mechanisms need not be an alternative to the trichromatic theory of color vision (Young, 1802; Helmholtz, 1909/1962) but an elaboration: opponent mechanisms may constitute a second or later stage (cones being the first) in the processing of chromatic information (Hurvich & Jameson in Hering, 1878/1964; see also Judd, 1949). It was not until the middle

of this century, however, that physiological (Svaetichin & MacNichol, 1958; DeValois, 1965) and psychophysical (Jameson & Hurvich, 1955; Hurvich & Jameson, 1955, 1957) work put this hypothesis on a scientific footing. Jameson and Hurvich (1955) used a hue-cancellation technique to establish the spectral sensitivities of the red-green and yellow-blue mechanisms. The blueness of lights of varying wavelength, for instance, is cancelled by adding a single yellow light of appropriate intensity; the amount of yellow needed to cancel the blueness provides a measure of the spectral sensitivity of the blue mechanism. Hurvich and Jameson (1957) used these measurements of appearance to model the sensitivities of the opponent mechanisms as appropriate linear combinations of the sensitivities of cone mechanisms. More recently there has been a call for mechanisms with spectral sensitivities that match intermediate hues such as orange or purple. Krauskopf and his collaborators first proposed higher-order mechanisms of color vision tuned to intermediate colors, drawing on their psychophysical work with a chromatic contrast habituation technique (Krauskopf, Williams & Heeley, 1982; Krauskopf, Williams, Mandler & Brown, 1986). The elevation of thresholds, caused by pre-exposure to high-contrast modulations of chromaticity along an intermediate-color axis in the isoluminant plane, was minimal for color contrast signals presented along the perpendicular axis, suggesting the presence of 951

952

MICHAELD'Zuut~

mechanisms tuned chromatically to the signal and relatively insensitive to the habituating stimulus. D'Zmura and Lennie (1986) speculated o n the existence of a multiple-mechanism organization for higher-level chromatic processing based on their study of surface color appearance; they showed how such mechanisms might be constructed from the responses of lower-level mechanisms of color vision characterized by electrophysiological experiments in macaque monkey (Derrington, Krauskopf & Lennie, 1984). Such mechanisms were pursued psychophysically by D'Zmura, Krauskopf and Lennie (1987), who reported on the bandwidths and spectral sensitivities of color-opponent mechanisms found using a heterochromatic noise masking technique. Results found by masking signals of intermediate hue (e.g. orange) demonstrated the color-vision analog to "off-frequency listening" in audition (Patterson & Nimmo-Smith, 1980) and "off-frequency looking" in spatial vision (Pelli, 1981); the results, furthermore, suggested the presence of detection mechanisms tuned to a wide variety of hues (D'Zmura, 1990). The results of the chromatic contrast habituation experiments suggest that such higherorder mechanisms of color vision exist in a "hard-wired" state: there are neurons tuned to many directions in the color circle that habituate to high-contrast stimuli (Krauskopf et al., 1982, 1986). Physiological results also suggest such an organization for higher-level mechanisms. Work on the chromatic sensitivities of single neurons in macaque V1 by Lennie, Krauskopf and Sclar (1990) and in macaque V4 by Schein and Desimone (1990) shows that the peak hue sensitivities of colorselective neurons in these visual areas are scattered more uniformly, rather than clustered about two axes as in macaque lateral geniculate nucleus (Derrington et al., 1984). The physiological evidence from studies of macaque cortex is consistent with the notion that we possess a fine-grained representation of hue within the central visual field. To test whether such mechanisms play a role in tasks met outside the laboratory, I transfer the experimental logic behind the contrast habituation and noise masking experiments to the visual search paradigm used by Treisman and her colleagues (Treisman, Sykes & Gelade, 1977; Triesman & Gelade, 1980; Treisman & Gormican, 1988). Observers in the search task

are required to signal, as quickly as possible, whether a known target item (e.g. an orange disk) is present within a display that may simultaneously include nonoverlapping distracting items (e.g. yellow-green and violet disks). The target is present on 50% of the trials, chosen randomly; the number of distractors that are displayed varies randomly from trial to trial. The independence of reaction time from number of distractors, in signalling correctly that the target is present, is strong evidence that search is conducted spatially in parallel. The target "pops out" from the field of distractors. Such distractors evidently provide negligible noise power to the mechanism(s) that mediate target detection. It is often the case, however, that the reaction times increase monotonically with the number of distracting items within the display. Such distractors furnish too much noise power to spatially-extended detection mechanisms. The observer can respond accurately on trials only if he or she restricts the receptive field of the detection mechanism and applies this localized mechanism to individual items or clusters of items (Pashler, 1987) in a serial fashion. The following experiments are designed to answer two basic questions. Do colored targets pop out of displays under circumstances in which the standard red-green and yellow-blue mechanisms cannot possibly mediate detection? Experimental results show that this is so; they furthermore suggest that the detection mechanisms responsible are each chromatically linear and, as a group, form a fine-grained representation of hue within the central visual field. Second, is it possible for observers to detect a target with unique-hue appearance, for instance red, using a mechanism tuned to, perhaps, orange or violet? This can be an optimal strategy if distractors are chosen to prohibit detection by the standard color-opponent mechanisms. Results show that targets with unique-hue appearance can be sought spatially in parallel by mechanisms tuned to neighboring intermediate hues. This work was described elsewhere in preliminary form (D'Zmura, Mailer & Lennie, 1988). METHODS

Stimuli were presented on a Tektronix 690SR color monitor that was viewed binocularly at a distance of 2 m in a dimly-lit room. Software on a PDP 11/60 computer controlled an Adage 3000 color raster display processor that

Color in visual search

provided eight bits of chromatic information for each of the 512 by 432 pixels presented on the monitor at a field rate of 119 Hz (interlaced). Applied voltage vs phosphor intensity nonlinearities were corrected for each gun with 10-bit lookup tables in the Adage. The spectra of the three EBU phosphors of the monitor were measured with an Optronics 730A spectroradiometer, while the radiance and luminance of each gun were measured with a UDT 80x Opto-Meter. The screen was set to provide a steady, neutral background with luminance l l 0 c d / m 2 and CIE 1931 (x,y) chromaticity (0.36, 0.37). Colored disks of diameter 0.78 deg of visual angle (43 pixels) were used as items in the experiments; they were equal in luminance to the background. Their chromaticities were chosen from the set of 24 shown in the CIE diagram in Fig. 1A. One of the observers (the

A 0.50 YG 0.45 ~ - ~ - -

-(

03St-0.25

0.25 0.30

0.35 x 0.40

B

G

0.45

0.50

Y

w v

w¥

R

B

Fig. 1. Chromaticities of colored disks used in the experiments. (A) Chromaticities of unique-hue disks (R red, G green, Y yellow, B b l u e ) a n d intermediate-hue disks (O orange, YG yellow-green, BG blue-green, P purple) at low, moderate and high saturations plotted in CIE 1931 x, y coordinates. The neutral background chromaticity is

marked by the central dot. (B) Schematiccolor circle with cardinal axescorrespondingto the uniquehues in whichare represented the colors of the unique-hueand intermediatehue disks arrayed about the neutral color (W) of the background. To each color corresponds a chromaticityin the chromaticitydiagram in (A); low, moderateand highsaturation colors lie on circles in the color plane with increasing radii.

953

author) determined, using single disks presented foveally for 300msec at 5 sec intervals, the directions about the white point that corresponded best, at a moderate saturation, to the unique hues red, green, yellow and blue and to the intermediate (viz. binary) hues orange, yellow-green, blue-green and purple. The observer first adjusted disk hue at fixed saturation to find unique red; this red standard was then alternated with disks whose saturations and hues were adjusted to determine the chromaticities of the other unique and binary hues at a similar perceived saturation. These settings were also made at high and low saturation levels to determine the 24 chromaticities shown in the figure. The settings agree well with earlier work on constant-hue loci (Dimmick & Hubbard, 1939a, b; Larimer, Krantz & Cicerone, 1974, 1975; Burns, Eisner, Pokorny & Smith, 1984; Ayama, Nakatsue & Kaiser, 1987) that suggests that the red-green mechanism responds linearly to low and moderate-contrast signals and that the yellow-blue mechanism possesses a mild spectral nonlinearity. The 24 disk chromaticities are also shown in a schematic color circle (Fig. 1B); its basic axes correspond to unique red and green and to unique yellow and blue, while its intermediate axes correspond to orange and blue-green and to yellow-green and purple. The schematic color circle is related to the CIE diagram in a quasi-linear fashion; it departs most strongly in the green region of the diagram. The schematic color circle is used below to describe the experiments. Each experimental run had four blocks of 40 trials. In a single block, eight trial types were each presented in random order five times: (1, 8, 16, 32 displayed items) x (target present, absent). On target-present trials with n > 1 displayed items, the display contained n/2 and n/2 - 1 instances of two classes of distractors; the two values were assigned randomly to distractor class across trials. On target-absent trials with more than one displayed item, equal numbers of the two classes of distractors were displayed; for single-item trials the distractor was assigned randomly. Each trial started with a "reminder" interval lasting one second, whose onset was signalled by a brief tone (beep), during which the observer saw the target for 300msec at a random location within the background area. The target then disappeared. A second later, two further beeps sounded at an interval of one second,

954

MICHAEL D'ZMURA

followed one second later by the target/ distractor display. The disks in the display were placed randomly on a 6 x 6 square lattice (Ax = Ay = 1.2 deg = 64 pixels) of locations, centered on the background, from which the corners had been removed. The display disappeared promptly after the observer signalled either the presence or the absence of the target; one-half second later, a signal indicated whether the target had been present (two beeps) or absent (four beeps). Observers used the dominant hand to push a "target-present" switch and the other hand to push a "target-absent" switch. Results from a single condition for a single observer are based on reaction times from the latter three blocks (120 trials) of a single experimental run. Fifteen reaction times were collected for each of the eight trial types presented in the run. There were two primary observers (MD and PL). An inexperienced observer (PM) also participated in the basic orange-target experiment (see immediately below). All observers had normal color vision and were properly refracted. RESULTS

Search for orange The first experiment probed the detectability of orange target disks of moderate saturation. Figure 2 shows the results for three observers obtained in four conditions. In the first condition (Fig. 2A), the observers sought the orange target, indicated in the color diagram by the open circle, in the presence of red and green distractors, marked by solid circles. All disks had approximately the same saturation. The results for three observers, shown to the right of the corresponding color diagrams, are presented as plots of mean reaction time vs number of items. The open symbols refer to mean reaction times for trials in which the target was present, while the solid symbols refer to target-absent trials. Reaction times in this condition (Fig. 2A) do not depend on the number of distractors present: search for the orange target in the presence of red and green distractors is conducted spatially in parallel. A tentative explanation for this result is that the observer responds to yellow events in the visual field; the yellow component of the orange target allows it to be detected in a parallel fashion by a "yellow" detection mechanism in the presence of red (and green) distractors.

Figure 2B shows the result of search for the orange target in the presence of yellow and blue distractors. The target again pops out, and a tentative explanation is that this performance is mediated by a mechanism tuned to red events: the orange target is visible to the "red" mechanism, but the yellow (and green) distractors are not. The results suggest that observers use a mechanism to search for the orange target that depends in its chromatic properties on the colors of the distractors. This is supported by the results of the third condition (Fig. 2C), in which an orange target is presented in the midst of yellow-green and purple distractors: search is again conducted in parallel. If the observers had used their "yellow" filters in this task, then the orange target presumably could not be sought in parallel because the yellow-green distractors have a significant yellow component. If the observers had selected a "red" filter for this task, then search would again not be conducted in parallel, because the purple distractors have a significant red component. The result suggests that observers can attend selectively to a combination of red and yellow: the observer detects the orange target using a color tuned to orange events, to which yellow-green and purple distractors would be invisible. Shown in Fig. 2D, finally, is the result of search for orange in the presence of red and yellow distractors. Reaction times increase noticeably with the number of items; reaction times for target-absent trials increase more quickly than those for target-present trials. Such a pattern of results is typically taken as evidence for a serial component in search (Sternberg, 1966; Egeth, Virzi & Garbart, 1984; Pashler, 1987). Observers apparently could not deploy, in a spatially-parallel fashion, a chromatically-narrowband filter tuned to "orange" to which the red and yellow distractors would be invisible; instead they scanned the display using a spatially-localized filter. A tentative interpretation of the observers' strategies in these tasks is depicted in the color circles of Fig. 3. In the parallel-search conditions, observers can set a threshold on a spatially-extended chromatic mechanism that adequately distinguishes, in the local response of the mechanism to the display, the target from the distractors. Such a mechanism is chromatically linear, apart from a possible halfwave rectification that eliminates responses

Color in visual search

955

Y

A

~

R

1500

1500

"g 1000

looO.

E ~

2OO0

"i 50o B~

40001

0

0

5oo~ =~... ° . . . . 10 20

30

0 , ~ 0 10

~, 20

30

1oo0

0

!. 0

10

. 20

. 30

10

20

30

Y

" 10oo

R

looO

~

2000. 500 00

C

500 10

20

30

•'

00

1500.

1500 .

10OO.

IOO0

l0

-

~ 20

30 4000 .

!

R

IOOO.

3OOO.

~

2000. 500.

500. ~

104)0.

O~~ ~'1500, ~[ 1500,3000,4000' 0 , 0

B

. - . 10 20

3"0

0

0

l0

20

30

0

0

10

20

30

10

20

30

Y

L

G

R

~' 1000. E ~

IOOO.

2000.

5oo

500

B

. . . . . 10 20

30

Number of Items

0

0

1000

10

20

30

Number of Items

0

0

Number of Items

Fig. 2. Results of search for an orange disk of moderate saturation among moderately-saturated red and green distractors (A), yellow and blue distractors (B), yellow-green and purple distractors (C) and red and yellow distractors (D). These conditions are represented in the schematic color circles in the first column; open circles in these diagrams refer to target color while solid circles refer to distractor colors. Results for observers MD, PL and PM are shown in the second, third and fourth columns, respectively. Average reaction times (msec) from an experimental run are plotted as a function of number of items (1, 8, 16, 32) in the display. Open symbols refer to average reaction times for trials on which the target was present; solid symbols refer to average reaction times for trials on which the target was absent. Error bars show the standard error of the mean computed from the 15 reaction times collected for each data point. Note that the scale along PM's ordinate differs from that of MD and PL. Percent correct rates in conditions A, B, C and D for observers MD, PL and PM, respectively, were: (A) 98, 98, 99; (B) 95, 98, 98; (C) 95, 96, 100; (D) 97, 95, 98. See text for discussion.

to events with complementary chromaticities (Kranda & King-Smith, 1979; Krauskopf et al., 1982; Stromeyer, Cole & Kronauer, 1985). A threshold on the response of a "yellow" mechanism is shown by the horizontal line marked T in panel A's color circle (see Fig. 3). The orange target produces a response that surpasses the threshold, while the red and green distractors and background noise produce responses that

lie below threshold. A threshold on the response of a "red" filter (Fig. 3B) separates the orange target from the yellow and blue distractors, while a threshold on the response of an "orange" filter (C) distinguishes the target from the yellow-green and purple distractors. The lines marked T in Fig. 3 are schematic; they correspond to appropriate lines in the linear CIE color space of Fig. IA.

956

MICHAEL D'ZMURA

a Parallel

B

Y

Parallel

Parallel

Not Parallel G

B

R

B

Fig. 3. Tentative interpretation of results from search for a moderately-saturated orange disk among similarly-saturated red and green distractors (A), yellow and blue distractors (B), yellow-green and purple distractors (C) and red and yellow distractors (D). In the parallel search cases (A, B and C), the lines marked "T" represent thresholds on half-wave rectified but otherwise linear chromatic mechanisms tuned to "yellow", "red" and "orange", respectively. In (D) are shown two possible nonlinear mechanisms for detecting the orange target disk among similarly-saturated red and yellow distractors. At the left is depicted the color sensitivity of a narrowband mechanism sensitive to the orange target but not to the red or the yellow distractors. At the right is a mechanism that detects the orange target by establishing that it is neither red enough to be a red distractor nor yellow enough to be a yellow distractor.

Observers apparently do not possess mechanisms that can simultaneously distinguish the orange target from both red and yellow distractors (Fig. 3D) across the entirety of the display. The chromatically-narrowband tuning required of this spatially-localized filter can be implemented either by a mechanism that is sensitive to orange but not to red or yellow (left) or by identifying the orange target as an item that is neither sufficiently red nor sufficiently yellow (right); the alternatives are not distinguished experimentally in this work.

An explanation of these results that relies solely on features provided by the standard color-opponent mechanisms runs into difficulties. One might argue, for instance, that the orange target (Fig. 3A) possesses a yellow feature that pops out among red and green distractors; (B) possesses a red feature that pops out among yellow and blue distractors, and (D) is a conjunction of red and yellow features so that it does not pop out among red and yellow distractors. Yet parallel search for orange targets among yellow-green and purple distractors (C)

Color in visual search is not readily explained in terms o f standard c o l o r - o p p o n e n t features: the orange target shares its color features with the distractors.

Control for saturation: parallel search If the p r o p o s e d interpretation o f results is correct, then varying the saturations o f the distractors in cases o f parallel search for the orange target should have no effect on reaction time. I n Fig. 4 are shown the results o f search for the moderately-saturated orange target a m o n g distractors presented at low, m o d e r a t e and high saturations. Saturations within a class o f identically-hued distractors were chosen to appear with approximately equal frequency within single trials. Results for observer M D show that the orange target is sought in parallel a m o n g (Fig. 4A) red and green distractors, (B) yellow and blue distractors and (C) yellow-

957

green and purple distractors that appear at low, moderate and high saturations. The last condition (Fig. 4C) ensures that the yellow-green distractors possess sufficient yellowness to hinder detection by a "yellow" mechanism and that the purple distractors possess sufficient redness to hinder detection by a "red" mechanism; the parallel-search result in this condition helps confirm the activity of a mechanism tuned to orange events.

Effects of saturation: nonparallel search According to the tentative interpretation, varying the saturations of red and yellow distractors should have an effect on search for the orange target. In particular, although orange targets presented among equally-saturated red and yellow distractors are not sought in parallel, as shown for low-saturation items in Fig. 5A,

Y

1500

MD

1000 R

5oo

B

0

10

20

30

0

10

20

30

Y

g

1500

~ 1000 R

•~ O

0

B

Y

1500,

,

G

500

i !

lO00.

500

0

B

-

0

.

-

•

-

.

"'

10 20 30 Number ofltems

Fig. 4. Results of control for saturation in cases of parallel search for the moderately-saturated orange target (observer MD). Diagrams and graphs are interpreted as in Fig. 2. Search for the orange target among red and green distractors (A), yellow and blue distractors (B), and yellow-green and purple distractors (C) is conducted spatially in parallel. Error bars showing the standard error of the mean are smaller than the symbols in all cases. Percent correct rates in conditions A, B and C were 98, 97 and 97, respectively. See text for discussion.

958

MICHAEL D'ZMURA 1500

A

R R

1500

MD

1ooo •~ 500

500

ee

0

B

Y

10

20

30

1500

B G

0

R

~

10

20

30

10

20

30

1500 t

1000

== .-~ 500

B

0

o

500

0

10 20 30 Number of Items

ol 0

Number of Items

Fig. 5. Results for search for low saturation (A) and high saturation (B) orange targets among low saturation red and yellow targets (observers M D and PL). Diagrams and graphs are presented in the format of Fig. 2. Percent correct rates in conditions A and B for observers M D and PL, respectively, were (A) 95, 95 and (B) 95, 98. See text for discussion.

orange targets of high saturation presented among the less-saturated red and yellow distractors can be sought in parallel, as shown in Fig. 5B. Observers are presumably able, in the latter condition, to place a threshold on the response of a spatially-extended mechanism tuned to orange events that distinguishes target from distractors with sufficiently high accuracy. The result suggests that the slopes of the best-fit lines to target-present reaction times decrease monotonically as the saturation of the target, initially similar to the saturations of the distractors, is increased. Control for luminance contrast

It seemed very unlikely that mechanisms sensitive to black and white events played a significant role in these results with isoluminant items. A control experiment was nevertheless performed to dispose of the possibility that the results in the initial experiments were due to observers' use of a filter tuned to some differences in luminance between the orange target and pairs of blue, purple, red, yellow, yellow-green and green distractors. The initial experiments with orange targets were modified so that three types of distractor within each hue class were presented that had the same moderate saturation but straddled the isoluminant plane. Each distractor chromaticity was presented at 10% luminance contrast, at isoluminance, and at - 1 0 % luminance contrast with approxi-

mately equal frequency within a trial. As expected, results (see Fig. 6) follow the pattern found in the first experiment. If the parallel searches for orange in the first experiment depended on the detection of some brightness difference between target and distractors, then observers would not have been able to search in parallel under the corresponding conditions in the present control. The identity of results in both cases shows that the search results are mediated by color-sensitive mechanisms. Simultaneous search

A further implication of the interpretation pursued here is that one should be able to search in parallel simultaneously for targets with similar but distinct hues. Consider the search for an orange target in the presence of red and green distractors. If search is mediated by a mechanism tuned to yellow events, then it should be possible to show that orange and yellow targets presented singly in randomly interleaved trials among red and green distractors can be sought in parallel. This is the case, as shown in Fig. 7A: reaction times for the three types of trials--no target, yellow target and orange target--do not depend on the number of items. The results of a similar condition are shown in Fig. 7B: orange and red targets can be sought in parallel simultaneously among yellow and blue distractors, presumably through the use of

959

Color in visual search

1500 A

1:o

White

Green ~

Red

Black

B

i

MD

0 •

0

.

.

.

.

.

10

20

30

0

10

20

30

0

10

20

30

1500. White

glOOO .~ 500

0 Black

C

1500

wite

~1000 Oreeo

Red

500

Black

D

1500 White

~I000 Green

Black

0

0

10 20 30 Number of Items

Fig. 6. Results of search for a moderately-saturated orange target among similarly-saturated distractors which appear at three luminances spanning the isoluminant color plane (observer MD). The target color is represented in the color diagrams by the open circle; the distractors are represented by stacks of three solid circles extending in the white-black dimension• Search for the orange target among red and green distractors (A), yellow and blue distractors (B), the yellow-green and purple distractors (C) is conducted spatially in parallel; search among red and yellow distractors (D) is not. Graphs are interpreted as in Fig. 2. Percent correct rates in conditions A, B, C and D were 96, 98, 96 and 97, respectively. See text for discussion•

a filter tuned to red events. Results shown in Fig. 7C and D for one observer suggest, finally, that orange and red (C) or orange and yellow (D) targets can be sought simultaneously in parallel among yeUow-green and purple distractors, presumably through the use of a filter tuned to orange events.

Targets with other colors The results suggest that orange targets may be sought in a spatially-parallel fashion using

mechanisms tuned either to yellow events, to red events or to a chromatic combination of the two (orange events). The observers appear to use their expectations about the color appearances of target and distractors to select, adaptively, the chromatic properties of the filter attended under given circumstances, and this selection is consistent with the optimal choice that both ignores the noise and responds to the signal. The ability to search in parallel simultaneously for orange and red or orange and yellow targets,

960

MICHAEL D'ZMURA Y ~ 500

R

500

MD

~ 250 "

-

g~ o

o

B

10

0

PL

250

20

30

0

10

20

30

10

20

30

Y 500

R

~ 250

o

250

i 0

B

10

0

20

30

Number of Items

13

R

-J

0

B

R

0

10

20

30

i 250

.[ B

o

0

10 20 Number of Items

30

Fig. 7. Results of simultaneous search for two targets with different colors. The open circles and squares in the schematic color circles mark the colors of the targets while the solid circles mark the colors of the distractors. Open circles (squares) in the reaction-time plots refer to average reaction times for trials on which the target with color marked in the diagram by the open circle (square) was present. Solid circles in the reaction-time plots refer to average reaction times for trials on which no target was present. (A) Simultaneous search for orange and yellow targets among red and green distractors (observers MD and PL). (B) Simultaneous search for orange and red targets among yellow and blue distractors (observers MD and PL). (C) Simultaneous search for orange and red targets, and (D) for orange and yellow targets among yellow-green and purple distractors (observer MD alone). Nine trial types [(target 1, target 2, target absent) x (1, 14, 28 items)] were presented with equal frequency within four blocks of 45 trials; average reaction times are calculated from results from the latter three blocks. Percent correct rates in conditions A and B for observers MD and PL, respectively, were: (A) 97, 96; (B) 98, 98. Rates in conditions (C) and (D) for the single observer were 94 and 95, respectively. See text for discussion.

and the inability to search in parallel for an orange target among similarly-saturated red and yellow distractors suggest, furthermore, that such filters combine in an approximately linear way the responses of the cone mechanisms. A strong implication of a model comprising detection mechanisms sensitive to a variety of

hues is that this pattern of results can be found for targets of arbitrary chromaticity, provided that the distractors are altered appropriately. This was tested through experiments, like the initial one with the orange target, using red and yellow targets of moderate saturation (see Fig. 8). Experiments with red targets (Fig. 8A-D) and yellow targets (Fig. 8E-H) show that the

Color in visual search

A

y

,~o1~o

B

B

C

y

,~o]

961

B

y

,~]

D

B

v

,~o

B

Y e¢

~

O[ ~ 0

. 20

10

.

OI

30

500

. 30

20

30

Oo7_ 0

10

20

30

0

Number of Items

E

F

~5oo ]

o

10

20

0

10

°ol. 30

0

Numberof Items

y

,,o

10

10

20

y

G

,,o

,~o

5oo

5oo

30

~ 0

10

20

30

Number of Items

10

20

0

Number of Items

20

30

H

~

30

10

20

30

o

750

25o~

, ~ o ~

~o

20

30

Numberof Items

o

.

20

.

30

y

500

750

10

.

,,o

750

o

.

10

Number of Items

y

1. . . . 0

750

~

. 20

. . . .

i

s

OF --c . 0 10

500

,

'~

. ' ........ . I0 20 30

0

10

20

30

Number of Items

I0

20

30

10

20

30

Number of Items

Fig. 8. Results of search for unique-red (top) and unique-yellow (bottom) targets among various combinations of distractors. Results for observers MD and PL are Shown for the moderately-saturated unique-red target presented among (A) yellow and blue, (B) orange and blue-green, (C) yellow-green and purple and (D) orange and purple distractors of similar saturation. Results for observers MD and PL are shown for the moderately-saturated unique-Yellow target presented among (E) red and green, (F) orange and blue-green, (G) yellow-green and purple and (H) orange and yellow-green distractors of similar saturation. Diagrams and graphs are interpreted as in Fig. 2. Percent correct rates for MD and PL, respectively, were: (A) 98, 95; (B) 98, 98; (C) 95, 98; (D) 96, 98; (E) 95, 98; (F) 93, 98; (G) 93, 98; (H) 95, 98.

same pattern of results is found in all cases. The red targets are sought in parallel among orange (B) or purple (C) distractors but not among

both orange and purple distractors (D). The yellow targets are sought in parallel among orange (F) or yellow-green (G).distractors

962

MICHAEL D'ZMURA

but not among both orange and yellow-green distractors (H). These findings are not readily reconciled with explanations of detectability based solely on features provided by the standard coloropponent mechanisms. A red target has but a single color feature, namely red, yet it is not effectively masked either by orange (Fig. 8B) or purple (C) distractors that possess this feature. Spatially-extended detection mechanisms tuned to "purple" and "orange", respectively, can account for these parallel search results. Parallel search for the yellow target among orange (Fig. 8F) or yellow-green (G) distractors can, likewise, be accounted for by relying on spatially-extended mechanisms tuned to "yellow-green" and "orange", respectively. Note that the reaction times in the nonparallel search conditions are faster for the red and yellow targets than for the orange target (see also Fig. 2). A possible explanation for these overall faster reaction times is that the choices of chromaticities, although giving rise to similarity in perceived saturation, presented a higher signal-to-noise ratio to the optimal filters in the red- and yellow-target cases than in the orange-target case. This is seen to be plausible by drawing on the CIE chromaticity diagram of Fig. 1A lines that link the distractor pairs in the four cases, and noting the distance between these lines and the corresponding targets. I do not pursue this issue here, however. The aim has been to rely on a simple distinction between spatially parallel and non-parallel search to probe the organization of higher-level chromatic mechanisms used in detection tasks. DISCUSSION

Observers can base their decisions in a search task on the response of a spatially-extended chromatic detection mechanism with a spectral sensitivity that best distinguishes the target from the distractors. The use of mechanisms with distinct spectral sensitivities to detect a single target in a way that depends on the chromaticities of distractors is analogous to "off-frequency listening" phenomena in audition (Patterson & Nimmo-Smith, 1980) and to "off-frequency looking" reported by Pelli (1981); this result confirms earlier findings with a heterochromatic noise masking technique (D'Zmura et al., 1987). The present results show further that the spectral sensitivity of the attended mechanism need not correspond to

the sensitivity of a standard color-opponent mechanism. Rather, color mechanisms for parallel search, taken as a whole, form a more fine-grained representation of hue within the central visual field. The data suggest that these mechanisms are chromatically linear and that the chromaticallyoptimal mechanism mediates parallel search if, its local signal-to-noise ratio, measured across trials at all locations, is sufficiently high. According to this tentative interpretation, observers do not rely on this bank of mechanisms if it cannot accurately distinguish target from distractors, rather they restrict in space and in wavelength the receptive field of the detector on which they base their decisions. Narrowband chromatic filters of restricted spatial extent must be swept from item to item; their use is signalled by an increase in reaction time with number of displayed items.

Adaptive discrimination The results that implicate mechanisms tuned to intermediate hues in the performance of visual search tasks suggest that two wellregarded explanations of performance in colorbased visual search are, at best, incomplete. The first of these explanations relies on similarity or "simple discriminability". Target-distractor similarity plays a role in the work of many (Green & Anderson, 1956; Smith, 1962; Farmer & Taylor, 1980; Carter, 1982; Nagy & Sanchez, 1988; Duncan & Humphreys, 1989), and while it is certainly true that the detectability of a target depends on distractors according to their chromatic "distance" from one another, simple discriminability fails to provide for the adaptive nature of search. For instance, it is hard to see how two distractors that are not similar to a target (e.g. the red and yellow distractors of Fig. 2A and B) can suddenly be similar when used together (Fig. 2D). An adaptive model of discriminability like that suggested by Wandell (1982) provides a simple and more mechanistic account: similarities among a set of chromaticities must be measured with respect to the mechanism used in the detection task. Orange targets and red distractors are easily discriminated by a detection mechanism tuned to yellow events; likewise, orange targets and yellow distractors are highly discriminable to a detection mechanism tuned to red events. However, orange targets prove to be similar to both red and yellow distractors when detection is mediated by a chromatically-

Color in visual search broadband mechanism tuned to orange events. In this emendation to "simple discriminability", the similarity between two chromaticities is no longer a single scalar value but a function of the spectral sensitivity of particular detection mechanisms. Knowledge of single mechanisms' response properties and of how such a mechanism may be selected through attention or driven by visual information are prerequisites to quantifying further the use of chromatic information in visual search tasks.

Color features A second explanation of color-based search, developed by Treisman and her colleagues, relies on the responses of a small set of prototypical color feature maps, for instance, spatially-extended banks of detection mechanisms tuned to red, yellow, green and blue events (Treisman, 1985; Treisman & Gormican, 1988). While such an organization of chromatic mechanisms provides a plausible account of color appearance (Hurvich & Jameson, 1957) and identification (Treisman & Schmidt, 1982), its application to visual search tasks is not clear. The thrust of the present results is that we possess many further color-sensitive mechanisms that can mediate spatially-parallel search. The notions of feature and conjunction break down under these conditions. Although an orange target, for instance, appears as a conjunction of red and yellow, search is nevertheless conducted in parallel when the target is presented among distractors that share the red and yellow features (Figs 2C and 4C). And while it is true that red and yellow distractors make search for the orange "conjunction" difficult, the. notion is perhaps less appropriate in the case where violet and orange distractors make search for the red target difficult. We were, furthermore, unable to replicate the search asymmetry for colored items reported by Treisman and Gormican (1988). Reaction times in seach for a red target among magenta distractors, for instance, were found by Treisman and Gormican to increase mildly with number of distractors and to differ significantly from reaction times in search for a magenta target among red distractors. This was taken to agree with stronger results in experiments, not dealing with color, that showed search for targets defined by the absence of a feature to be more difficult than search for targets with an additional, "deviating" feature. Treisman and Gormican (1988) recognized the very small magnitude of their VR 31/6--D

963

effect with colored items; the effect is completely absent in the present results. Searches for a red target among either orange or purple distractors (Fig. 8B and C, respectively) or for a yellow target among yellow-green or orange distractors (Fig. 8F and G, respectively) were conducted spatially in parallel. A number of methodological differences might account for the small discrepancy between the two sets of results; these include differences in task familiarity, chromatic uncertainty, and choice of item chromaticities. That observers possess mechanisms tuned to intermediate hues which can be used to detect unique-hue targets under appropriate circumstances remains a plausible account of the results.

Detection mechanism bandwidths Inferences discussed above regarding the chromatic bandwidths and spatial extents of chromatic detection mechanisms are probably of limited applicability. For instance, small and moderate chromatic contrasts in a steady state of light adaptation were used in the present experiments so that low-level chromatic mechanisms would more likely operate linearly, and the findings almost certainly depend on this choice. One cannot suppose, for instance, that the present account describes the detection of highly-saturated lights presented on a black background; one would certainly expect nonlinearities other than rectification to complicate matters. Indeed, the highly-saturated disks used in these experiments (see Fig. 5) sometimes appeared to fluoresce rather than to appear as surfaces that are not optically active. Although it is difficult to say whether "fluorence" (Evans, 1959) acts as a primitive feature that could allow a target to be sought in parallel among nonfluorent items, such effects (e.g. Wolfe & Franzel's, 1988, spatially-parallel search for binocular luster) certainly lie outside the scope of the present account. The idea that spatially-parallel search mechanisms combine in an approximately linear fashion signals from cones and that chromatically-nonlinear mechanisms must be applied to displays in a serial fashion has a second problem: the implicit reliance on the unexamined notion of signal-to-noise ratio. For instance, while the present results suggest that observers attend to the output of a chromatically linear mechanism if a sufficiently high signal-to-noise ratio obtains in its response (so that a threshold separates adequately target from distractor

964

MICHAEL D'ZMURA

chromaticities), the results provide no principled way to determine what is meant by sufficiently high. As suggested in the earlier discussion of similarity, a firmer picture of detection mechanism bandwidths and spatial extents depends on detailed measurements of the response sensitivities of individual mechanisms. On the other hand, the results suggest that there is nothing special about unique and binary hues at perceptually-matched saturations and brightnesses. The presence or absence of parallel search is consistent with explanations in terms of item chromaticity (Fig. 1A) rather than of item color (Fig. 1B). The situation resembles that in which Nakayama and Silverman (1986) found themselves in explaining parallel search for items defined by stereo and other cues. They could not distinguish direct use of retinal disparity information from the use of apparent depth information.

Multiple mechanism organization The relationship between the chromatic detection mechanisms suggested by these experiments and the standard color-opponent mechanisms is unknown, so affording considerable room for speculation. A first point is that there is little reason to replace the standard color-opponent mechanisms with a multiple-mechanism organization. Red-green and yellow-blue mechanisms provide a plausible, parsimonious account of much data on color appearance. The issue is at what stage(s) in color processing the multiple mechanisms are elaborated. One possibility is that the multiple mechanisms do not exist simultaneously, rather observers create from the responses of the "secondstage" color-opponent mechanisms a single "soft-wired" mechanism as the need arises. An appropriate linear combination of "red" and "yellow" mechanism sensitivities formed over the central visual field can provide, for instance, a detection mechanism useful in parallel search with a peak sensitivity to orange events. The observer would, in this view, create such a "third-stage" chromatic filter and set a threshold on its input to localization and decision-making mechanisms in a way that depends on the observer's goals and the expected color appearance of stimuli. Yet there is experimental evidence that supports a "hard-wired" alternative, in which the multiple mechanisms are ready and waiting

(D'Zmura & Lennie, 1986). Unprepared observers report that orange targets pop out among yellow-green and purple distractors, which suggests that an "orange" detection mechanism can operate independently of "top-down" observer strategies. The chromatic contrast habituation results of Krauskopf and his colleagues also suport this view (Krauskopf et al., 1982, 1986), although there is yet no reason to identify the intermediate-direction mechanisms revealed by these very different experiments. Physiological results, too, support the hard-wired alternative. The distribution of the peak spectral sensitivities of nonoriented, color-sensitive cells in macaque visual area V1 is far more uniform than that found in macaque lateral geniculate nucleus (Lennie et al., 1990), which suggests that the multiple-mechanism organization is first elaborated at an early stage in the cortical processing of color. The peak spectral sensitivities of color-sensitive neurons in macaque V4 are also uniformly scattered in wavelength (Schein & Desimone, 1990). The electrophysiological study of state-dependent activity in cortical neurons of alert macaque monkeys by Haenny and Schiller (1988) suggests that color-sensitive neurons in V4 (but probably not V1) may play a direct role in mediating visual attention to events of a particular color. Hard-wired multiple mechanisms can presumably be driven in a "bottom-up" fashion by chromatic information, so allowing unattended pop out. The soft-wired, cognitive component to color detection tasks more likely resides at a higher level: a single detection mechanism can presumably be selected by an observer in a top-down fashion. By routing selected chromatic information to localization, decisionmaking and other resources, the activation of a single such mechanism has a profound effect on performance, yet seems to have little effect on color appearance. Both selected and ignored chromatic information appear, which suggests that there is some degree of parallelism in the operation of the standard color-opponent mechanisms and the multiple mechanisms for color signal detection. Chromatic contrast habituation not only reduces color signal detectability (Krauskopf et al., 1982, 1986) but also affects color appearance, primarily by reducing perceived saturation. This observation lets us tentatively place habituating mechanisms in a pathway common to both detection and appearance mechanisms.

Color in visual search

Acknowledgements--I thank AI Ahumada, Mary Hayhoe, John Krauskopf, Peter Lennie and Donald MacLeod for helpful discussions. This work was supported by NEI Grants EY0440 and EY01319.

REFERENCES

Ayama, M., Nakatsue, T. & Kaiser, P. K. (1987). Constant hue loci of unique and binary balanced hues at 10, 100, and 1000 td. Journal of the Optical Society of America A, 4, 1136-1144. Burns, S. A., Eisner, A. E., Pokorny, J. & Smith, V. C. (1984). The Abney effect: Chromaticity coordinates of unique and other constant hues. Vision Research, 24, 479-489. Carter, R. C. (1982). Visual search with color. Journal

of Experimental Psychology: Human Perception and Performance, 8, 127-136. Crane, H. D. & Piantanida, T. P. (1983). On seeing reddish green and yellowish blue. Science, 221, 1078-1080. Derrington, A. M., Krauskopf, J. & Lennie, P. (1984). Chromatic mechanisms in lateral geniculate nucleus of macaque. Journal of Physiology, London, 357, 241-265. DeValois, R. L. (1965). Behavioral and electrophysiological studies of primate vision. In Neff, W. D. (Ed.), Contributions to sensory physiology (Vol. 1, pp. 137-138). New York: Academic. Dimmick, F. L. & Hubbard, M. R. (1939a). The spectral location of psychologically unique yellow, green and blue. American Journal of Psychology, 52, 242-254. Dimmick, F. L. & Hubbard, M. R. (1939b). The spectral components of psychologically unique red. American Journal of Psychology, 52, 348-353. Duncan, J. & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Reviews, 96, 433-458. D'Zmura, M. (1990). Surface color psychophysics. Ph.D. dissertation, University of Rochester. D'Zmura, M. & Lennie, P. (1986). Mechanisms of color constancy. Journal of Optical Society of America A, 3, 1662-1672. D'Zmura, M., Krauskopf, J. & Lennie, P. (1987). Hue selectivity revealed by heterochromatic noise masking.

Investigative Ophthalmology and Visual Science, 28 (Suppl.), 92. D'Zmura, M., Moller, P. & Lennie, P. (1988). Attentional selection of chromatic mechanisms. Investigative Ophthalmology and Visual Science, 28 (Suppl.), 162. Egeth, H. E., Virzi, R. A. & Garbart, H. (1984). Searching for conjunctively defined targets. Journal of Experimental

Psychology: Human Perception and Performance, 10, 32-39. Evans, R. M. (1959). Fluorescence and gray content of surface colors. Journal of the Optical Society of America, 49, 1049-1059. Farmer, E. W. & Taylor, R. M. (1980). Visual search through color displays: Effects of target-background similarity and background uniformity. Perception and Psychophysics, 27, 267-272. Green, B. F. & Anderson, L. K. (1956). Color coding in a visual search task. Journal of Experimental Psychology, 51, 19-24. Haenny, P. E. & Schiller, P. H. (1988). State dependent activity in monkey visual cortex--I. Single cell activity in V1 and V4 on visual tasks. Experimental Brain Research, 69, 225-244.

965

Helmholtz, H. von (1909/1962). Southall, J. P. C. (Ed.), Treatise on physiological optics (Vol. II). New York: Dover. Hering, E. (1878/1964). Outlines of a theory of the light sense (Translated by Hurvich, L. M. & Jameson, D.). Cambridge: Harvard University Press. Hurvich, L. M. & Jameson, D. (1955). Some quantitative aspects of an opponent colors theory--II. Brightness, saturation and hue in normal and dichromatic vision. Journal of the Optical Society of America, 45, 602-616. Hurvich, L. M. & Jameson, D. 0957). An opponent-process theory of color vision. Psychological Reviews, 64, 384-404. Jameson, D. & Hurvich, L. M. (1955). Some quantitative aspects of an opponent-colors theory--I. Chromatic responses and spectral saturation. Journal of the Optical Society of America, 45, 546-552. Judd, D. B. (1949). Response functions for types of vision according to the Muller theory. Journal of Research NBS, RP1946, 42, 1-16. Kranda, K. & King-Smith, P. E. (1979). Detection of coloured stimuli by independent linear systems. Vision Research, 19, 733-745. Krauskopf, J., Williams, D. R. & Heeley, D. W. (1982). Cardinal direction of color space. Vision Research, 22, 1123-1131. Krauskopf, J., Williams, D. R., Mandler, M. B. & Brown, A. M. (1986). Higher order color mechanisms. Vision Research, 26, 23-32. Latimer, J., Krantz, D. H. & Cicerone, C. M. (1974). Opponent-process additivity--I. Red-green equilibria. Vision Research, 14, 1127-1140. Latimer, J., Krantz, D. H. & Cicerone, C. M. (1975). Opponent-process additivity--II. Yellow/blue equilibria and nonlinear models. Vision Research, 15, 723-731. Lennie, P., Krauskopf, J. & Solar, G. (1990). Chromatic mechanisms in striate cortex of macaque. Journal of Neuroscience, I0, 649-669. Nagy, A. L. & Sanchez, R. R. (1988). Large color differences measured with a search task. Investigative Ophthalmology and Visual Science, 29 (Suppl.), 407. Nakayama, K. & Silverman, G. H. (1986). Serial and parallel processing of visual feature conjunctions. Nature, London, 320, 264-265. Pashler, H. (1987). Detecting conjunctions of color and form: reassessing the serial search hypothesis. Perception and Psychophysics, 41, 191-201. Patterson, R. D. & Nimmo-Smith, I. (1980). Off-frequency listening and auditory-filter asymmetry. Journal of the Acoustic Society of America, 67, 229-245. Pelli, D. G. (1981). Effects of visual noise, Ph.D. dissertation, Cambridge University. Schein, S. J. & Desimone, R. (1990). Spectral properties of V4 neurons in macaque and their relationship to a physiological model of color vision, preprint. Smith, S. L. (1962). Color coding and visual search. Journal of Experimental Psychology, 64, 434-440. Sternberg, S. (1966). High-speed scanning in human memory. Science, 153, 652-654. Stromeyer, C. F. III., Cole, G. R. & Kronauer, R. E. (1985). Second-site adaptation in the red-green chromatic pathways. Vision Research, 25, 219-237. Svaetichin, G. & MacNichol, E. F. (1958). Retinal mechanisms for chromatic and achromatic vision. Annals of the New York Academy of Science, 74, 385-397.

966

MICHAELD'ZMURA

Treisman, A. (1985). Preattentive processing in vision. CVGIP, 31, 156-177. Treisman, A. M. & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12, 97-136. Treisman, A. M. & Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Reviews, 95, 15-48. Treisman, A. & Schmidt, H. (1982). Illusory conjunctions in the perception of objects. Cognitive Psychology, 14, 107-141.

Treisman, A., Sykes, M. & Gelade, G. (1977). Selective attention and stimulus integration. In Dornic, S. (Ed.), Attention and performance VI (pp. 333-361). Hillsdale, N J: Erlbaum. Wandell, B. A. (1982). Measurement of small color differences. Psychological Reviews, 289, 281-302. Wolfe, J. M. & Franzel, S. L. (1988). Binocularity and visual search. Perception and Psyehophysics, 44, 81-93. Young, T. (1802). On the theory of light and colours. Philosophical Transactions, 12-48.