Neuropsychologia 72 (2015) 43–51

Contents lists available at ScienceDirect

Neuropsychologia journal homepage: www.elsevier.com/locate/neuropsychologia

ERP evidence for the influence of scene context on the recognition of ambiguous and unambiguous objects Melissa Dyck a, Mathieu B. Brodeur b,c,n a

Integrated Program in Neuroscience, Faculty of Medicine, McGill University, Montréal, Canada Department of Psychiatry, McGill University, Montréal, Canada c Douglas Mental Health University Institute, Montréal, Canada b

art ic l e i nf o

a b s t r a c t

Article history: Received 30 September 2014 Received in revised form 30 March 2015 Accepted 21 April 2015 Available online 22 April 2015

We are used to seeing objects in specific settings, and in association with other related objects. This contextual information allows for fast and efficient object recognition and influences brain-related processes. The influence of scene context has been studied using event-related potentials (ERPs) in order to further our understanding of the underlying brain mechanisms. Current ERP studies have focused on effects related to the incongruity between unambiguous objects and their scenes, rather than the specific influence of a congruent scene. The present study sought to examine ERPs associated with the beneficial influence of scene context on object recognition. This influence was examined using ambiguous objects that required a congruent scene in order to be recognized, as well as unambiguous objects, to determine whether scene processing occurs even when it is unnecessary for recognizing the object. Twenty healthy subjects were instructed to indicate whether they recognized, had a vague idea, or did not recognize target objects that appeared within congruent and neutral scenes. ERPs from 250 to 1000 ms, including the N300 and N400, were more positive at anterior sites and more negative at posterior sites, when objects appeared in congruent scenes as opposed to when they appeared in neutral scenes, with a larger effect seen for ambiguous objects. Upon further examination, the results showed that the ERPs to ambiguous objects became similar to those of unambiguous objects when they appeared in congruent contexts. These findings indicated that a congruent context exerted its influence by reducing the ambiguity of objects. & 2015 Elsevier Ltd. All rights reserved.

Keywords: Object recognition ERPs Scene context Visual ambiguity

1. Introduction Previous literature has demonstrated that scene context facilitates object recognition. Objects perceived in appropriate or congruent scenes are recognized more quickly and accurately than objects perceived in inappropriate or incongruent scenes (Biederman, 1972; Oliva and Torralba, 2007). Event related potential (ERP) differences between conditions of congruity further demonstrate that context influences object recognition processing (Ganis and Kutas, 2003; Demiral et al., 2012; Mudrik et al., 2010, 2014). However, the lack of a neutral condition limits the conclusions that can be made about this influence. Congruent scenes may facilitate the recognition of objects; whereas, incongruent scenes could impede recognition and account for the brain n Corresponding author at: Douglas Mental Health University Institute, FBC Pavilion, 6875 Boulevard LaSalle, Verdun, Québec, Canada H4H 1R3. Fax: þ1 514 888 4064. E-mail address: [email protected] (M.B. Brodeur).

http://dx.doi.org/10.1016/j.neuropsychologia.2015.04.023 0028-3932/& 2015 Elsevier Ltd. All rights reserved.

response differences observed between the two conditions of congruity. ERP studies have demonstrated that ERP context effects are highly similar to the well-known N400 (Demiral et al., 2012; Mudrik et al., 2010; Ganis and Kutas, 2003; Goto et al., 2009; Võ and Wolfe, 2013), which is associated with semantic incongruity between words, between words and sentences and between incongruent pictures (Kutas and Federmeier, 2011; Holcomb and McPherson, 1994). For instance, Mudrik et al. (2010) found that incongruent stimuli resulted in ERPs that were more negative over central-frontal electrodes during the 300–500 ms time window. Similarly, Ganis and Kutas (2003) found a greater N390 for objects presented within incongruent scenes as compared to objects presented within congruent scenes. Consequently, brain response differences between congruent and incongruent scenes are likely due to a monitoring of incongruity rather than a facilitative influence of context. To study the facilitative influence of a scene on object recognition, it is important to consider how much a scene is needed for the object to be recognized and how much the scene can help

44

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

in recognizing the object. Objects do not systematically require scene context to be recognized. For instance, the functional isolation model suggests that bottom up visual analysis is sufficient for object identification and that scene and object processing are independent when response bias and search strategy are controlled for (Henderson and Hollingworth, 1999; Hollingworth and Henderson, 1998). Scene and object processing may interact, but the use of objects that are recognized too easily may minimize the requirement of context, and result in the assumption that scene context does not facilitate the recognition of objects. In support of this assumption, Palmer (1975) reported a subgroup of objects that were so easy to recognize that they were equally identifiable regardless of scene congruity. In some circumstances, scenes can even be distracting, regardless of their congruity, affecting accuracy and reducing the speed at which subjects recognize objects (Davenport and Potter, 2004). When an object is difficult to recognize, a scene must contain information that is congruent with the object in order to help determine its identity. For example, ambiguous objects are recognized more readily when presented with contextually relevant objects (Bar and Ullman, 1996). In his model on the influence of context, Bar (2004) suggests that context frames are processed systematically and quickly prior to the identification of an object, in order to narrow the range of guesses that can be made about an object's identity. Without contextual information, guesses depend solely on how recognizable the object is. The brain responses thought to reflect those guesses, or what Stuss et al. (1992) called the generation of hypotheses of an object's identity, are reflected by more negative ERPs in time windows of 250–350 ms and 350– 450 ms (Doniger et al., 2001; Stuss et al., 1992). This effect was demonstrated by using fragmented images that, at a certain level of fragmentation, became unrecognizable. Specifically, the less complete the image was, the more negative the ERP was, because more possibilities regarding the objects identity were generated (Stuss et al., 1992; Viggiano and Kutas, 1998, 2000). However, if those fragmented or ambiguous stimuli were presented within congruent contexts, it could be predicted that fewer hypotheses would be generated, resulting in reduced ERP responses. Despite a large body of literature, the beneficial effect of scene context on object recognition remains unknown. The majority of studies to date have used unambiguous objects or conditions of incongruity. In this study, ambiguous and unambiguous target objects were presented within congruent and neutral scenes. By combining conditions of stimulus ambiguity with scene context congruity, the beneficial influence of context will be examined as a function of how much an object required a scene to be recognized. We will address whether a congruent scene context can disambiguate an ambiguous object, and examine the resulting brain activities using ERPs. To avoid confounding our effects with the

monitoring of incongruity, we have used neutral scene contexts as opposed to the incongruent scenes that were used in previous studies. In the neutral scenes, the target objects were all possible components within the physical space of the scene, although they were not expected or probable. Seeing these objects was not surprising or disturbing in the sense that they did not prompt a reinterpretation of the scene. By comparison, the objects in the congruent contexts were probable components of the scene and were expected where they appeared in the scene.

2. Methods 2.1. Subjects Twenty right-handed healthy subjects (female: 12) were recruited for the project and signed informed consent, approved by the Research Ethics Board of the Douglas Institute. Subjects were recruited by placing ads on Kijiji and Craigslist. Subjects were between the ages of 20 and 35 (mean: 25.9, SD: 5.0), with no color vision defect. 2.2. Stimuli The stimuli consisted of 360 photos of scenes, each associated with one of 360 target objects. As shown in Fig. 1, three versions of the stimuli were created for our stimulus presentation procedure. Scenes were first photographed with a digital camera, without the target. Adding a fixation cross at the location where the target object would appear created a second version of the stimulus. The third version of the stimulus included the target object, in place of the fixation cross. To ensure that the target object reflected the same lighting conditions of the scene, it was photographed separately in the same environment, cut out from its original photo and pasted onto the version of the image with the fixation cross. The photos were distributed across two conditions, each with two levels; therefore, 90 trials per level (Fig. 2). First, the target objects were either ambiguous or unambiguous and second, the scenes were either congruent or neutral with respect to the target objects. Ambiguity was defined as an object being mostly unrecognizable when presented alone over a white background. Congruity was defined as an object being probable in the scene. Congruent scenes allowed for an ambiguous object to be recognized. Neutral scenes were considered neutral rather than incongruent, as they did not facilitate recognition, but they also did not impede recognition by creating a conflict between the scene and the object. Target objects were the appropriate size for their given scene in all conditions. Prior to the experiment, the ambiguity of target objects was

Fig. 1. The three versions of the stimuli used in each trial. Left: a congruent scene; Middle: a congruent scene with a fixation cross indicating the location where the target object will appear; Right: a congruent scene with an ambiguous target object. In each trial, the scene was displayed for 1.6 s and the cross and target object were each displayed for 1 s.

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

45

Fig. 2. Examples of stimuli used in the experiment. Top left: an ambiguous object (computer speaker) placed in a neutral scene; Top right: an ambiguous object (weight) placed in a congruent scene; Bottom left: an unambiguous (banana) object placed in a neutral scene; Bottom right: an unambiguous object (electric razor) placed in a congruent scene.

validated by presenting the objects to 15 pilot volunteers and asking them to indicate whether they recognized the object, had a vague idea, or did not recognize the object. These responses were assigned a “recognition value” of 2, 1, or 0, respectively, and the mean response value was computed for each object. Those with a mean recognition value below or equal to 1 were considered ambiguous and those with a mean recognition value over 1 were considered unambiguous. An additional 15 volunteers were recruited to complete the same recognition task; however, the objects were presented in a scene. When the recognition score of an ambiguous object was over 1 in this new experiment, the scene was considered congruent because it brought the object to a level where it was now recognizable. The mean recognition score of ambiguous objects increased from .43 (SD: .28, min: .00, max: .91) without context to 1.73 (SD: .25, min: 1.20, max: 2.00) when presented in a congruent context. No such increase was found for ambiguous objects presented in neutral scenes, which had a mean recognition score of .43 (SD: .25, min: .00, max: .92) without context and .65 (SD: .25, min: .01, max: 1.00) when presented in a neutral context. Unambiguous objects were not significantly influenced by context. A total of 498 objects were validated, and the 360 objects with the strongest context effects were used in the experiment. 2.3. Procedure Subjects were seated in a soundproof, dimly lit, electrically shielded room at the Brain Imaging Centre of the Douglas Institute, in front of a computer screen used for stimulus presentation. A practice session was completed prior to the full 30-min task to familiarize subjects with the procedure. The task consisted of a blank screen that was presented for 1 s, followed by a scene for 1.6 s. Two stimuli then appeared over the scene, one after the other, each for 1 s; a fixation cross and the target object, respectively (Fig. 1). Following the appearance of the target, subjects were asked to respond whether they recognized the object, had a vague idea, or did not recognize the object by pressing one of three keys on a keyboard. The response “vague idea” was included as a response option in order to be able to separate trials in which recognition was more or less effective and to ensure recognized trials were indeed recognized. Stimulus presentation was monitored by E-Prime software 2.0 (Psychology Software Tools, Inc.).

2.4. ERP recording The EEG was recorded from 62 Ag/AgCl ActiCap electrodes attached to an elastic cap, placed according to the 10–20 system (American Electrophysiological Society, 1994). One additional electrode was placed below the right eye to monitor ocular movements. The impedance was reduced to below 5 kΩ. The electrode Fcz was used as the reference during the recording. The sampling rate was set to 1000 Hz and the EEG was amplified 20,000 times using a BrainAmp DC amplifier (Brain Products Inc.). The recording was monitored using Brain Vision Recorder (Brain Products Inc.) software. To minimize muscular and ocular artifacts, subjects were instructed to stay as still as possible during the recording and to refrain from blinking during the target appearance. 2.5. Behavioral analysis A 2  2 repeated measures ANOVA was conducted on the number of recognized trials with within subject variables of ambiguity (unambiguous, ambiguous) and context (congruent, neutral). The reaction times (RT) for recognized trials were analyzed using the same ANOVA. The response indicating that the subjects recognized the object was the only response that guaranteed that recognition processes were fully activated. It was therefore the only response included in the analyses. Accurate recognition was not taken into account, as even if an object was incorrectly identified, recognition processes would still have been activated. Conditions in which the target object was unambiguous, or when ambiguous objects were presented in congruent contexts yielded very few unrecognized trials. Therefore, unrecognized trials were excluded from the analyses. The frequency of vague idea responses varied considerably between subjects, with some subjects never giving the response; therefore, this response was also excluded from the analyses. 2.6. ERP analysis ERPs were computed and analyzed offline using Brain Vision Analyzer (Brain Products Inc.). High and low pass filters were set to cut-offs of .01 Hz and 100 Hz (12 dB/oct), respectively. The data was re-referenced offline to an average of all electrodes. Independent component analysis was used to remove eye blinks and

46

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

ocular movements. Sections of the VEOG channel with strong variations in gradient were detected and removed. As a result, 6.4% (SD: 7.6%) of trials were removed due to such artifacts. Segments from  200 to 1000 ms, relative to target object onset, were extracted from the continuous EEG recording. Brain responses to recognition were measured in time windows of 250–350 ms, 350– 500 ms and 500–1000 ms to measure activity in the N300, N400 and late recognition effects, respectively. Voltage amplitudes were measured from a baseline of  200 to 0 ms relative to the appearance of the target object. Electrodes were grouped into four regions per hemisphere (frontal: Fp1/2, F1/2, F3/4, F5/6, F7/8, Af3/ 4, temporal: Ft7/8, Fc5/6, C5/6, Cp5/6, T7/8, Tp7/8, Tp9/10, central: Fc1/2, Fc3/4, C1/2, C3/4, Cp1/2, Cp3/4, and posterior: P1/2, P3/4, P5/6, P7/8, Po3/4, Po7/8, Po9/10, O1/2), as well as a single midline region (Fcz, Fz, Cz, Cpz, Pz, Poz, Oz) to cover the whole scalp. The statistical analysis was conducted using repeated measures ANOVAs on the mean voltage amplitude of ERPs in the aforementioned time windows. The effect of context and the effect of ambiguity were analyzed using ANOVAs with 5 within subject variables: ambiguity (unambiguous, ambiguous); context (congruent, neutral); region (frontal, central, temporal, posterior); hemiscalp (right, left); and time window (N300, N400, and late window). A separate ANOVA with no hemiscalp variable was conducted for the midline electrode region. Additional ANOVAs were conducted to test the differences over P1 and N1. A short 20 ms time window centered on the peak of the P1 (105 ms) and N1 (170 ms) were used to measure a mean voltage for each of these deflections. Measures were taken on electrodes Po7 and Po8, where the greatest amplitudes for these deflections were found in previous studies. Violation of the assumption of sphericity was corrected using the Greenhouse–Geisser adjustments to degrees of freedom. The context effect was examined by comparing the mean voltage amplitudes for ambiguous and unambiguous objects placed in congruent and neutral scenes. The ambiguity effect was examined by comparing the difference in mean voltage amplitudes for ambiguous and unambiguous objects presented in congruent and neutral scenes. The mean number of trials included in the analysis, per condition, was as follows: ambiguous/congruent (68.9), ambiguous/neutral (32.4), unambiguous/congruent (76.8), unambiguous/neutral (70.3). All subjects completed a minimum of 20 trials in each condition. Fewer trials were available for analysis in the ambiguous/neutral condition. This was a limitation of our study design, as ambiguous objects in neutral scenes were harder to recognize and subjects were less likely to respond “recognize” and more likely to respond “did not recognize” or “vague idea” for these trials.

Fig. 3. Summary of the number of responses and error bars per condition.

Fig. 4. Summary of reaction time for recognized objects and error bars per condition.

conducted on objects presented in neutral scenes showed an effect of ambiguity, in that unambiguous objects in neutral scenes were recognized faster than ambiguous objects (t(19) ¼6.145, p o.001). The effect of context was also significant (F(1,19) ¼63.763, po .001) and present for both unambiguous and ambiguous objects. Unambiguous (t(19) ¼  4.871, p o.001) and ambiguous (t (19) ¼7.319, p o.001) objects were recognized faster when placed in congruent scenes compared to neutral scenes, and this effect was stronger for ambiguous objects, as indicated by the significant context  ambiguity interaction. 3.2. Event-related potentials in time windows

3. Results 3.1. Behavioral data Responses and reaction times are presented in Figs. 3 and 4, respectively. There was a context  ambiguity interaction (F (1,19) ¼ 110.682, p o.001) on the number of trials recognized. The effect of ambiguity for objects presented in neutral scenes was significant (t(19) ¼ 13.421, po .001); unambiguous objects were recognized more often than ambiguous objects. An effect of context was also found (F(1,19) ¼ 189.201, p o.001) and this effect was significant when tested separately for unambiguous (t(19) ¼3.142, p ¼.005) and ambiguous (t(19) ¼ 13.052, p o.001) objects. The context  ambiguity interaction above indicates that the effect of context was significantly greater for ambiguous objects. Analyses on RTs for recognized trials revealed a context  ambiguity interaction (F(1.19) ¼31.516, p o.001). The t-tests

Ambiguity and context significantly interacted with electrode region (F(3,57) ¼6.909, p¼ .008), and with electrode region and hemiscalp (F(3,57) ¼2.994, p ¼.045). The ambiguity effect, the context effect and the interaction of these two variables were therefore examined separately in each electrode region. Statistical results are presented in Table 1. 3.2.1. Ambiguity effect The ambiguity effect reflects the voltage difference between ambiguous and unambiguous objects. It consists of more negative voltages in the fronto-central areas, and more positive voltages in the posterior areas for ambiguous objects relative to unambiguous objects. The main effect of ambiguity was highly significant in all regions (see statistic #8 in Table 1) and it interacted with the hemiscalp and time window variables in the central region (stats #5) and with the time window in temporal and midline regions (stats #7).

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

47

Table 1 Statistical results. Stats # ANOVA

Frontal

Ambiguity  context effect 1 Ambiguity  context  hemiscalp  time window n.s. 2 Ambiguity  context  hemiscalp F(1,19) ¼5.304, p ¼.033 3 Ambiguity  context  time window n.s. 4 Ambiguity  context F(1,19) ¼6.621, p ¼.019 Ambiguity effect 5 Ambiguity  hemiscalp  time window

n.s.

6

Ambiguity  hemiscalp

n.s.

7

Ambiguity  time window

n.s.

8

Ambiguity

8a

–Ambiguity effect in neutral scenes

8b

–Ambiguity effect in congruent scenes

F(1,19) ¼11.540, p ¼.003 F(1,19) ¼11.314, p ¼.003 n.s.

Context effect 9 Context  hemiscalp  time window 10 Context  hemiscalp 11 12

Context  time window Context

12a

–Context effect on unambiguous objects

12b

–Context effect on ambiguous objects

a b c

n.s. n.s.

Central

Temporal

Posterior

Midline

n.s. n.s.

n.s. n.s.

n.s. n.s.

NAa NAa

n.s. F(1,19) ¼ 5.733, p ¼ .027

n.s. n.s.

n.s. F(1,19)¼ 7.982, p¼ .011

n.s. n.s.

F(2,38) ¼7.124, p ¼ .003 F(2,38) ¼10.232, p ¼ .005 F(2,38) ¼3.825, p ¼ .047 F(1,19) ¼ 15.220, p ¼ .001 F(1,19) ¼ 11.451, p ¼ .003 F(1,19) ¼ 11.482, p ¼ .003

n.s.

n.s.

NAa

n.s.

n.s.

NAa

F(2,38) ¼ 9.183, p ¼.001 F(1,19) ¼6.073, p o.001 NAb

n.s.

F(2,38) ¼ 7.889, p ¼.003 F(1,19) ¼24.323, p o.001 NAb

n.s. F(1,19) ¼17.575, p o.001 n.s.

n.s. F(1,19) ¼ 7.098, p ¼ .015 n.s. F(1,19) ¼ 7.060, p ¼ .016 n.s.

F(1,19) ¼24.708, p o.001

F(1,19) ¼ 9.127, p ¼ .007

NAb

n.s. F(1,19) ¼5.934, p ¼.025 n.s. n.s. NAb NAb

F(1,19)¼ 17.095, p¼ .001 F(1,19)¼ 14.689, po .001 F(1,19)¼ 4.432, p¼ .049

NAb

n.s. n.s.

NAa NAa

n.s. F(1,19)¼ 26.201, po .001 F(1,19)¼ 4.187, p¼ .055c F(1,19)¼ 23.770, po .001

n.s. n.s. NAb NAb

There was no hemiscalp variable in the midline subset of electrodes. The effect was not statistically tested because the ambiguity  context was not significant. Marginally significant.

The interactions of interest for testing our hypotheses were those between ambiguity and context. Ambiguity significantly interacted with context in the frontal, central and posterior regions (stats #4) and this effect significantly interacted with the hemiscalp variable in the frontal region (stats #2). Analyses conducted separately on each frontal hemiscalp indicated that the interaction between ambiguity and context was significant over the right hemiscalp (F(1,19) ¼10.435, p ¼.004), but not over the left hemiscalp. The ERPs are presented in Fig. 5. The ambiguity effect was examined separately for objects presented in neutral scenes and objects presented in congruent scenes over the regions where ambiguity significantly interacted with context. The ambiguity effect was significant over the frontal, central, and posterior regions for neutral contexts (stats #8a), and over the central and posterior regions for congruent contexts (stats #8b). 3.2.2. Context effect The context effect reflects the voltage difference between objects presented in neutral scenes and objects presented in congruent scenes. It consists of more positive voltages in the frontocentral regions, and more negative voltages in the temporal and posterior regions for objects presented in congruent scenes relative to objects presented in neutral scenes. The main effect of context was significant in the frontal, central, and posterior regions (stats #12). It interacted with the hemiscalp variable in the central and temporal regions (stats #10). Since the context effect was not significant in the temporal region, further analysis was done to see if the context effect was significant in one of the two hemiscalps. The context effect over the left temporal region was

almost significant (F(1,19) ¼3.981, p¼ .061). The interaction between ambiguity and context (stats #4) made it possible to look at the context effect in ambiguous and unambiguous objects separately in several electrode regions. The ERPs are illustrated in Fig. 6. In the posterior region, the context effect was significant for ambiguous objects (stats #12b) and marginally significant for unambiguous objects (stats #12a). The context effect was also significant for ambiguous objects in the central and frontal regions (stats #12b). However, the context  ambiguity interaction varied as a function of the hemiscalps in the frontal region (stats #2), therefore, the context effect was analyzed separately in each hemiscalp. The context effect was significant over both hemiscalps for ambiguous objects (left: F (1,19) ¼9.968, p ¼.005; right: F(1,19) ¼21.073, p o.001) and over the left hemiscalp for unambiguous objects (F(1,19) ¼12.766, p¼ .002). The largest context effect across all conditions was recorded over electrode Po7 in the posterior area and over Fcz over the fronto-central area. These ERPs are illustrated in Fig. 7. This figure allows the comparison of the context effect in all conditions.

3.2.3. P1 and N1 The P1 (F(1,19) ¼ 13.267, p ¼.002) and the N1 (F(1,19) ¼25.305, po .001) were both significantly larger for unambiguous objects relative to ambiguous objects. When comparing congruent contexts with neutral contexts, only the N1 was significantly different (F(1,19) ¼10.602, p¼ .004). There were no significant interactions.

48

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

Fig. 5. Effect of ambiguity. Top: ERPs to ambiguous (black) and unambiguous (orange) objects presented in neutral (left panel) and congruent (right panel) scenes. Bottom: Voltage maps at consecutive 40 ms time windows representing the difference in activity between ambiguous and unambiguous objects. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

4. Discussion This study examined the electrophysiological brain correlates underlying the use of scene context in the recognition of ambiguous and unambiguous objects. First, we found that scene context influences the recognition of ambiguous and unambiguous target objects. The effect of context is strongly influenced by the ambiguity of these objects. Specifically, congruent scene contexts produced stronger brain modulation during the recognition of ambiguous objects, compared to unambiguous objects. Second, the results showed that object ambiguity altered recognition processes (what we called the ambiguity effect), and that the context effect consisted primarily of a modulation of this effect over frontocentral and posterior areas. Congruent contexts reduced the effect of ambiguity and caused ambiguous objects to be treated as unambiguous. This can be seen in Fig. 7. Third, the influence of context began to facilitate recognition at early stages of perceptual object analysis, directly modulating the influence ambiguity has on the initial steps of recognition processing. The present experiment examined the contextual influence that facilitates the recognition of objects. As a result, the incongruities between scenes and target objects commonly observed in previous studies had to be reduced as much as possible. Incongruities are typically created by implementing contextual violations (Biederman et al., 1982). By comparing congruent and

incongruent stimuli, several teams were able to examine the cognitive processes underlying contextual violations, such as semantic and syntactic violations (Võ and Wolfe, 2014). Although these effects clearly depend on context, they do not make objects easier to perceive. The effects resulting from contextual violations and incongruent contexts were minimized in the present study by using neutral scene contexts. Despite their neutrality, these scenes could not control for all contextual effects and violations. For example, neutral contexts could not prevent subjects from using the scenes to estimate the size of the target objects. Nevertheless, neutral scenes likely contributed less to recognition compared to congruent scenes, as indicated by the behavioral results. The behavioral results have validated the norms that were used for selecting the stimuli by demonstrating that congruent scenes increased the speed and rate of recognition for ambiguous objects. These findings were in agreement with studies that showed comparable influences of context on the accuracy of recognition (Biederman et al., 1982; Davenport and Potter, 2004) and naming of objects (Boyce and Pollatsek, 1992). However, accuracy was not verified in the present study because it was assumed that recognition processing operates regardless of the real identity of the object. A perceived object can match stored representations to various degrees, influencing the confidence of a response. Confusing one object for another object, such as a ping-pong ball for a golf ball, does not change the fact that it matched a stored

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

49

Fig. 6. Effect of context. Top: ERPs to ambiguous (left panel) and unambiguous (right panel) objects presented in congruent (black) and neutral (orange) scenes. Bottom: Voltage maps at consecutive 40 ms time windows representing the difference in activity between objects in congruent and neutral objects. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

representation of the object in memory. Therefore, the confidence of a response cannot be dissociated from the definition of recognition. The analyses were limited to recognized responses only, to match the level of confidence across all experimental conditions to account for the lack of accuracy data. The effect of context seen in the present project is primarily a modulation of the effect that ambiguity has on recognition. This claim is supported by the similarities between the spatio-temporal ambiguity effect and context effect for ambiguous objects. The ambiguity effect consisted of more negative voltage amplitudes to ambiguous objects in frontal, central, and midline regions and

more positive voltage amplitudes in temporal and posterior regions relative to unambiguous objects. It was diffuse, being uniformly present across all time windows, aside from temporal and midline regions. The ambiguity effect observed in the present project shares similarities with the ERP responses to object recognition reported in previous studies (Johnson and Olshausen, 2003). By definition, ambiguity varies with the capacity to recognize a stimulus, and therefore the ambiguity effect observed may reflect the activation of object recognition processes. The effect of ambiguity can also be compared to ERPs elicited by fragmented line drawings. Studies presenting line drawings, at

Fig. 7. ERPs to all four conditions: ambiguous objects in neutral scenes (thin grey line); ambiguous objects in congruent scenes (thick grey line); unambiguous objects in neutral scenes (thin black line); and unambiguous objects in congruent scenes (thick black line).

50

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

different levels of completeness, found increasingly negative potentials for less complete pictures in time windows of 250– 1000 ms (Stuss et al., 1992; Viggiano and Kutas, 1998, 2000). Less complete pictures, or ambiguous objects are less predictable, likely resulting in increased hypothesis generation of possible object identities (Stuss et al., 1992), and therefore, more negative potentials. It has also been shown that responses of higher certainty are reflected by a diffuse response from 250 to 1600 ms of more positive potentials across frontal and central electrodes. The responses to more complete or unambiguous objects are likely given with greater certainty, as they are more easily recognized, contributing to the more positive potentials found. Therefore, the ambiguity effect observed in the present study likely reflects the generation of hypotheses related to the recognition of an object, as well as the certainty of recognition. The context effect had a very similar pattern of activation to the ambiguity effect, but with opposite polarity, as if it canceled the ambiguity effect (see Fig. 6). This would explain why the ERPs to ambiguous objects presented within a congruent scene were very comparable to those of unambiguous objects. Sereno et al. (2003) reported comparable findings with ambiguous words placed in sentences. They found that when placed in a biasing context, the responses to ambiguous words became similar to those of unambiguous words. In another study, Lee and Federmeier (2009) reported that semantic constraints eliminated a frontal ambiguity effect to homographs. Our results provide further support that context likely attenuates ambiguity-related activities effecting recognition processes rather than generating additional processing. Specifically, it narrows the range of identities for an object and increases the confidence the subjects have in their answers. The effect of context began remarkably early, around the P1, before object processing ended. This demonstrated that the scene influenced the processing of an object not once it had been decoded, but right at the start, through immediate and direct modulation. Such an early effect occurred because the scene preceded the perception of the target object. Demiral et al. (2012) reported an early effect of context when the scene was presented prior to the target but not when the scene and the object were presented simultaneously. In another study, the amplitude of early ERPs over N1, evoked by superimposed simple shapes, were strongly determined by whether these shapes were presented simultaneously or in sequence (Brodeur et al., 2008a, 2008b). The influence of scene context on object recognition was not limited to a modulation of ambiguity but was also found with unambiguous objects. The effect in response to unambiguous objects (also found with ambiguous object) was present over the left frontal region and to a lesser extent over the posterior region. It was different from the effect resulting from the modulation of ambiguity, which was distributed more diffusely across the scalp. The central and posterior context effects might be elicited more strongly in neutral scenes than in congruent scenes. Even if scenes were not highly incongruent, objects in neutral scenes were unexpected and thus, prone to elicit posterior P300 or P3b (Polich, 2007). When objects are unexpected, novelty-related processes are initiated (Dien et al., 2003), and an update of the actual scene stimulus is activated (Donchin, 1981). This process is commonly held responsible for the P3b. In line with the view of Kok (2001), it could also be argued that the P3b was enhanced with neutral scenes, because identifying and matching the object with internal representations built from scene information was more difficult. Fewer of these activations would be required in conditions of congruity, as the objects add less information to the scenes. The differences in the experimental design and stimuli account for most of the differences between the results of the present study and those from previous investigations of scene and object congruency. In these studies, the N300 and N400 components

were generally the main effects observed. It has been suggested that the N300/400 response results from a violation of previously formed expectancies from the initial presentation of the scene context (Demiral et al., 2012). Specifically, the N300 reflecting perceptual object processing (Mudrik et al., 2010; Barrett and Rugg, 1990; Federmeier and Kutas, 2001; Sitnikova et al., 2008) and structural description matching (Doniger et al., 2000; Holcomb and McPherson, 1994; Schendan and Kutas, 2002) and the N400 reflecting post identification processes at the level of semantic analysis (Ganis and Kutas, 2003; Mudrik et al., 2010). Among the notable differences from the present project, the experimental design in these studies allowed for the anticipation of a specific target (e.g., a football player kicking a target). The scenes used in the present study did not include individuals acting upon objects and objects could not always be predicted with precision, even in the congruent conditions. Another notable difference is that the present study did not include incongruent scene-target conditions, that sometimes represented impossible situations (Demiral et al., 2012). Mudrik et al. (2014) offered a different interpretation in which the effect of incongruity was explained as relying on the processing of the semantic relationship between the scene and its constituents. Our results are more consistent with this view. Although the neural sources of the ERP effects cannot be determined with great precision, the present findings fit with models developed using fMRI results. The Bar model (Bar, 2004) for context processing states that probable context frames are activated in the parahippocampal cortex (PHC) based on incoming low spatial frequency information from the visual cortex. In parallel, the prefrontal cortex (PFC) develops guesses as to the identity of the target object from low spatial frequency information that is centered on foveal vision. The expectancies from both the PFC and PHC are then delivered to the inferior temporal cortex, where they interact and allow for the identification of a single object. The present frontal and central/posterior context effects could respectively be related to the PFC and the inferior temporal activities of this model. However, these conclusions are premature and require further investigation with brain imaging techniques. Together, the present findings demonstrate that context modulates object recognition processing and is of particular importance in the recognition of ambiguous objects. Context is able to exert its influence at early stages of perceptual analysis, when ambiguity begins to present an obstacle to recognition. This study shows that N300 and N400 like responses can occur without the generation of expectancies and without semantic context violations, indicating a response due to a congruent context rather than an inconsistency between a scene and an object. The results also suggest that the effect of context observed is mostly an indirect effect; context appears to disambiguate ambiguous objects. A congruent context modulates the recognition processes of ambiguous objects so that they are treated like unambiguous objects.

Acknowledgments We are grateful to Laurent Lestage, Sanela Music, and Asra Toobaie for their contribution on this work. This study was funded by the Natural Sciences and Engineering Research Council of Canada (#388752-2012).

References American Electrophysiological Society, 1994. Guideline for standard electrode position nomenclature. J. Clin. Neurophysiol. 11, 111–113. Bar, M., 2004. Visual objects in context. Nat. Rev. Neurosci. 5, 617–629.

M. Dyck, M.B. Brodeur / Neuropsychologia 72 (2015) 43–51

Bar, M., Ullman, S., 1996. Spatial context in recognition. Perception 25, 343–352. Biederman, I., Mezzanotte, R.J., Rabinowitz, J.C., 1982. Scene perception: detecting and judging objects undergoing relational violations. Cogn. Psychol. 14, 143–177. Barrett, S.E., Rugg, M.D., 1990. Event-related potentials and the semantic matching of pictures. Brain Cognit. 14 (2), 201–212. Biederman, I., 1972. Perceiving real-world scenes. Science 177, 77–80. Boyce, S.J., Pollatsek, A., 1992. Identification of objects in scenes: the role of scene background in object naming. J. Exp. Psychol.: Learn. Mem. Cognit. 18, 531–543. Brodeur, M., Bacon, B.A., Renoult, L., Prevost, M., Lepage, M., Debruille, J.B., 2008a. On the functional significance of the P1 and N1 effects to illusory figures in the notch mode of presentation. PLoS One 3, e3505. Brodeur, M., Lepore, F., Lepage, M., Bacon, B.A., Jemel, B., Debruille, J.B., 2008b. Alternative mode of presentation of Kanizsa figures sheds new light on the chronometry of the mechanisms underlying the perception of illusory figures. Neuropsychologia 46, 554–566. Davenport, J.L., Potter, M.C., 2004. Scene consistency in object and background perception. Psychol. Sci. 15, 559–564. Demiral, S.B., Malcolm, G.L., Henderson, J.M., 2012. ERP correlates of spatially incongruent object identification during scene viewing: contextual expectancy versus simultaneous processing. Neuropsychologia 50, 1271–1285. Dien, J., Spencer, K.M., Donchin, E., 2003. Localization of the event-related potential novelty response as defined by principal components analysis. Cogn. Brain Res. 17, 637–650. Donchin, E., 1981. Surprise!… surprise? Psychophysiology 18, 493–513. Doniger, G.M., Foxe, J.J., Murray, M.M., Higgins, B.A., Snodgrass, J.G., Schroeder, C.E., Javitt, D.C., 2000. Activation timecourse of ventral visual stream object-recognition areas: high density electrical mapping of perceptual closure processes. J. Cogn. Neurosci. 12, 615–621. Doniger, G.M., Foxe, J.J., Schroeder, C.E., Murray, M.M., Higgins, B.A., Javitt, D.C., 2001. Visual perceptual learning in human object recognition areas: a repetition priming study using high-density electrical mapping. NeuroImage 13, 305–313. Federmeier, K.D., Kutas, M., 2001. Meaning and modality: influences of context, semantic memory organization, and perceptual predictability on picture processing. J. Exp. Psychol.: Learn. Mem. Cognit. 27, 202–224. Ganis, G., Kutas, M., 2003. An electrophysiological study of scene effects on object identification. Cogn. Brain Res. 16, 123–144. Goto, S.G., Ando, Y., Huang, C., Yee, A., Lewis, R.S., 2009. Cultural differences in the visual processing of meaning: Detecting incongruities between background and foreground objects using the N400. Social cognitive and affective neuroscience, nsp038. Henderson, J.M., Hollingworth, A., 1999. High-level scene perception. Annu. Rev. Psychol. 50, 243–271.

51

Holcomb, P.J., McPherson, W.B., 1994. Event-related brain potentials reflect semantic priming in an object decision task. Brain Cognit. 24, 259–276. Hollingworth, A., Henderson, J.M., 1998. Does consistent scene context facilitate object perception? J. Exp. Psychol.: Gen. 127, 398–415. Johnson, J.S., Olshausen, B.A., 2003. Timecourse of neural signatures of object recognition. J. Vis. 3, 499–512. Kok, A., 2001. On the utility of P3 amplitude as a measure of processing capacity. Psychophysiology 38, 557–577. Kutas, M., Federmeier, K.D., 2011. Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). Annu. Rev. Psychol. 62, 621–647. Lee, C.L., Federmeier, K.D., 2009. Wave-ering: an ERP study of syntactic and semantic context effects on ambiguity resolution for noun/verb homographs. J. Mem. Lang. 61, 538–555. Mudrik, L., Lamy, D., Deouell, L.Y., 2010. ERP evidence for context congruity effects during simultaneous object-scene processing. Neuropsychologia 48, 507–517. Mudrik, L., Shalgi, S., Lamy, D., Deouell, L.Y., 2014. Synchronous contextual irregularities affect early scene processing: replication and extension. Neuropsychologia 56, 447–458. Oliva, A., Torralba, A., 2007. The role of context in object recognition. Trends Cogn. Sci. 11, 520–527. Palmer, S.E., 1975. The effects of contextual scenes on the identification of objects. Mem. Cognit. 3, 519–526. Polich, J., 2007. Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 118, 2128–2148. Schendan, H.E., Kutas, M., 2002. Neurophysiological evidence for two processing times for visual object identification. Neuropsychologia 40, 931–945. Sereno, S.C., Brewer, C.C., O’Donne, P.J., 2003. Context effects in word recognition evidence for early interactive processing. Psychol. Sci. 14, 328–333. Sitnikova, T., Holcomb, P.J., Kiyonaga, K.A., Kuperberg, G.R., 2008. Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. J. Cogn. Neurosci. 20, 2037–2057. Stuss, D.T., Picton, T.W., Cerri, A.M., Leech, E.E., Stethem, L.L., 1992. Perceptual closure and object identification: electrophysiological responses to incomplete pictures. Brain Cognit. 19, 253–266. Viggiano, M.P., Kutas, M., 1998. The covert interplay between perception and memory: event-related potential evidence. Electroencephalogr. Clin. Neurophysiol. 108, 435–439. Viggiano, M.P., Kutas, M., 2000. Overt and covert identification of fragmented objects inferred from performance and electrophysiological measures. J. Exp. Psychol.: Gen. 129, 107–125. Võ, M.L.H., Wolfe, J.M., 2013. Differential electrophysiological signatures of semantic and syntactic scene processing. Psychol. Sci. 24, 1816–1823.