J. Phyjiol. (1978), 28, pp. 193-201 With 4 plate. and 3 text-figure. Printed inGrbea Britain
193
A COMPARISON OF THRESHOLD AND SUPRATHRESHOLD APPEARANCE OF GRATINGS WITH COMPONENTS IN THE LOW AND HIGH SPATIAL FREQUENCY RANGE
By F. W. CAMPBELL, E. R. HOWELL* AND J. R. JOHNSTONEt From the Phygiological Laboratory, University of Cambridge, Cambridge CB2 3E0
(Received 22 July 1977) SUMMARY
1. The appearance of square gratings with some of their Fourier components missing has been investigated for both threshold and suprathreshold contrasts. 2. If high frequency components are removed from a square grating there is only a very small effect on the detection threshold, or suprathreshold appearance, unless the components are visible by themselves. 3. If the fundamental frequency is removed from a square-wave grating which has a spatial frequency lower than 1 cycle per degree (c/d) the contrast sensitivity is not altered. This is a generalization of the Craik-Cornsweet illusion. If the contrast is raised above the detection threshold the grating is indistinguishable from a square grating, unless the contrast is high enough to see the fundamental when it is presented alone. 4. If the fundamental is removed from a square grating which has a spatial frequency higher than 1 c/d the contrast threshold and the appearance at all contrasts are changed. At threshold it appears as a sinusoidal grating of three times the fundamental frequency. The threshold is dictated solely by the amplitude of the third harmonic. If the contrast is further raised, so that the fifth harmonic also reaches threshold, the periodictiy of the fundamental is seen. 5. Therefore, gratings of many different luminance profiles (including the CraikCornsweet profile) all produce the perception of a square grating simply because those missing components which would be required in each case to produce a perfect square are by themselves undetectable. The visual system responds as though hardwired to detect square gratings and edges by means of quasi-Fourier analysis. 6. These results are analogous to the missing fundamental, or residue, effect in hearing. INTRODUCTION
The idea of using gratings with different harmonic contents dates from Ernst Mach, who described a mechanical method of producing them in 1866 (see Ratliff, 1965, pp. 291-292 for translation). He was investigating what is now called Mach Bands and he wrote, 'On the occasion of these experiments I tired of painting sectors and devised Present address: National Vision Research Institute, 374 Cardigan Street, Carlton, Victoria 3053, Australia. t Present address: Department of Physiology, University of Western Australia, Nedlands 6009, Western Australia. *
7
PHY
284
F. W. CAMPBELL AND OTHERS 194 a simple device by means of which one can at least represent a very great number of different light curves. This is based on the Fourier series'. However, it is not certain that he used the apparatus described to investigate the visual system. Campbell & Robson (1968) showed that square-wave gratings were always visible at a lower contrast than sine-wave gratings of the same spatial frequency. At high frequencies they found that the increased contrast sensitivity to square-wave gratings could be accounted for quantitatively by the fact that the ratio of the contrast at the fundamental frequency of a square-wave compared with a sine wave grating is 4/1 = 1*27. They were unable to account for their results below about 1 c/d. Above this frequency, they could not only account for the thresholds of sine and square-wave gratings but also of gratings with a wide variety of different wave forms. Their failure to account for thresholds at low spatial frequencies suggests that there is some basic difference in our mode of seeing below and above 1 c/d. In this paper we consider the threshold and suprathreshold appearance of square gratings with either low or high frequency components missing. Part of these results has been presented elsewhere (Campbell, Howell & Robson, 1971). METHODS
Square-wave gratings with various low and high frequency components missing were generated analogically and digitally and displayed on a cathode ray tube 25 cm x 20 cm of luminance 15 cd/mi. They were either observed directly or photographed for later use as prints (e.g. Pls. 1-4) or slides. When observed directly the contrast of the gratings was raised slowly by the subject until they reached threshold. If the contrast is raised rapidly or switched on and off suddenly, gratings below 1 c/d are seen more readily at the time of the transient and then they fade away (Robson, 1966). These observations are graphed in Text-figs. 1-2 where each point is the mean of ten observations. Viewing distance was 57 cm. RESULTS
Removal of high harmonica from a square grating The reader may repeat our observations using Pls. 1 and 4. The first five gratings of P1. 1 are successive, nearer approximations to a square grating, being 1st harmonic only, 1st + 3rd, 1st + 3rd + 5th, 1st + 3rd + 5th + 7th and 1st + 3rd + 5th + 7th + 9th. The 6th has fifteen harmonics which is approximately as many as can be resolved by the printing process. P1. 1 F is as nearly square as can be reproduced. P1. 4 shows the individual harmonics up to the 9th and 15th, 11 and 13 being omitted. When viewed from 10 m, the gratings of P1. 1 all look the same (except for slight variations in mean luminance due to limitations of photography and printing) and appear equally square. As they are approached, the first to look different from the rest is the fundamental. This happens at that distance at which the 3rd harmonic (P1. 4B) is just detectable. Closer still, the next grating in P1. 1 to look different is the 1st + 3rd (1 B), and again this occurs at that distance at which the 5th by itself (P1. 4 C) is just visible. Similarly for the other harmonics. Gratings are therefore indistinguishable from a perfect square if the missing higher harmonics are by themselves undetectable. This was found to be not exactly correct
195 VISIBILITY OF GRATINGS and harmonics were very slightly more readily detected when they formed part of a square grating, i.e. it was necessary to approach the photograph of the harmonic alone more closely to detect it than to detect its absence from a square. However, this effect was very small and corresponded to a contrast difference of less than 0- 1 log unit. With that proviso, the rule describing these observations is simple: a square grating is indistinguishable from one with some of its higher harmonics missing unless the missing ones are detectable by themselves. 1000
-.-.-5- -5-
*
-
-d.
0'
100
a)
0 _,/
0 U,
10
1 001
11111111 01
11111111
I,
1
I1
111 10
100
Spatial frequency (c/d)
Text-fig. 1. The open circles are the thresholds for sine gratings and the filled squares for a square-wave form. The large filled square to the far left is the threshold for a single edge bisecting the screen. The data for the sine gratings have been moved upwards by the ratio 4. i. The curves through the results have been drawn by eye and are transferred to Text-fig. 2.
Removal of lower harmonics Intially we repeated the observations of Campbell & Robson (1968) on the visibility of square-waves. In Text-fig. 1 the contrast sensitivity for the sine gratings are shown as open circles and the sensitivity for square-wave gratings are shown as squares (U). The contrast relation between these waveforms is 4/iT, but for convenience the amplitude of the sine wave is normalized. Because of this correction the results for the two waveforms overlap at frequencies higher than 1 c/d. Now both sets of data can be fitted with one curve for frequencies higher than 1 c/d, the continuous curve. Below 1 c/d, two curves are required, one to describe the continuing low frequency attenuation of the sine-wave results (dashed curve) and a second horizontal line (dot-dashed) for the square-wave results. The attenuation at low frequencies has a slope of about + 1 which means that the 7-2
196 F. W. CAMPBELL AND OTHERS contrast sensitivity is directly proportional to spatial frequency. If this slope is extrapolated to a contrast sensitivity of 1, which is the maximum contrast physically possible, the lowest perceivable spatial frequency is about 1 cycle per 1800 of visual angle. By viewing the sinusoidal grating with our eye very close to the screen, and optically corrected for the viewing distance, we have confirmed that a high contrast (0.8) grating is not visible at these very low spatial frequencies. However, the visual response to a low frequency square-wave grating is quite different. Consider a square-wave grating of very low frequency with only one edge visible. Lowering the spatial frequency further would not change in any way the appearance of the step, providing the electronics generating the square-wave has a fast enough response time to keep the step infinitely steep. In a sense, a step function of luminance is a square wave of zero spatial frequency. The large square (U) to the left in Text-fig. 1 is the contrast sensitivity of a single edge centred on the screen. Thus, contrast sensitivity is constant from 1 c/d right down to zero spatial frequency, which is a single edge or step function. In terms of the Fourier series most of the 'energy' in a square wave is at the fundamental frequency (81 %). But for frequencies less than 1 c/d, where the sensitivity to the fundamental is decreasing rapidly, the sensitivity to a square wave remains high and constant. Does the fundamental then contribute to the contrast sensitivity of a low frequency square-wave? This can be answered readily by removing the fundamental while leaving all the higher harmonics intact. In order to help the reader understand what is meant by a missing fundamental luminance profile, we have prepared Pls. 2 and 3. Position the journal so that P1. 2 is above P1. 3. The upper half of P1. 2 shows a sine wave of amplitude equal to the fundamental but displaced in phase by 1800 (A). The next down illustrates a squarewave (C) with the above sine (A) added to it, resulting in a missing fundamental square-wave (B). (The rise time of the oscilloscope beam is so rapid that the vertical transients are not photographed.) The lower half of P1. 2 and P1. 3 show gratings which correspond to these luminance profiles. The reader should first examine P1. 3 from a distance of 10 cm wearing positive power lenses if insufficient accommodation is available. At that distance the spatial frequency of the square grating is about 0 75 c/d. Note that the appearance of the two gratings is very similar although their luminance profile is dramatically different. Indeed, as can be noted from P1. 2B part of the dark bar of the missing fundamental profile is physically lighter than part of the light. The reason for this similarity is that the only difference between them is a sine grating which is itself undetectable, as can be confirmed by examining P1. 2D from the same distance. This is the explanation of the illusion described by Craik (1940, 1966) and Cornsweet (1970). The inset in Text-fig. 3 shows the relationship between the missing fundamental and Craik-Cornsweet profiles. The advantage of our approach is that it is easy to generate the luminance profiles and vary their contrast so that thresholds and breakdown contrast levels can readily be measured. We now measure the threshold contrast for a missing fundamental square-wave grating. For ease of illustration, the curves (without the data) have been transferred from Text-fig. 1 to Text-fig. 2. The results are shown as open squares (Z) in Text-fig. 2. It is clear that below 1 c/d there is good agreement between the thresholds for
VISIBILITY OF GRATINGS 197 square-wave gratings (dot-dashed horizontal line) and square-wave gratings with the fundamental frequency removed. Also as soon as the square, or the square with the missing fundamental, reaches threshold, they are seen as identical square-wave gratings. Therefore at these low spatial frequencies the fundamental frequency in the square wave does not contribute to its detection or appearance at threshold. The next question now arises: at what contrast level will a square-wave grating without its fundamental appear different from a square-wave grating? (The reader may appreciate this point better if P1. 4 is viewed from about 3 m (2.2 c/d.) It will be noted that the fundamental is now clearly missing. 1000 lo
11-111 '
1 1 111111
I
I
fil-li
I
1
I
eil
100 ._
\~U
C
0
10
0-01
0.1
I
10
100
Spatial frequency (c/d) Text-fig. 2. The continuous curve and the interrupted lines have been transferred from Text-fig. 1. The open squares are the thresholds for the missing fundamental grating. Likewise, the crossed squares, but in this instance the 3rd harmonic appears at threshold. The double arrow illustrates how the continuous line has been transferred a factor of 3 in the contrast sensitivity and frequency domains. The bisected circles represent the threshold for noting that the fundamental was missing and that the grating was different from a standard square-wave grating. The darkened area is the region where the phenomenon of the missing fundamental exists.
The contrast level at which the missing fundamental profile ceased to look like a square-wave was measured at a number of different frequencies. This was done by providing the subject with a potentiometer so that he could change slowly in time from a square-wave to a square wave without its fundamental. By rotating the potentiometer he could constantly remind himself of what a square-wave grating looks like at the particular contrast level that had been set. The contrast levels at which the two profiles were first seen to be different are shown as half-filled circles in Text-fig. 2. The appearance of the square-wave at these contrasts remains a square
198 F. W. CAMPBELL AND OTHERS but the square minus its fundamental begins to look slightly bowed in its luminance profile (see Text-fig. 2 inset). The 'break-down' in appearance occurs at that contrast where the fundamental would have reached its threshold if it had been presented on its own. This is shown by the hatched line transferred from Text-fig. 1 which represents the threshold for sine-wave gratings of these low frequencies. 1000
C
100
0201
01
1 Spatial frequency (cideg)
10
100
Text-fig. 3. The continuous curve is the contrast sensitivity of a sine wave grating. The horizontal line scaled in metres is positioned at the contrast level of the gratings shown in Pls. 1 and 4 (marked by an arrow on the left scale). The distances marked A, B, C, D and E are the positions where changes in the appearance of the missing fundamental gratings occurs using P1. 4 viewed in good lighting by an observer with normal vision. The details are given in the text.
The dark triangular area is the region where the fundamental is not required and the two profiles appear as a square wave. (By analogy with the missing fundamental effect in hearing this might be called the 'existence' region.) These findings may be summarized if the reader makes a final inspection of Pls. 2 and 3. Start from a distance of 30 m and note that none of the three gratings can be detected. The reason is clear from Texrt-fig. 3 which shows the contrast sensitivity curve for sine gratings transferred from Text-figs. 1 and 2. The contrast of the gratings displayed is about 4 %/ which represents a contrast sensitivity of 25. The horizontal line with a scale of viewing distances is placed at this contrast sensitivity. At 30 m a grating of this contrast does not reach threshold. Now advance to about 25 m (position A, in Texrt-fig. 3). Both the sine and square-wave grating can be seen. They look identical because the fundamental of the square has reached threshold at the same time as the sine grating, which has the same contrast as the fundamental in the square wave (the 4/i has been allowed for). Now, advance slowly and note that at
199 VISIBILITY OF GRATINGS about 6 m the sine and square-wave gratings look different and that it is possible to tell which one is which (position B). The reason is that the 3rd harmonic has now
reached threshold; this is indicated by the arrowed lines intersecting the distance scale at B in Text-fig. 3. Note that this is also the point at which the missing fundamental grating is now seen as a sine grating with 3 times its fundamental frequency. The obvious question now arises. What do all the other higher odd harmonics present in the square-wave do? Something should happen when the 5th gets above threshold. Approach slowly to a distance of about 2-5-3 m (position C on the scale). Note that now you can see for the first time the periodicity in the missing fundamental photograph, although there is no component of this frequency present in the grating. We can conclude that when the 3rd and the 5th harmonics are present they can generate the periodicity from which they must have arisen. The other harmonics become visible at the distances numbered (7, 9, 11 up to 29). The importance of high harmonics in generating the perception of squares can readily be demonstrated by masking three or four of the edges of the two gratings in PI. 3 with straight slivers of paper. Square and missing fundamental gratings both appear uniform. Removal of the sliver produces, after a few seconds delay, the original apparent contrast. As previously described, a square-wave grating looks like a square when it reaches threshold providing its frequency is less than 1 c/d. This is not so for a square-wave above 1 c/d for when it reaches threshold it is indistinguishable from a sine grating of the same spatial frequency as the fundamental. Campbell & Robson (1968) went on to prove that the contrast had to be increased to the point where the third harmonic was detected before the grating could be differentiated from a sinusoidal grating. We confirm this observation. The contrast for detecting the square with the missing fundamental is also quite different from a square wave. These measurements are shown as crossed squares in Text-fig. 2. It can be seen that the contrast'sensitivity is very much less for this wave form as compared with either a square or sine profile. How can these results be accounted for? Because the 3rd harmonic is perceived as the grating reaches threshold, the proposition can be advanced that detection is occurring because of the 3rd harmonic which is acting in the absence of the fundamental. If this be so, then translating the continuous curve, which represents the contrast threshold for sine waves, 3 times to the left and 3 times down should give the threshold for the square wave with its fundamental missing. This operation is done because the 3rd harmonic is 3 times up in frequency and 3 times down in amplitude. The dashed curve and two arrows represent this translation. The fit is good. DISCUSSION
There are two equally valid ways of looking at these results. On the one hand, removal of a sinusoidal grating from a complex one causes a detectable change only if the grating removed would have been detectable by itself. This is scarcely surprising. Therefore, many physically different gratings must necessarily be perceived as identical square gratings. On the other hand, why should they be perceived as square gratings rather than some other profile? There is independent evidence that harmonics from a square-wave sequence are
200 F. W. CAMPBELL AND OTHERS treated uniquely by the visual system. Atkinson & Campbell (1973) studied the effect of phase on the appearance of gratings made up of the sum of the 1st and 3rd harmonics. Generally the gratings were perceived to alternate in time. The higher frequency grating was first seen, then the low and so on. Only when the gratings were in the phase required to generate a square grating, was there stability. Even when the harmonics were in triangular mode phase they alternated. Here the visual system behaves as though hardwired to detect square waves. The results of the present paper support this conclusion. The visual input appears to be analysed into its Fourier components and if these constitute a square sequence with no above-threshold components missing then the perception of a square-grating results. This solves the problem, considered by Helmholtz & Mach (Ratliff, 1965, p. 265), of why the aberrations of the emmetropic eye do not result in all edges appearing blurred. Only when the blurring is great enough to remove independently detectable high frequency components does the blur become detectable. Otherwise many edges with different luminance gradients will be indistinguishable from a sharp one, as when those in P1. 1 are viewed from a sufficient distance. Part at least of the neural mechanism required for such processing is known to be present. Blakemore & Campbell (1969) showed the existence of spatial frequency channels in human vision by an adaptation technique. Campbell et al. (1969) measured the spatial frequency responses of individual neurones in the visual cortex of the cat and found them to be tuned to specific and different spatial frequencies. A phenomenon similar to the visual missing fundamental is the residue (missing fundamental) effect in audition. The pitch of a square-wave sound is perceived to be the fundamental and is unaffected by removal of the fundamental. Also, the pitch of a sequence of harmonics is the fundamental even if the fundamental itself is absent. Goldstein (1973) has suggested that the auditory system is hardwired to extract as the pitch a greatest common divisor for a set of harmonics. Here also the analytical mechanisms are certainly known to exist (in the cochlea) but as in vision the nature of the deductive mechanism is unknown. Auditory neurophysiologists have often used as a stimulus a click. The reason for this is that an impulse function contains all frequencies and therefore activates neurones of all characteristic frequencies. Thereafter, a pure tone of variable frequency can be used to measure the tuning characteristic of that particular neurone. Likewise, when searching for a visual neurone in the cortex an edge or slit (sharply focused on the retina) is used to stimulate the entire population of neurones responding to that orientation and direction of movement. However, once found, it is essential to use a sinusoidal grating to define the tuning characteristic. This suggestion is supported by the recent finding that there are neurones in area 17 of the cat whose response is similar to sinewave and square-wave gratings yet they are unresponsive if the edge alone is presented (Maffei, Morrone, Pirchio & Sandini, 1977). Thus a brisk response to a sharp edged stimulus or square-wave grating does not by itself supply any information concerning its basic trigger characteristic. J. R. Johnstone is supported by the Raine Medical Research Foundation and the National Health and Medical Research Foundation of Australia.
The Journal of Physiology, Vol. 284
Plate I
3 =~ ~ ~ ~Ot~'1 con = E ;
193
A COMPARISON OF THRESHOLD AND SUPRATHRESHOLD APPEARANCE OF GRATINGS WITH COMPONENTS IN THE LOW AND HIGH SPATIAL FREQUENCY RANGE
By F. W. CAMPBELL, E. R. HOWELL* AND J. R. JOHNSTONEt From the Phygiological Laboratory, University of Cambridge, Cambridge CB2 3E0
(Received 22 July 1977) SUMMARY
1. The appearance of square gratings with some of their Fourier components missing has been investigated for both threshold and suprathreshold contrasts. 2. If high frequency components are removed from a square grating there is only a very small effect on the detection threshold, or suprathreshold appearance, unless the components are visible by themselves. 3. If the fundamental frequency is removed from a square-wave grating which has a spatial frequency lower than 1 cycle per degree (c/d) the contrast sensitivity is not altered. This is a generalization of the Craik-Cornsweet illusion. If the contrast is raised above the detection threshold the grating is indistinguishable from a square grating, unless the contrast is high enough to see the fundamental when it is presented alone. 4. If the fundamental is removed from a square grating which has a spatial frequency higher than 1 c/d the contrast threshold and the appearance at all contrasts are changed. At threshold it appears as a sinusoidal grating of three times the fundamental frequency. The threshold is dictated solely by the amplitude of the third harmonic. If the contrast is further raised, so that the fifth harmonic also reaches threshold, the periodictiy of the fundamental is seen. 5. Therefore, gratings of many different luminance profiles (including the CraikCornsweet profile) all produce the perception of a square grating simply because those missing components which would be required in each case to produce a perfect square are by themselves undetectable. The visual system responds as though hardwired to detect square gratings and edges by means of quasi-Fourier analysis. 6. These results are analogous to the missing fundamental, or residue, effect in hearing. INTRODUCTION
The idea of using gratings with different harmonic contents dates from Ernst Mach, who described a mechanical method of producing them in 1866 (see Ratliff, 1965, pp. 291-292 for translation). He was investigating what is now called Mach Bands and he wrote, 'On the occasion of these experiments I tired of painting sectors and devised Present address: National Vision Research Institute, 374 Cardigan Street, Carlton, Victoria 3053, Australia. t Present address: Department of Physiology, University of Western Australia, Nedlands 6009, Western Australia. *
7
PHY
284
F. W. CAMPBELL AND OTHERS 194 a simple device by means of which one can at least represent a very great number of different light curves. This is based on the Fourier series'. However, it is not certain that he used the apparatus described to investigate the visual system. Campbell & Robson (1968) showed that square-wave gratings were always visible at a lower contrast than sine-wave gratings of the same spatial frequency. At high frequencies they found that the increased contrast sensitivity to square-wave gratings could be accounted for quantitatively by the fact that the ratio of the contrast at the fundamental frequency of a square-wave compared with a sine wave grating is 4/1 = 1*27. They were unable to account for their results below about 1 c/d. Above this frequency, they could not only account for the thresholds of sine and square-wave gratings but also of gratings with a wide variety of different wave forms. Their failure to account for thresholds at low spatial frequencies suggests that there is some basic difference in our mode of seeing below and above 1 c/d. In this paper we consider the threshold and suprathreshold appearance of square gratings with either low or high frequency components missing. Part of these results has been presented elsewhere (Campbell, Howell & Robson, 1971). METHODS
Square-wave gratings with various low and high frequency components missing were generated analogically and digitally and displayed on a cathode ray tube 25 cm x 20 cm of luminance 15 cd/mi. They were either observed directly or photographed for later use as prints (e.g. Pls. 1-4) or slides. When observed directly the contrast of the gratings was raised slowly by the subject until they reached threshold. If the contrast is raised rapidly or switched on and off suddenly, gratings below 1 c/d are seen more readily at the time of the transient and then they fade away (Robson, 1966). These observations are graphed in Text-figs. 1-2 where each point is the mean of ten observations. Viewing distance was 57 cm. RESULTS
Removal of high harmonica from a square grating The reader may repeat our observations using Pls. 1 and 4. The first five gratings of P1. 1 are successive, nearer approximations to a square grating, being 1st harmonic only, 1st + 3rd, 1st + 3rd + 5th, 1st + 3rd + 5th + 7th and 1st + 3rd + 5th + 7th + 9th. The 6th has fifteen harmonics which is approximately as many as can be resolved by the printing process. P1. 1 F is as nearly square as can be reproduced. P1. 4 shows the individual harmonics up to the 9th and 15th, 11 and 13 being omitted. When viewed from 10 m, the gratings of P1. 1 all look the same (except for slight variations in mean luminance due to limitations of photography and printing) and appear equally square. As they are approached, the first to look different from the rest is the fundamental. This happens at that distance at which the 3rd harmonic (P1. 4B) is just detectable. Closer still, the next grating in P1. 1 to look different is the 1st + 3rd (1 B), and again this occurs at that distance at which the 5th by itself (P1. 4 C) is just visible. Similarly for the other harmonics. Gratings are therefore indistinguishable from a perfect square if the missing higher harmonics are by themselves undetectable. This was found to be not exactly correct
195 VISIBILITY OF GRATINGS and harmonics were very slightly more readily detected when they formed part of a square grating, i.e. it was necessary to approach the photograph of the harmonic alone more closely to detect it than to detect its absence from a square. However, this effect was very small and corresponded to a contrast difference of less than 0- 1 log unit. With that proviso, the rule describing these observations is simple: a square grating is indistinguishable from one with some of its higher harmonics missing unless the missing ones are detectable by themselves. 1000
-.-.-5- -5-
*
-
-d.
0'
100
a)
0 _,/
0 U,
10
1 001
11111111 01
11111111
I,
1
I1
111 10
100
Spatial frequency (c/d)
Text-fig. 1. The open circles are the thresholds for sine gratings and the filled squares for a square-wave form. The large filled square to the far left is the threshold for a single edge bisecting the screen. The data for the sine gratings have been moved upwards by the ratio 4. i. The curves through the results have been drawn by eye and are transferred to Text-fig. 2.
Removal of lower harmonics Intially we repeated the observations of Campbell & Robson (1968) on the visibility of square-waves. In Text-fig. 1 the contrast sensitivity for the sine gratings are shown as open circles and the sensitivity for square-wave gratings are shown as squares (U). The contrast relation between these waveforms is 4/iT, but for convenience the amplitude of the sine wave is normalized. Because of this correction the results for the two waveforms overlap at frequencies higher than 1 c/d. Now both sets of data can be fitted with one curve for frequencies higher than 1 c/d, the continuous curve. Below 1 c/d, two curves are required, one to describe the continuing low frequency attenuation of the sine-wave results (dashed curve) and a second horizontal line (dot-dashed) for the square-wave results. The attenuation at low frequencies has a slope of about + 1 which means that the 7-2
196 F. W. CAMPBELL AND OTHERS contrast sensitivity is directly proportional to spatial frequency. If this slope is extrapolated to a contrast sensitivity of 1, which is the maximum contrast physically possible, the lowest perceivable spatial frequency is about 1 cycle per 1800 of visual angle. By viewing the sinusoidal grating with our eye very close to the screen, and optically corrected for the viewing distance, we have confirmed that a high contrast (0.8) grating is not visible at these very low spatial frequencies. However, the visual response to a low frequency square-wave grating is quite different. Consider a square-wave grating of very low frequency with only one edge visible. Lowering the spatial frequency further would not change in any way the appearance of the step, providing the electronics generating the square-wave has a fast enough response time to keep the step infinitely steep. In a sense, a step function of luminance is a square wave of zero spatial frequency. The large square (U) to the left in Text-fig. 1 is the contrast sensitivity of a single edge centred on the screen. Thus, contrast sensitivity is constant from 1 c/d right down to zero spatial frequency, which is a single edge or step function. In terms of the Fourier series most of the 'energy' in a square wave is at the fundamental frequency (81 %). But for frequencies less than 1 c/d, where the sensitivity to the fundamental is decreasing rapidly, the sensitivity to a square wave remains high and constant. Does the fundamental then contribute to the contrast sensitivity of a low frequency square-wave? This can be answered readily by removing the fundamental while leaving all the higher harmonics intact. In order to help the reader understand what is meant by a missing fundamental luminance profile, we have prepared Pls. 2 and 3. Position the journal so that P1. 2 is above P1. 3. The upper half of P1. 2 shows a sine wave of amplitude equal to the fundamental but displaced in phase by 1800 (A). The next down illustrates a squarewave (C) with the above sine (A) added to it, resulting in a missing fundamental square-wave (B). (The rise time of the oscilloscope beam is so rapid that the vertical transients are not photographed.) The lower half of P1. 2 and P1. 3 show gratings which correspond to these luminance profiles. The reader should first examine P1. 3 from a distance of 10 cm wearing positive power lenses if insufficient accommodation is available. At that distance the spatial frequency of the square grating is about 0 75 c/d. Note that the appearance of the two gratings is very similar although their luminance profile is dramatically different. Indeed, as can be noted from P1. 2B part of the dark bar of the missing fundamental profile is physically lighter than part of the light. The reason for this similarity is that the only difference between them is a sine grating which is itself undetectable, as can be confirmed by examining P1. 2D from the same distance. This is the explanation of the illusion described by Craik (1940, 1966) and Cornsweet (1970). The inset in Text-fig. 3 shows the relationship between the missing fundamental and Craik-Cornsweet profiles. The advantage of our approach is that it is easy to generate the luminance profiles and vary their contrast so that thresholds and breakdown contrast levels can readily be measured. We now measure the threshold contrast for a missing fundamental square-wave grating. For ease of illustration, the curves (without the data) have been transferred from Text-fig. 1 to Text-fig. 2. The results are shown as open squares (Z) in Text-fig. 2. It is clear that below 1 c/d there is good agreement between the thresholds for
VISIBILITY OF GRATINGS 197 square-wave gratings (dot-dashed horizontal line) and square-wave gratings with the fundamental frequency removed. Also as soon as the square, or the square with the missing fundamental, reaches threshold, they are seen as identical square-wave gratings. Therefore at these low spatial frequencies the fundamental frequency in the square wave does not contribute to its detection or appearance at threshold. The next question now arises: at what contrast level will a square-wave grating without its fundamental appear different from a square-wave grating? (The reader may appreciate this point better if P1. 4 is viewed from about 3 m (2.2 c/d.) It will be noted that the fundamental is now clearly missing. 1000 lo
11-111 '
1 1 111111
I
I
fil-li
I
1
I
eil
100 ._
\~U
C
0
10
0-01
0.1
I
10
100
Spatial frequency (c/d) Text-fig. 2. The continuous curve and the interrupted lines have been transferred from Text-fig. 1. The open squares are the thresholds for the missing fundamental grating. Likewise, the crossed squares, but in this instance the 3rd harmonic appears at threshold. The double arrow illustrates how the continuous line has been transferred a factor of 3 in the contrast sensitivity and frequency domains. The bisected circles represent the threshold for noting that the fundamental was missing and that the grating was different from a standard square-wave grating. The darkened area is the region where the phenomenon of the missing fundamental exists.
The contrast level at which the missing fundamental profile ceased to look like a square-wave was measured at a number of different frequencies. This was done by providing the subject with a potentiometer so that he could change slowly in time from a square-wave to a square wave without its fundamental. By rotating the potentiometer he could constantly remind himself of what a square-wave grating looks like at the particular contrast level that had been set. The contrast levels at which the two profiles were first seen to be different are shown as half-filled circles in Text-fig. 2. The appearance of the square-wave at these contrasts remains a square
198 F. W. CAMPBELL AND OTHERS but the square minus its fundamental begins to look slightly bowed in its luminance profile (see Text-fig. 2 inset). The 'break-down' in appearance occurs at that contrast where the fundamental would have reached its threshold if it had been presented on its own. This is shown by the hatched line transferred from Text-fig. 1 which represents the threshold for sine-wave gratings of these low frequencies. 1000
C
100
0201
01
1 Spatial frequency (cideg)
10
100
Text-fig. 3. The continuous curve is the contrast sensitivity of a sine wave grating. The horizontal line scaled in metres is positioned at the contrast level of the gratings shown in Pls. 1 and 4 (marked by an arrow on the left scale). The distances marked A, B, C, D and E are the positions where changes in the appearance of the missing fundamental gratings occurs using P1. 4 viewed in good lighting by an observer with normal vision. The details are given in the text.
The dark triangular area is the region where the fundamental is not required and the two profiles appear as a square wave. (By analogy with the missing fundamental effect in hearing this might be called the 'existence' region.) These findings may be summarized if the reader makes a final inspection of Pls. 2 and 3. Start from a distance of 30 m and note that none of the three gratings can be detected. The reason is clear from Texrt-fig. 3 which shows the contrast sensitivity curve for sine gratings transferred from Text-figs. 1 and 2. The contrast of the gratings displayed is about 4 %/ which represents a contrast sensitivity of 25. The horizontal line with a scale of viewing distances is placed at this contrast sensitivity. At 30 m a grating of this contrast does not reach threshold. Now advance to about 25 m (position A, in Texrt-fig. 3). Both the sine and square-wave grating can be seen. They look identical because the fundamental of the square has reached threshold at the same time as the sine grating, which has the same contrast as the fundamental in the square wave (the 4/i has been allowed for). Now, advance slowly and note that at
199 VISIBILITY OF GRATINGS about 6 m the sine and square-wave gratings look different and that it is possible to tell which one is which (position B). The reason is that the 3rd harmonic has now
reached threshold; this is indicated by the arrowed lines intersecting the distance scale at B in Text-fig. 3. Note that this is also the point at which the missing fundamental grating is now seen as a sine grating with 3 times its fundamental frequency. The obvious question now arises. What do all the other higher odd harmonics present in the square-wave do? Something should happen when the 5th gets above threshold. Approach slowly to a distance of about 2-5-3 m (position C on the scale). Note that now you can see for the first time the periodicity in the missing fundamental photograph, although there is no component of this frequency present in the grating. We can conclude that when the 3rd and the 5th harmonics are present they can generate the periodicity from which they must have arisen. The other harmonics become visible at the distances numbered (7, 9, 11 up to 29). The importance of high harmonics in generating the perception of squares can readily be demonstrated by masking three or four of the edges of the two gratings in PI. 3 with straight slivers of paper. Square and missing fundamental gratings both appear uniform. Removal of the sliver produces, after a few seconds delay, the original apparent contrast. As previously described, a square-wave grating looks like a square when it reaches threshold providing its frequency is less than 1 c/d. This is not so for a square-wave above 1 c/d for when it reaches threshold it is indistinguishable from a sine grating of the same spatial frequency as the fundamental. Campbell & Robson (1968) went on to prove that the contrast had to be increased to the point where the third harmonic was detected before the grating could be differentiated from a sinusoidal grating. We confirm this observation. The contrast for detecting the square with the missing fundamental is also quite different from a square wave. These measurements are shown as crossed squares in Text-fig. 2. It can be seen that the contrast'sensitivity is very much less for this wave form as compared with either a square or sine profile. How can these results be accounted for? Because the 3rd harmonic is perceived as the grating reaches threshold, the proposition can be advanced that detection is occurring because of the 3rd harmonic which is acting in the absence of the fundamental. If this be so, then translating the continuous curve, which represents the contrast threshold for sine waves, 3 times to the left and 3 times down should give the threshold for the square wave with its fundamental missing. This operation is done because the 3rd harmonic is 3 times up in frequency and 3 times down in amplitude. The dashed curve and two arrows represent this translation. The fit is good. DISCUSSION
There are two equally valid ways of looking at these results. On the one hand, removal of a sinusoidal grating from a complex one causes a detectable change only if the grating removed would have been detectable by itself. This is scarcely surprising. Therefore, many physically different gratings must necessarily be perceived as identical square gratings. On the other hand, why should they be perceived as square gratings rather than some other profile? There is independent evidence that harmonics from a square-wave sequence are
200 F. W. CAMPBELL AND OTHERS treated uniquely by the visual system. Atkinson & Campbell (1973) studied the effect of phase on the appearance of gratings made up of the sum of the 1st and 3rd harmonics. Generally the gratings were perceived to alternate in time. The higher frequency grating was first seen, then the low and so on. Only when the gratings were in the phase required to generate a square grating, was there stability. Even when the harmonics were in triangular mode phase they alternated. Here the visual system behaves as though hardwired to detect square waves. The results of the present paper support this conclusion. The visual input appears to be analysed into its Fourier components and if these constitute a square sequence with no above-threshold components missing then the perception of a square-grating results. This solves the problem, considered by Helmholtz & Mach (Ratliff, 1965, p. 265), of why the aberrations of the emmetropic eye do not result in all edges appearing blurred. Only when the blurring is great enough to remove independently detectable high frequency components does the blur become detectable. Otherwise many edges with different luminance gradients will be indistinguishable from a sharp one, as when those in P1. 1 are viewed from a sufficient distance. Part at least of the neural mechanism required for such processing is known to be present. Blakemore & Campbell (1969) showed the existence of spatial frequency channels in human vision by an adaptation technique. Campbell et al. (1969) measured the spatial frequency responses of individual neurones in the visual cortex of the cat and found them to be tuned to specific and different spatial frequencies. A phenomenon similar to the visual missing fundamental is the residue (missing fundamental) effect in audition. The pitch of a square-wave sound is perceived to be the fundamental and is unaffected by removal of the fundamental. Also, the pitch of a sequence of harmonics is the fundamental even if the fundamental itself is absent. Goldstein (1973) has suggested that the auditory system is hardwired to extract as the pitch a greatest common divisor for a set of harmonics. Here also the analytical mechanisms are certainly known to exist (in the cochlea) but as in vision the nature of the deductive mechanism is unknown. Auditory neurophysiologists have often used as a stimulus a click. The reason for this is that an impulse function contains all frequencies and therefore activates neurones of all characteristic frequencies. Thereafter, a pure tone of variable frequency can be used to measure the tuning characteristic of that particular neurone. Likewise, when searching for a visual neurone in the cortex an edge or slit (sharply focused on the retina) is used to stimulate the entire population of neurones responding to that orientation and direction of movement. However, once found, it is essential to use a sinusoidal grating to define the tuning characteristic. This suggestion is supported by the recent finding that there are neurones in area 17 of the cat whose response is similar to sinewave and square-wave gratings yet they are unresponsive if the edge alone is presented (Maffei, Morrone, Pirchio & Sandini, 1977). Thus a brisk response to a sharp edged stimulus or square-wave grating does not by itself supply any information concerning its basic trigger characteristic. J. R. Johnstone is supported by the Raine Medical Research Foundation and the National Health and Medical Research Foundation of Australia.
The Journal of Physiology, Vol. 284
Plate I
3 =~ ~ ~ ~Ot~'1 con = E ;
