Biological Cybernetics
Biol.Cybernetics31, 163 168 (1978)
@ by Springer-Verlag 1978
On the Correlation Model: Performance of a Movement Detecting Neural Element in the Fly Visual System W. H. Zaagman, H. A. K. Mastebroek, and J. W. Kuiper Laboratoryof GeneralPhysics,Departmentof Biophysics,Westersingel34, 9718 CM Groningen,The Netherlands
Abstract. The applicability of the basic principles of the
correlation model to the description of the activity of a movement detecting neuron in the third optic ganglion of the fly's visual system has been investigated. This wide field neuron is supposed to sum the outputs of a large number of correlators (i.e. multiplying units followed "by time averagers) that are distributed over almost the entire eye. The model describes and predicts the experimental results in a satisfactory way if a uniformly distributed system of correlators is assumed. The sampling base of the correlators in this system equals the interommatidial angle A~b. The half width of the spatial sensitivity distribution of the visual inputs of the correlators, A~, is equal to the half width of the retinula cells of the 1-6 system.
1. Introduction
In the study of the information processing in visual neural systems there are two major reasons to work with flies. First, the extensive unravelling of the anatomical architecture of the fly's visual system has yielded a wealth of details fitting together better and better in the resulting picture of the system as a whole (Strausfeld, 1976). The three optic ganglia of the fly, the lamina, the medulla and the lobula complex are built of periodic ordered columns of neurons. The retinal mosaic structure of the compound eye of the fly is mapped retinotopically onto these columns in such a way that each column corresponds to a certain optical axis in the visual field of the compound eye. In the 3rd order ganglion, the lobula complex, giant neurons are found which are functionally involved in the processing of the information-flow generated in this visual system by stimuli which move in the visual field of the eye. These giant elements can be divided into two systems: a
horizontal movement processing system and a vertical movement processing system (Dvorak et al., 1975; Strausfeld, 1976; Hausen, 1976; Hengstenberg, 1977; Eckert, 1978). Related to the giant neurons of these systems, there exist a few wide-field elements having visual fields which equal the visual field of the whole eye (Dvorak et al., 1975; Strausfeld, 1976; Hausen, 1976). These wide field elements may hold a key position in the processing of moving stimuli in the fly's visual system. Second, an overwhelming number of broadly varied (behavioural) optomotor experiments on insects has markedly deepened the understanding of the functioning of the visual system of the fly. On the base of this type of experiments Reichardt developed the so-called correlation model which has been shown to exhibit a strongly predictive character and which plays a fundamental role in the description and investigation of the visual system of insects (Reichardt, 1957, 1969; Hassenstein, 1959; Reichardt and Varjfi, 1959; Varjfi, 1959; Poggio and Reichardt, 1976; Reichardt and Poggio, 1976; Buchner, 1976). The successful application of the correlation model in the description of the fly's behaviour on the one side, together with the growing knowledge of the anatomy of the fly's visual system on the other side, constitute a challenge to investigate in how far the fundamental performances of the correlation model can be retraced in the neural system of the fly. It is the aim of this paper to describe the applicability of the basic principles of the correlation model to the nervous activity of the wide-field neurons in the lobula complex of the fly. Marmarelis and McCann already showed that it is possible to describe this nervous activity !n terms of a second-order non-linear system (Marmarelis and McCann, 1973), In the present paper, the applicability of the correlation model is described against the background of the architecture of the compound eye, i.e. the retinal mosaic structure of the eye and its retinotopic projection onto the optic 0340-1200/78/0031/0163/$01.20
164
(!)
where q 5 is the autocorrelation of the function x(t). If we eliminate the response to the average light flux C, we get the stationary pattern-specific response R*(ti according to :
(2)
R*(ti= ~ hi(t/) ~ h2(~)~bxx(t/- ~ - At)d~dt7. 0
(2)
o
In the model the inputs are related now to the retinula cells of the fly which means that the needleshaped sensitivity distributions of the inputs have to be replaced by Gaussian functions with half-width AQ. The result of this correction is that R*(t), when calculated for a sine-wave pattern with spatial wavelength 3, must be multiplied by a factor (G6tz, 1964):
,L
g(A 0/2) = exp { - (zc2/(21n2))(A~O/.~)2 }.
Fig. 1. Basic configuration for direction-sensitive movement detection. 1 and 2 inputs. I linear filter of the integrating type D linear filter of the differentiating type. M multiplicative interaction. A averager
ganglia of the fly's visual system. The morphological and physiological properties of the different retina cell systems form the starting point in the understanding and description of the results to be presented9
2. Basic Theory of the Correlation Model
A basic configuration for direction-sensitive movement detection is drawn in Fig. 1. This system consists of two inputs (1 and 2) followed by two linear filters (I and D), a multiplicative interaction (M) and an averager (A). Let us assume - for the time being - that the inputs 1 and 2 sample the environment with needle-shaped spatial sensitivity distributions and that their sample directions are separated by an angle A~b.Ira light pattern L(~b) C +x(q~) (q5= space-coordinate, C=average light flux of the pattern, x(~b)=fluctuating lightflux) moves in the direction 1--,2 with constant angular velocity w, the input-output relationship is easily derived. If the momentaneous signal at input 1 is denoted by x~(t) =C+x(t) the signal at input 2 can be written as x2(t ) = C + x ( t - A t ) with At=A~/w. Let hi(t) and h2(t) represent the impulse responses of the linear filters I and D, then (after calculating the outputs yl(t) and y2(t) of the filters, multiplication of yl (t) and y2(t) and averaging the so-obtained product yl(t).yz(t)) the final response R(t)=yl(t).y2(t ) appears to be (Reichardt and Varjfl, 1959; Reichardt, 1969):
R(t) =yl(t)'y2(t): ~ hl(rl) ~ h2(~_) o
o
9[C 2 + ~xx(tl - ~ - At)]d~dtl,
(1)
(3)
A0 is taken to be equal to the half-width of the sensitivity distributions of receptors belonging to the retinal system 1 6 or 7/8. The processing of the signals by the filters I and D and the multiplicative interaction M is thought to be located at the level of the lamina and the medulla. Finally it is assumed that the outputs of a large number of multiplying units [-i.e. products yt(t).yz(t)] are spatially summed by the wide-field neurons (see introduction) at the level of the lobula complex, in other words, the time-averaging action as present in the model is replaced by a spatial summation in the visual system (Bishop et al., 1968).
3. Methods
3.1. Preparation of Animals Experimenting is done on adult blowflies (Calliphora erythrocephaIa M.) of both sexes, In winter we use flies which have been bred in the laboratory. In spring summer and autumn, however, we prefer to use flies that are caught outdoors. The fly is fixed on a perimeter. The micromanipulators which hold the micro-electrodes are mounted in such a way on the perimeter that the fly with the electrodes in position can be rotated around a vertical as well as a horizontal axis through the head of the animal, s~ that it is possible to achieve any desired position of the fly's head with respect to the visual stimulus. Further preparation details have been described elsewhere (Mastebroek et al., 1977 ; Zaagman et al., 1977).
3.2. Generation of Visual Stimuli The continuously moving patterns are generated on a visual display (HP 1311 A, phosphor P 31). The stimulus field is generated by horizontal and vertical sawtooth voltages. The raster repetition frequency is 180 cps. The
165 z-axis voltage (which determines the pattern wave form) is driven directly by a Wavetek 110 function generator in the case of sinewave patterns, Via a set of register circuits driven by the Wavetek 110 other types of patterns can be chosen.
i
The action potentials (spikes) generated by a wide-field neuron in the lobula complex are recorded extracellularly with Pt Ir micro-electrodes, tip diameters ranging from 3-7 g. The interspike intervals are digitized by a general purpose NOVA computer. The digitizing is performed by an interface which counts the number of 100kHz pulses in each interval. All data are stored directly on disc. On-line analysis is done by programs running in the computer which is also used for the final data processing. Only stationary spike activity, i.e. activity in which there is no drift or irregular burst activity, is studied. The statistical criteria concerning the conditions of stationarity which have to be fulfilled by the several aspects of the spike activity ~ are described elsewhere (Mastebroek, 1974).
3.4. Model Calculations, Model Parameters The model presented in Fig. 1 acts as a purely direction sensitive detector when the filters I and D are taken to be an integrating and differentiating filter respectively having equal time constants (Buchner, 1976). This situation is approached in our study by representing I by a filter of the integrating type, impulse response h~(t) = (t/'c~)exp(-t/'Cl) and D by a filter of the differentiating type, impulse response h2(t)=(c~t/z~l)exp(-t/z21 ) - (t~t/~2z) e x p ( - t/~2 ) with ~21 < r22The model inputs 1 and 2 are assumed to be visual inputs of the retinal system 1-6 sampling the environment under an angle A 4) ( = interommatidial angle). The retinal system 7/8 is thought to give no contribution in the motion detection process, an idea which is supported by investigations of Heisenberg and Buchner (1977). The experimental results are fitted by model curves without using a least squares method; i.e. the optimizing of the model curves is obtained by adjusting the parameters via direct visual inspection. The stationary pattern-specific part of the measured spike activity is fitted, using the relations (2) and (3). This part of the response is superimposed on the steady state activity of the neuron which is generated when the stimulus in the visual field does not move. ,
t The spike rate of the neuron is measured in Adrians (1 Adrian = 1 spike.s- i)
I
I
[
[
I
~120 < E ~: 100 LU r~ w Y
3.3. Recording and Data-processing
I
80
~_ 60 i/l
40 0.2
0.4 1 2 4 10 CONTRAST FREQUENCY(c/s)
20
Fig. 2. The response of the movementdetector as a function of the contrast frequency of a moving sine-wave pattern with spatial wavelength 22= 13.7~ Dots : experimental results. Continuous line: model fit
4. Results
We confine ourselves to the presentation of results obtained from recordings fi'om the wide-field neuron which is horizontally selective in its motion detection performance. This neuron, which exhibits a spontaneous dark activity ranging from 5 to 25 Adrian, is activated by stimuli which move horizontally inward, i.e. from back to front. Stimuli moving in the opposite direction suppress the activity of the neuron. General properties concerning the receptive field characteristics of this movement detector have been described in an earlier paper (Zaagman et al., 1977). The stimuli are presented to the frontal part of the eye around the equator. In the model calculations we took for the value of the effective interommatidial angle AqSh: A~bh=l.45 ~ (Zaagman et al., 1977). The value of A~ohas been inferred from measurements of Horridge et al. (1976) on the retinal 1-6 system, yielding: A~ = 1.59 ~ The three experiments to be presented (4.1-4.3) stem from three different flies.
4.1. The Response as a Function of the Contrast Frequency A sine-wave pattern (modulation depth 50%) with spatial wavelength 2 s = 13.7 ~moves in a stimulus field of 47 x 38 ~ The response R(t) of the neuron is estimated as a function of the contrast fi'equency f~=w/2 s. The spontaneous activity of the neuron amounts 20 Adrian. In Fig. 2 the experimental results (dots) are presented together with the model fit (continuous line). The model parameters are: z 1 = 18 ms; z21 = 10ms, z22 = 12 ms ; c~= 1.03,/~ = 1. The experimental results are in good agreement with the model calculations.
166
120
J
4.2. The Response as a Function of the Spatial Wavelength
I
100
A sine-wave p a t t e r n ( m o d u l a t i o n d e p t h 50 %) moves in a stimulus field of 40 x 35 ~ At each spatial wavelength 2 s the velocity of the pattern, w, is a d j u s t e d in such a w a y that the c o n t r a s t frequency fc = w/2~ is k e p t c o n s t a n t at a value fc = 2 cps. The s p o n t a n e o u s activity a m o u n t s to 22 Adrian. In Fig. 3 the e x p e r i m e n t a l results (dots) are fitted well by the m o d e l curve (continuous line). The m o d e l p a r a m e t e r s are : -c1 = 18 ms ; "c2t= 10 ms ; r2a = 12 ms ; c~= 1.025, fi = 1.
rY
rl
< 80 III
~< 6o iii Y
~0 1
[
3
10
30
Xs/Aq0h
Fig. 3. The response of the movement detector as a function of the spatial wavelength of a sine-wave pattern which moves in such a way that the contrast frequency is kept constant at a value f~ = 2 cps. Dots : experime~atal results. Continuous line: model fit
k(
i o
~
[
1
I
1 1I
~
L_j-J--[2]
--_,qo Fig. 4. I and II: Two square wave patterns, ,I,1=8.4~ and 2~i=2.1 ~
! Pattern obtained by a direct superposition of patterns I and II. 3 Pattern obtained as 1 after a phase shift of Zll/4degrees of pattern II. 2 The same pattern as t but for an interchange within each period of 1 of a dark and a bright zone as indicated by the arrows
I
I
I
Z
< 24 E: rl
LLI
Itl [303 I
I
4
10
I
I
40 1100 w ( DEGREES. SEC- )
F i g . 5. T h e r e s p o n s e s o f t h e m o v e m e n t
d e t e c t o r as a f u n c t i o n o f t h e
angular pattern velocity w in the case of the patterns 1, 2, and 3 as depicted in Fig. 4. Squares, triangles and open circles : experimental results. Upper continuous line: model fit to the experiments with patterns i and 3. Lower continuous line: model prediction to the experiment with pattern 2
4.3. A Test Concerning the Correlation Principle of the Model T h e s t a t i o n a r y pattern-specific p a r t of the response, R*(t), is - according to (2) - d e t e r m i n e d b y the value of the a u t o c o r r e l a t i o n function q~xx(At) of the signal at the i n p u t level as generated there b y any m o v i n g stimulus pattern. If this i n p u t signal is periodic with p e r i o d T a n d m e a n value C it can be written in a F o u r i e r series:
x(t)=C + ~ Gsin(27cnt/T +~n).
(4)
n=l
a, a n d Pn being the a m p l i t u d e s a n d phases of the successive p e r i o d i c c o m p o n e n t s . T a k i n g the a u t o c o r relation function of x(t), we o b t a i n :
~xx(At) = C a +89 ~ a 2 cos(27znAt/T).
(5)
n=l
This result has two consequences with respect to the p e r f o r m a n c e of the c o r r e l a t i o n model. First, the m o d e l response in reaction to a m o v i n g p e r i o d i c p a t t e r n consists of a s u p e r p o s i t i o n of the responses to the F o u r i e r c o m p o n e n t s which constitute the p a t t e r n a n d d e p e n d s on the squared a m p l i t u d e s of these components. Second, the phase relations of the p a t t e r n constituting c o m p o n e n t s are lost in the final m o d e l response, i.e. the m o d e l is " p h a s e b l i n d " with respect to phase shifts of p a t t e r n c o m p o n e n t s . In an a t t e m p t to test whether these m o d e l predictions are confirmed by the processing p r o p e r t i e s of m o v i n g p a t t e r n s in the case of the wide-field neuron, we p e r f o r m an e x p e r i m e n t a n a l o g o u s to well k n o w n (behavioural) e x p e r i m e n t s p e r f o r m e d on the beetle Chlorophanus viridis b y Hassenstein a n d R e i c h a r d t (Hassenstein, 1959 ; R e i c h a r d t a n d Varjfi, 1959 ; Varjfi, 1959; Reichardt, 1969). T w o square wave patterns, I a n d II, )Lt= 8.4 ~ a n d )oit= 2.1 ~ (Fig. 4), are s u p e r i m p o s e d directly resulting in p a t t e r n 1, a n d after a p h a s e shift of
167
OJ[pattern "~II, resulting in pattern 3. Still another pattern is made by interchanging in pattern 1 within each period a dark zone with a bright one as indicated by the arrows; this is called pattern 2. The three patterns thus obtained all have the same mean brightness. Patterns 1 and 3 are constituted by the same Fourier components with - partly - different phases. Pattern 2 is composed of another series of Fourier components with their own weights. [For the analysis of this pattern, see Varjfi (1959).] As a result of these pattern properties the model predicts equal responses in reaction to the moving patterns i and 3 respectively, and different responses to the moving pattern 2. In three experiments we presented the three patterns in the order i, 2, and 3 in a stimulus field of 17 x 17~ The responses of the wide-field neuron as a function of pattern velocity w together with the model calculations are depicted in Fig. 5. The spontaneous activity of the neuron is 12 Adrian. The responses in reaction to patterns 1 and 3 approximate one another rather closely, whereas the responses to pattern 2 are significantly different. The model parameters are chosen in such a way that a curve results which fits as well as possible the two sets of measurements belonging to the experiments with patterns 1 and 3. Their values are : z 1 = 36 ms, T21 = 18 ms, %2 =20 ms; e = 1.04 and/~ = 1. With these parameters the model curve which should predict the experimental results obtained with pattern 2 is calculated. The resulting curve fits the experimental results very well indeed. In both model calculations the contributions to the final response of Fourier components with spatial wavelengths 2s< 1.2A4)h are neglected because of the fact that for such spatial wavelengths the contrast transfer factor (3) approximates zero. The experimental results presented in Fig. 5 are - as a whole - in rather good agreement with the theoretical results obtained with the model. ~ii/4 ~
5. Conclusions and Comments
The experimental results presented in this paper were fitted with model curves which have been obtained with a minimum number of parameters. Besides the filter time constants zl, ~21, z22 and the parameters c~and fl we only used the interommatidial angle A q5(being the sampling base of the uniformly distributed system of correlators) and the half-width AO of the sensitivity distributions from the retinula cells of the 1-6 type. From the fact that the model describes and predicts the experimental results in a satisfactory way we conclude that the basic principles of the correlation
model - which was developed on the base of behavioural optomotor experiments by Reichardt and coworkers - do indeed hold for the description of the neural activity at the level of the lobula complex in the fly's visual system. The applicability of the correlation model studied so far forms an incentive to further investigate the transmission and processing of information by these types of wide field elements in the nervous system of the fly.
Acknowledgements. The authors thank their coworkers Mr. B. A. Pijpker and Mr. T. Manders for their valuable technical and experimental contributions to the work presented in this paper.
References Bishop, L.G., Keehn, D.G., McCann, G.D. : Motion detection by interneurons of optic lobes and brain of the flies Calliphora phaenicia and Musca domestica. J. Neurophysiol. 31, 509-525 (1968) Buchner, E.: Elementary movement detectors in an insect visual system. Biol. Cybernetics 24, 85-101 (1976) Dvorak, D.R., Bishop, L.G., Eckert, H.E. : On the identification of movement detectors in the fly optic lobe. J. comp. Physiol. 100, 5-23 (1975) Eckert, H.E.: Response properties of dipteran giant visual interneurons involved in control of optomotor behaviour. Nature 271, 358-360 (1978) G6tz, K.G. : Optomotorische Untersuchung des visuellen Systems einiger Augenmutanten der Fruchtfliege Drosophila. Kybernetik 2, 77-92 (1964) Hassenstein, B. : Optokinetische Wirksamkeit bewegter periodischer Muster. Z. Naturforsch. 14b, 659-674 (1959) Hausen, K.: Functional characterization and anatomical identification of motion sensitive neurons in the lobula plate of the blowfly Calliphora erythrocephala. Z. Naturforsch. 31c, 629-633 (1976) Heisenberg, M., Buchner, E. : The r61e of retinula cell types in visual behaviour of Drosophila melanogaster. J. comp. Physiol. 117, 12%162 (1977) Hengstenberg, R.: Spike responses of "non-spiking" visual interneurones. Nature 270, 338-340 (1977) Horridge, G.A., Mimura, K., Hardie, R.C. : Fly photoreceptors. III. Angular sensitivity as a function of wavelength and the limits of resolution. Proc. R. Soc. (Lond.) B 194, 151-177 (1976) Marmarelis, P.Z., McCann, G.D. : Development and application of white-noise modelling techniques for studies of insect visual nervous systems. Kybernetik 12, 74-89 (1973) Mastebroek, H.A.K. : Stochastic structure of neural activity in the visual system of the blowfly. Thesis. Groningen University 1974 Mastebroek, H.A.K., Zaagman, W.H., Kuiper, J.W. : Intensity and structure of visually evoked neural activity : Rivals in modelling a visual system. Vision Res. 17, 29-35 (1977) Poggio, T., Reichardt, W. : Visual control of orientation behaviour in the fly. II. Towards the underlying neural interactions. Quart. Rev. Biophys. 9, 377-439 (1976) Reichardt, W. : Autokorrelations-Auswertung als Funktionsprinzip des Zentralnervensystems. Z. Naturforsch. 12b, 448457 (1957)
168 Reichardt, W., Varjfi, D.: lJbertragungseigenschaften im Auswertesystem fiir das Bewegungssehen. Z. Naturforsch. 14b, 674-689 (1959) Reichardt, W. : Movement perception in insects. In: Processing of optical data by organisms and by machines. Reichardt, W. (ed.) pp. 465-493. New York-London: Academic Press 1969 Reichardt, W., Poggio, T. : Visual control of orientation behaviour in the fly. I. A quantitative analysis. Quart. Rev. Biophys. 9, 311-377 (1976) Strausfeld, N.J. : Atlas of an insect brain, Berlin-Heidelberg-New York: Springer 1976 Varjti, D.: Optomotorische Reaktionen auf die Bewegung periodischer Helligkeitsmuster. Z. Naturforsch. 14b, 724-735 (1959) Zaagman, W.H.: Some characteristics of the neural activity of directionally selective movement detectors in the visual system of the blowfly. Thesis. Groningen University 1977
Zaagman, W.H., Mastebroek, H.A.K., Buyse, T., Kuiper, J.W.: Receptive field characteristics of a directionally selective movement detector in the visual system of the blowfly. J. comp. Physiol. 116, 39-50 (1977)
Received: August 27, 1978
Prof, Dr. J. W. Kuiper Lab. of General Physics Dept. of Biophysics Westersinge134 9718 CM Groningen The Netherlands
Biol.Cybernetics31, 163 168 (1978)
@ by Springer-Verlag 1978
On the Correlation Model: Performance of a Movement Detecting Neural Element in the Fly Visual System W. H. Zaagman, H. A. K. Mastebroek, and J. W. Kuiper Laboratoryof GeneralPhysics,Departmentof Biophysics,Westersingel34, 9718 CM Groningen,The Netherlands
Abstract. The applicability of the basic principles of the
correlation model to the description of the activity of a movement detecting neuron in the third optic ganglion of the fly's visual system has been investigated. This wide field neuron is supposed to sum the outputs of a large number of correlators (i.e. multiplying units followed "by time averagers) that are distributed over almost the entire eye. The model describes and predicts the experimental results in a satisfactory way if a uniformly distributed system of correlators is assumed. The sampling base of the correlators in this system equals the interommatidial angle A~b. The half width of the spatial sensitivity distribution of the visual inputs of the correlators, A~, is equal to the half width of the retinula cells of the 1-6 system.
1. Introduction
In the study of the information processing in visual neural systems there are two major reasons to work with flies. First, the extensive unravelling of the anatomical architecture of the fly's visual system has yielded a wealth of details fitting together better and better in the resulting picture of the system as a whole (Strausfeld, 1976). The three optic ganglia of the fly, the lamina, the medulla and the lobula complex are built of periodic ordered columns of neurons. The retinal mosaic structure of the compound eye of the fly is mapped retinotopically onto these columns in such a way that each column corresponds to a certain optical axis in the visual field of the compound eye. In the 3rd order ganglion, the lobula complex, giant neurons are found which are functionally involved in the processing of the information-flow generated in this visual system by stimuli which move in the visual field of the eye. These giant elements can be divided into two systems: a
horizontal movement processing system and a vertical movement processing system (Dvorak et al., 1975; Strausfeld, 1976; Hausen, 1976; Hengstenberg, 1977; Eckert, 1978). Related to the giant neurons of these systems, there exist a few wide-field elements having visual fields which equal the visual field of the whole eye (Dvorak et al., 1975; Strausfeld, 1976; Hausen, 1976). These wide field elements may hold a key position in the processing of moving stimuli in the fly's visual system. Second, an overwhelming number of broadly varied (behavioural) optomotor experiments on insects has markedly deepened the understanding of the functioning of the visual system of the fly. On the base of this type of experiments Reichardt developed the so-called correlation model which has been shown to exhibit a strongly predictive character and which plays a fundamental role in the description and investigation of the visual system of insects (Reichardt, 1957, 1969; Hassenstein, 1959; Reichardt and Varjfi, 1959; Varjfi, 1959; Poggio and Reichardt, 1976; Reichardt and Poggio, 1976; Buchner, 1976). The successful application of the correlation model in the description of the fly's behaviour on the one side, together with the growing knowledge of the anatomy of the fly's visual system on the other side, constitute a challenge to investigate in how far the fundamental performances of the correlation model can be retraced in the neural system of the fly. It is the aim of this paper to describe the applicability of the basic principles of the correlation model to the nervous activity of the wide-field neurons in the lobula complex of the fly. Marmarelis and McCann already showed that it is possible to describe this nervous activity !n terms of a second-order non-linear system (Marmarelis and McCann, 1973), In the present paper, the applicability of the correlation model is described against the background of the architecture of the compound eye, i.e. the retinal mosaic structure of the eye and its retinotopic projection onto the optic 0340-1200/78/0031/0163/$01.20
164
(!)
where q 5 is the autocorrelation of the function x(t). If we eliminate the response to the average light flux C, we get the stationary pattern-specific response R*(ti according to :
(2)
R*(ti= ~ hi(t/) ~ h2(~)~bxx(t/- ~ - At)d~dt7. 0
(2)
o
In the model the inputs are related now to the retinula cells of the fly which means that the needleshaped sensitivity distributions of the inputs have to be replaced by Gaussian functions with half-width AQ. The result of this correction is that R*(t), when calculated for a sine-wave pattern with spatial wavelength 3, must be multiplied by a factor (G6tz, 1964):
,L
g(A 0/2) = exp { - (zc2/(21n2))(A~O/.~)2 }.
Fig. 1. Basic configuration for direction-sensitive movement detection. 1 and 2 inputs. I linear filter of the integrating type D linear filter of the differentiating type. M multiplicative interaction. A averager
ganglia of the fly's visual system. The morphological and physiological properties of the different retina cell systems form the starting point in the understanding and description of the results to be presented9
2. Basic Theory of the Correlation Model
A basic configuration for direction-sensitive movement detection is drawn in Fig. 1. This system consists of two inputs (1 and 2) followed by two linear filters (I and D), a multiplicative interaction (M) and an averager (A). Let us assume - for the time being - that the inputs 1 and 2 sample the environment with needle-shaped spatial sensitivity distributions and that their sample directions are separated by an angle A~b.Ira light pattern L(~b) C +x(q~) (q5= space-coordinate, C=average light flux of the pattern, x(~b)=fluctuating lightflux) moves in the direction 1--,2 with constant angular velocity w, the input-output relationship is easily derived. If the momentaneous signal at input 1 is denoted by x~(t) =C+x(t) the signal at input 2 can be written as x2(t ) = C + x ( t - A t ) with At=A~/w. Let hi(t) and h2(t) represent the impulse responses of the linear filters I and D, then (after calculating the outputs yl(t) and y2(t) of the filters, multiplication of yl (t) and y2(t) and averaging the so-obtained product yl(t).yz(t)) the final response R(t)=yl(t).y2(t ) appears to be (Reichardt and Varjfl, 1959; Reichardt, 1969):
R(t) =yl(t)'y2(t): ~ hl(rl) ~ h2(~_) o
o
9[C 2 + ~xx(tl - ~ - At)]d~dtl,
(1)
(3)
A0 is taken to be equal to the half-width of the sensitivity distributions of receptors belonging to the retinal system 1 6 or 7/8. The processing of the signals by the filters I and D and the multiplicative interaction M is thought to be located at the level of the lamina and the medulla. Finally it is assumed that the outputs of a large number of multiplying units [-i.e. products yt(t).yz(t)] are spatially summed by the wide-field neurons (see introduction) at the level of the lobula complex, in other words, the time-averaging action as present in the model is replaced by a spatial summation in the visual system (Bishop et al., 1968).
3. Methods
3.1. Preparation of Animals Experimenting is done on adult blowflies (Calliphora erythrocephaIa M.) of both sexes, In winter we use flies which have been bred in the laboratory. In spring summer and autumn, however, we prefer to use flies that are caught outdoors. The fly is fixed on a perimeter. The micromanipulators which hold the micro-electrodes are mounted in such a way on the perimeter that the fly with the electrodes in position can be rotated around a vertical as well as a horizontal axis through the head of the animal, s~ that it is possible to achieve any desired position of the fly's head with respect to the visual stimulus. Further preparation details have been described elsewhere (Mastebroek et al., 1977 ; Zaagman et al., 1977).
3.2. Generation of Visual Stimuli The continuously moving patterns are generated on a visual display (HP 1311 A, phosphor P 31). The stimulus field is generated by horizontal and vertical sawtooth voltages. The raster repetition frequency is 180 cps. The
165 z-axis voltage (which determines the pattern wave form) is driven directly by a Wavetek 110 function generator in the case of sinewave patterns, Via a set of register circuits driven by the Wavetek 110 other types of patterns can be chosen.
i
The action potentials (spikes) generated by a wide-field neuron in the lobula complex are recorded extracellularly with Pt Ir micro-electrodes, tip diameters ranging from 3-7 g. The interspike intervals are digitized by a general purpose NOVA computer. The digitizing is performed by an interface which counts the number of 100kHz pulses in each interval. All data are stored directly on disc. On-line analysis is done by programs running in the computer which is also used for the final data processing. Only stationary spike activity, i.e. activity in which there is no drift or irregular burst activity, is studied. The statistical criteria concerning the conditions of stationarity which have to be fulfilled by the several aspects of the spike activity ~ are described elsewhere (Mastebroek, 1974).
3.4. Model Calculations, Model Parameters The model presented in Fig. 1 acts as a purely direction sensitive detector when the filters I and D are taken to be an integrating and differentiating filter respectively having equal time constants (Buchner, 1976). This situation is approached in our study by representing I by a filter of the integrating type, impulse response h~(t) = (t/'c~)exp(-t/'Cl) and D by a filter of the differentiating type, impulse response h2(t)=(c~t/z~l)exp(-t/z21 ) - (t~t/~2z) e x p ( - t/~2 ) with ~21 < r22The model inputs 1 and 2 are assumed to be visual inputs of the retinal system 1-6 sampling the environment under an angle A 4) ( = interommatidial angle). The retinal system 7/8 is thought to give no contribution in the motion detection process, an idea which is supported by investigations of Heisenberg and Buchner (1977). The experimental results are fitted by model curves without using a least squares method; i.e. the optimizing of the model curves is obtained by adjusting the parameters via direct visual inspection. The stationary pattern-specific part of the measured spike activity is fitted, using the relations (2) and (3). This part of the response is superimposed on the steady state activity of the neuron which is generated when the stimulus in the visual field does not move. ,
t The spike rate of the neuron is measured in Adrians (1 Adrian = 1 spike.s- i)
I
I
[
[
I
~120 < E ~: 100 LU r~ w Y
3.3. Recording and Data-processing
I
80
~_ 60 i/l
40 0.2
0.4 1 2 4 10 CONTRAST FREQUENCY(c/s)
20
Fig. 2. The response of the movementdetector as a function of the contrast frequency of a moving sine-wave pattern with spatial wavelength 22= 13.7~ Dots : experimental results. Continuous line: model fit
4. Results
We confine ourselves to the presentation of results obtained from recordings fi'om the wide-field neuron which is horizontally selective in its motion detection performance. This neuron, which exhibits a spontaneous dark activity ranging from 5 to 25 Adrian, is activated by stimuli which move horizontally inward, i.e. from back to front. Stimuli moving in the opposite direction suppress the activity of the neuron. General properties concerning the receptive field characteristics of this movement detector have been described in an earlier paper (Zaagman et al., 1977). The stimuli are presented to the frontal part of the eye around the equator. In the model calculations we took for the value of the effective interommatidial angle AqSh: A~bh=l.45 ~ (Zaagman et al., 1977). The value of A~ohas been inferred from measurements of Horridge et al. (1976) on the retinal 1-6 system, yielding: A~ = 1.59 ~ The three experiments to be presented (4.1-4.3) stem from three different flies.
4.1. The Response as a Function of the Contrast Frequency A sine-wave pattern (modulation depth 50%) with spatial wavelength 2 s = 13.7 ~moves in a stimulus field of 47 x 38 ~ The response R(t) of the neuron is estimated as a function of the contrast fi'equency f~=w/2 s. The spontaneous activity of the neuron amounts 20 Adrian. In Fig. 2 the experimental results (dots) are presented together with the model fit (continuous line). The model parameters are: z 1 = 18 ms; z21 = 10ms, z22 = 12 ms ; c~= 1.03,/~ = 1. The experimental results are in good agreement with the model calculations.
166
120
J
4.2. The Response as a Function of the Spatial Wavelength
I
100
A sine-wave p a t t e r n ( m o d u l a t i o n d e p t h 50 %) moves in a stimulus field of 40 x 35 ~ At each spatial wavelength 2 s the velocity of the pattern, w, is a d j u s t e d in such a w a y that the c o n t r a s t frequency fc = w/2~ is k e p t c o n s t a n t at a value fc = 2 cps. The s p o n t a n e o u s activity a m o u n t s to 22 Adrian. In Fig. 3 the e x p e r i m e n t a l results (dots) are fitted well by the m o d e l curve (continuous line). The m o d e l p a r a m e t e r s are : -c1 = 18 ms ; "c2t= 10 ms ; r2a = 12 ms ; c~= 1.025, fi = 1.
rY
rl
< 80 III
~< 6o iii Y
~0 1
[
3
10
30
Xs/Aq0h
Fig. 3. The response of the movement detector as a function of the spatial wavelength of a sine-wave pattern which moves in such a way that the contrast frequency is kept constant at a value f~ = 2 cps. Dots : experime~atal results. Continuous line: model fit
k(
i o
~
[
1
I
1 1I
~
L_j-J--[2]
--_,qo Fig. 4. I and II: Two square wave patterns, ,I,1=8.4~ and 2~i=2.1 ~
! Pattern obtained by a direct superposition of patterns I and II. 3 Pattern obtained as 1 after a phase shift of Zll/4degrees of pattern II. 2 The same pattern as t but for an interchange within each period of 1 of a dark and a bright zone as indicated by the arrows
I
I
I
Z
< 24 E: rl
LLI
Itl [303 I
I
4
10
I
I
40 1100 w ( DEGREES. SEC- )
F i g . 5. T h e r e s p o n s e s o f t h e m o v e m e n t
d e t e c t o r as a f u n c t i o n o f t h e
angular pattern velocity w in the case of the patterns 1, 2, and 3 as depicted in Fig. 4. Squares, triangles and open circles : experimental results. Upper continuous line: model fit to the experiments with patterns i and 3. Lower continuous line: model prediction to the experiment with pattern 2
4.3. A Test Concerning the Correlation Principle of the Model T h e s t a t i o n a r y pattern-specific p a r t of the response, R*(t), is - according to (2) - d e t e r m i n e d b y the value of the a u t o c o r r e l a t i o n function q~xx(At) of the signal at the i n p u t level as generated there b y any m o v i n g stimulus pattern. If this i n p u t signal is periodic with p e r i o d T a n d m e a n value C it can be written in a F o u r i e r series:
x(t)=C + ~ Gsin(27cnt/T +~n).
(4)
n=l
a, a n d Pn being the a m p l i t u d e s a n d phases of the successive p e r i o d i c c o m p o n e n t s . T a k i n g the a u t o c o r relation function of x(t), we o b t a i n :
~xx(At) = C a +89 ~ a 2 cos(27znAt/T).
(5)
n=l
This result has two consequences with respect to the p e r f o r m a n c e of the c o r r e l a t i o n model. First, the m o d e l response in reaction to a m o v i n g p e r i o d i c p a t t e r n consists of a s u p e r p o s i t i o n of the responses to the F o u r i e r c o m p o n e n t s which constitute the p a t t e r n a n d d e p e n d s on the squared a m p l i t u d e s of these components. Second, the phase relations of the p a t t e r n constituting c o m p o n e n t s are lost in the final m o d e l response, i.e. the m o d e l is " p h a s e b l i n d " with respect to phase shifts of p a t t e r n c o m p o n e n t s . In an a t t e m p t to test whether these m o d e l predictions are confirmed by the processing p r o p e r t i e s of m o v i n g p a t t e r n s in the case of the wide-field neuron, we p e r f o r m an e x p e r i m e n t a n a l o g o u s to well k n o w n (behavioural) e x p e r i m e n t s p e r f o r m e d on the beetle Chlorophanus viridis b y Hassenstein a n d R e i c h a r d t (Hassenstein, 1959 ; R e i c h a r d t a n d Varjfi, 1959 ; Varjfi, 1959; Reichardt, 1969). T w o square wave patterns, I a n d II, )Lt= 8.4 ~ a n d )oit= 2.1 ~ (Fig. 4), are s u p e r i m p o s e d directly resulting in p a t t e r n 1, a n d after a p h a s e shift of
167
OJ[pattern "~II, resulting in pattern 3. Still another pattern is made by interchanging in pattern 1 within each period a dark zone with a bright one as indicated by the arrows; this is called pattern 2. The three patterns thus obtained all have the same mean brightness. Patterns 1 and 3 are constituted by the same Fourier components with - partly - different phases. Pattern 2 is composed of another series of Fourier components with their own weights. [For the analysis of this pattern, see Varjfi (1959).] As a result of these pattern properties the model predicts equal responses in reaction to the moving patterns i and 3 respectively, and different responses to the moving pattern 2. In three experiments we presented the three patterns in the order i, 2, and 3 in a stimulus field of 17 x 17~ The responses of the wide-field neuron as a function of pattern velocity w together with the model calculations are depicted in Fig. 5. The spontaneous activity of the neuron is 12 Adrian. The responses in reaction to patterns 1 and 3 approximate one another rather closely, whereas the responses to pattern 2 are significantly different. The model parameters are chosen in such a way that a curve results which fits as well as possible the two sets of measurements belonging to the experiments with patterns 1 and 3. Their values are : z 1 = 36 ms, T21 = 18 ms, %2 =20 ms; e = 1.04 and/~ = 1. With these parameters the model curve which should predict the experimental results obtained with pattern 2 is calculated. The resulting curve fits the experimental results very well indeed. In both model calculations the contributions to the final response of Fourier components with spatial wavelengths 2s< 1.2A4)h are neglected because of the fact that for such spatial wavelengths the contrast transfer factor (3) approximates zero. The experimental results presented in Fig. 5 are - as a whole - in rather good agreement with the theoretical results obtained with the model. ~ii/4 ~
5. Conclusions and Comments
The experimental results presented in this paper were fitted with model curves which have been obtained with a minimum number of parameters. Besides the filter time constants zl, ~21, z22 and the parameters c~and fl we only used the interommatidial angle A q5(being the sampling base of the uniformly distributed system of correlators) and the half-width AO of the sensitivity distributions from the retinula cells of the 1-6 type. From the fact that the model describes and predicts the experimental results in a satisfactory way we conclude that the basic principles of the correlation
model - which was developed on the base of behavioural optomotor experiments by Reichardt and coworkers - do indeed hold for the description of the neural activity at the level of the lobula complex in the fly's visual system. The applicability of the correlation model studied so far forms an incentive to further investigate the transmission and processing of information by these types of wide field elements in the nervous system of the fly.
Acknowledgements. The authors thank their coworkers Mr. B. A. Pijpker and Mr. T. Manders for their valuable technical and experimental contributions to the work presented in this paper.
References Bishop, L.G., Keehn, D.G., McCann, G.D. : Motion detection by interneurons of optic lobes and brain of the flies Calliphora phaenicia and Musca domestica. J. Neurophysiol. 31, 509-525 (1968) Buchner, E.: Elementary movement detectors in an insect visual system. Biol. Cybernetics 24, 85-101 (1976) Dvorak, D.R., Bishop, L.G., Eckert, H.E. : On the identification of movement detectors in the fly optic lobe. J. comp. Physiol. 100, 5-23 (1975) Eckert, H.E.: Response properties of dipteran giant visual interneurons involved in control of optomotor behaviour. Nature 271, 358-360 (1978) G6tz, K.G. : Optomotorische Untersuchung des visuellen Systems einiger Augenmutanten der Fruchtfliege Drosophila. Kybernetik 2, 77-92 (1964) Hassenstein, B. : Optokinetische Wirksamkeit bewegter periodischer Muster. Z. Naturforsch. 14b, 659-674 (1959) Hausen, K.: Functional characterization and anatomical identification of motion sensitive neurons in the lobula plate of the blowfly Calliphora erythrocephala. Z. Naturforsch. 31c, 629-633 (1976) Heisenberg, M., Buchner, E. : The r61e of retinula cell types in visual behaviour of Drosophila melanogaster. J. comp. Physiol. 117, 12%162 (1977) Hengstenberg, R.: Spike responses of "non-spiking" visual interneurones. Nature 270, 338-340 (1977) Horridge, G.A., Mimura, K., Hardie, R.C. : Fly photoreceptors. III. Angular sensitivity as a function of wavelength and the limits of resolution. Proc. R. Soc. (Lond.) B 194, 151-177 (1976) Marmarelis, P.Z., McCann, G.D. : Development and application of white-noise modelling techniques for studies of insect visual nervous systems. Kybernetik 12, 74-89 (1973) Mastebroek, H.A.K. : Stochastic structure of neural activity in the visual system of the blowfly. Thesis. Groningen University 1974 Mastebroek, H.A.K., Zaagman, W.H., Kuiper, J.W. : Intensity and structure of visually evoked neural activity : Rivals in modelling a visual system. Vision Res. 17, 29-35 (1977) Poggio, T., Reichardt, W. : Visual control of orientation behaviour in the fly. II. Towards the underlying neural interactions. Quart. Rev. Biophys. 9, 377-439 (1976) Reichardt, W. : Autokorrelations-Auswertung als Funktionsprinzip des Zentralnervensystems. Z. Naturforsch. 12b, 448457 (1957)
168 Reichardt, W., Varjfi, D.: lJbertragungseigenschaften im Auswertesystem fiir das Bewegungssehen. Z. Naturforsch. 14b, 674-689 (1959) Reichardt, W. : Movement perception in insects. In: Processing of optical data by organisms and by machines. Reichardt, W. (ed.) pp. 465-493. New York-London: Academic Press 1969 Reichardt, W., Poggio, T. : Visual control of orientation behaviour in the fly. I. A quantitative analysis. Quart. Rev. Biophys. 9, 311-377 (1976) Strausfeld, N.J. : Atlas of an insect brain, Berlin-Heidelberg-New York: Springer 1976 Varjti, D.: Optomotorische Reaktionen auf die Bewegung periodischer Helligkeitsmuster. Z. Naturforsch. 14b, 724-735 (1959) Zaagman, W.H.: Some characteristics of the neural activity of directionally selective movement detectors in the visual system of the blowfly. Thesis. Groningen University 1977
Zaagman, W.H., Mastebroek, H.A.K., Buyse, T., Kuiper, J.W.: Receptive field characteristics of a directionally selective movement detector in the visual system of the blowfly. J. comp. Physiol. 116, 39-50 (1977)
Received: August 27, 1978
Prof, Dr. J. W. Kuiper Lab. of General Physics Dept. of Biophysics Westersinge134 9718 CM Groningen The Netherlands
