HHS Public Access Author manuscript Author Manuscript

Curr Biol. Author manuscript; available in PMC 2017 October 24. Published in final edited form as: Curr Biol. 2016 October 24; 26(20): R1062–R1072. doi:10.1016/j.cub.2016.08.003.

Parallel Computations in Insect and Mammalian Visual Motion Processing Damon A. Clark1 and Jonathan B. Demb2 1Department

of Molecular, Cellular, and Developmental Biology and Department of Physics, Yale University, New Haven, CT 06511, USA

Author Manuscript

2Department

of Ophthalmology and Visual Science and Department of Cellular and Molecular Physiology, Yale University, New Haven, CT 06511, USA

Abstract

Author Manuscript

Sensory systems use receptors to extract information from the environment and neural circuits to perform subsequent computations. These computations may be described as algorithms composed of sequential mathematical operations. Comparing these operations across taxa reveals how different neural circuits have evolved to solve the same problem, even when using different mechanisms to implement the underlying math. In this review, we compare how insect and mammalian neural circuits have solved the problem of motion estimation, focusing on the fruit fly Drosophila and the mouse retina. Although the two systems implement computations with grossly different anatomy and molecular mechanisms, the underlying circuits transform light into motion signals with strikingly similar processing steps. These similarities run from photoreceptor gain control and spatiotemporal tuning to ON and OFF pathway structures, motion detection, and computed motion signals. The parallels between the two systems suggest that a limited set of algorithms for estimating motion satisfies both the needs of sighted creatures and the constraints imposed on them by metabolism, anatomy, and the structure and regularities of the visual world.

Introduction

Author Manuscript

Sighted animals use their eyes to process visual information and guide behavior, and a wide range of behaviors depends on estimating motion. To estimate motion, neural circuits in the visual system must solve a specific problem: extract information about the direction and speed of motion in the environment by combining light intensity signals, as detected by photoreceptors, over space and time. Many animals have solved this problem, and by comparing the algorithms and associated neural circuitry across species, we can learn about the breadth of the problem's possible solutions. One can think about these solutions on two levels [1]: algorithmic, that is, the mathematical operations that describe the processing steps that occur when transforming luminance signals into motion signals; and mechanistic, that is, how the mathematical operations are implemented in the neural circuit's hardware, including the neurotransmitters, ion channels,

Correspondence: [email protected] (D.A.C.), [email protected] (J.B.D.).

Clark and Demb

Page 2

Author Manuscript

receptors, and so forth. The mathematical operations that make up the algorithm can be conceptualized independently from the underlying molecular and biophysical properties that produce them. For instance, two circuits might implement the same addition operation using different sets of neurotransmitters and receptors.

Author Manuscript

This review focuses on comparing the mathematical operations in invertebrates and vertebrates that compute visual motion. This computation is a natural choice for study because many animals compute motion signals, and there is a long history of modeling the mathematical operations in motion circuits [2–7]. We draw parallels between an archetypal invertebrate circuit in the fruit fly Drosophila and an archetypal vertebrate motion circuit in the mouse retina. These two systems have among the best understood connectivity and functional characterization of any motion-detection circuits. Other reviews have covered the anatomical and circuit design similarities between fly and mouse retina [8–10]. Here, we focus on comparing the common processing steps and transformations involved in motion detection in both systems (Figure 1). Invertebrates and vertebrates diverged many hundred million years ago [11], so that similarities between their motion processing circuits must be due either to convergent evolution or to the maintenance of a common feature over an extremely long time. Interestingly, in comparing the two motion detecting circuits, we find that the component mathematical processing steps look similar, even though the underlying molecular mechanisms look quite different. This suggests that natural selection favors a relatively narrow set of mathematical solutions to the problem of motion detection, whereas the exact mechanisms employed to implement these operations seem to be less important.

Author Manuscript

Gain Control in Photoreceptors Ensures Transmission of Luminance Information In every visual circuit, photoreceptors detect incoming photons using light-sensitive opsin molecules, and these photoreceptor signals limit what the eye knows about the world. In insects and mammals, the opsins that detect photons have an ancient, shared evolutionary origin [12,13]. The phototransduction cascades downstream of these molecules are different in the two taxa, however, as are the basic responses to light [14,15]. For example, in response to photon absorption, fly photoreceptors depolarize whereas mouse photoreceptors hyperpolarize. Despite the difference in response polarity, all photoreceptors need to perform similar computations and solve similar problems.

Author Manuscript

One major problem derives from the mismatch between the range of light input and the range of photoreceptor output. Photoreceptors translate photon flux into a synaptic release rate, but the number of incoming photons can change by a factor of 103 within a scene [16,17] and by up to a factor of 109 over the course of a day [18]. By contrast, photoreceptors have a much more limited dynamic range for signaling downstream circuitry. For example, vertebrate photoreceptor release rates at a single synapse are limited to ∼102 vesicles/s, or 101 vesicles within a ∼0.1 s integration time of a postsynaptic neuron [19,20]. Even pooling a reasonable number of synapses (101–102) in a postsynaptic neuron cannot increase the dynamic range of photoreceptor outputs to the range of potential inputs. How

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 3

Author Manuscript

can photoreceptors encode such a wide range of photon counts within a relatively narrow range of response levels? Photoreceptors Exhibit Weber's Law

Author Manuscript

Both insect and vertebrate eyes use physical shielding methods to regulate the range of light intensity incident on photoreceptors [21–23]. Because these mechanisms cannot adjust to the full range of input intensities, photoreceptors further adjust their own sensitivity to the mean light intensity of the immediate environment (Figure 2A). As the mean intensity becomes brighter, photoreceptors in both taxa reduce their sensitivity to incoming photons [24–27]. The sensitivity is adjusted so that a photoreceptor's voltage response depends on a fractional change in intensity relative to the mean, or contrast, rather than the absolute change in photons. This property approximates the well-known Weber–Fechner relation, which posits that sensory systems encode fractional changes in input, rather than absolute changes [18,28]. In mice and other vertebrates, there is a further hand-off between cell types suited to different light conditions: rod photoreceptors that encode dim light and cone photoreceptors that encode brighter light intensities [14,29]. Photoreceptors Exhibit Compressive Nonlinearities

Author Manuscript

Once sensitivity is adapted to a given mean intensity, there is a second problem to solve. How should a photoreceptor allocate its dynamic range compared to the immediate distribution of intensities (Figure 2B)? Within an adapted state, photoreceptors in both vertebrates and invertebrates capture a full range of light intensities by employing a compressive nonlinearity to map input intensity onto membrane voltage (and thus onto synaptic transmission). The compressive linearity is so named because it compresses the wide range of inputs onto a smaller range of outputs. The photoreceptor compressive nonlinearities in the two taxa look similar in their efficacy in compression and in their responses to natural time series, in spite of their divergent molecular implementations [30]. In vertebrates, additional adaptation to both the mean and the distribution of contrast occurs in the neurons postsynaptic to photoreceptors [29,31–34].

Author Manuscript

It has been proposed that photoreceptors use their compressive nonlinearity to efficiently encode the typical distribution of inputs in a natural scene [35] (Figure 2B). The most efficient encoding, in this case, means that photoreceptors respond to the distribution of light intensities by using each membrane voltage, in a physiological range, with equal probability. With this encoding strategy, the intensity-response curve would equal the cumulative distribution function of the intensities encountered within the scene. Such an encoding would contain the maximum information about the stimulus, given a limited range of voltages and a stimulus-independent noise source [18]. Thus, photoreceptors could use intensity gain control to shift their response curve to the correct mean input, and a compressive nonlinearity to match their response function to the distribution of inputs around the mean level.

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 4

Author Manuscript

Input Circuits Are Tuned for Specific Spatial and Temporal Structure in Scenes The natural environment contains strong correlations in luminance over both space and time. Correlations in space depend on the spatial structure of objects in the visual world [36,37]. Correlations in time depend on sources of motion on the retina, including motion of objects themselves — such as a tree branch swaying in a breeze — as well as the motion of the animal or eye movements relative to stationary objects. When the photoreceptors move relative to the world, spatial correlations translate into temporal correlations.

Author Manuscript

Inputs to the motion-detection circuits in flies and mice tune their sensitivity to contrasts by performing spatiotemporal filtering. For a given neuron, the filtering can be conceptualized as a receptive field, or the pattern of input over space and time that most readily generates a response. In the spatial dimension, a typical receptive field has an excitatory center region and an inhibitory surround region. The antagonistic relation between center and surround depends on circuits for lateral inhibition (Figure 3A). This interaction tunes the circuit to spatial stimuli at the scale of the center that do not invade the surround, and therefore emphasizes responses to small objects rather than regions of spatially homogeneous contrast. Beyond small objects, lateral inhibition can also emphasize spatial edges, where contrast changes abruptly. Lateral inhibition is widely seen in visual circuits [38–41] and in other sensory systems as well [42–44].

Author Manuscript

In the temporal dimension, a typical receptive field subtracts a past signal from a current signal, so that the cell becomes sensitive to changes in its input, in essence computing the derivative of the ongoing light signal (Figure 3B). This interaction tunes the circuit to respond to certain temporal frequencies or rates of change in the light intensity rather than to contrasts that are unchanging in time. In practice, a neuron's spatial and temporal receptive fields are characterized in the lab as if they exist independently of the adaptation mechanisms described above. In reality, adaptation must occur during the period of response used to measure the receptive field. Receptive fields therefore represent a linearized form of the full nonlinear neural response under specific conditions and likely include components of (nonlinear) adaptation. For example, in models that explicitly include nonlinear gain adaptation, the linear receptive field incorporates components both from direct excitation and from the gain adaptation [45].

Author Manuscript

In the fly visual system, lateral inhibition occurs at the periphery of the visual circuits, in the cells postsynaptic to the photoreceptors [46–49]. It also appears in some downstream cell types [50,51], where unique profiles of spatial tuning suggest additional circuit mechanisms that complement the lateral inhibition inherited from upstream neurons. Temporal derivatives also are taken in cells immediately postsynaptic to the photoreceptors [47,48,52,53],and those derivatives are passed on to subsequent processing steps. Nonlinearities may further modulate how lateral inhibition is inherited through the circuit. In vertebrate retina, there is a vertical excitatory pathway (photoreceptors → bipolar cells → ganglion cells) that is tuned by lateral inhibition at multiple levels [29,54]. Photoreceptors show lateral inhibition, which is mediated by horizontal cells at the first

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 5

Author Manuscript

synaptic layer. Bipolar cells also show lateral inhibition, some inherited from photoreceptors, some mediated by direct inhibition from horizontal cells in the first synaptic layer, and some mediated by amacrine cells at the second synaptic layer. Retinal ganglion cells inherit some lateral inhibition from bipolar cells and receive additional tuning from amacrine cells in the second synaptic layer. Temporal derivatives are taken at the synapse from photoreceptors onto bipolar cells [55,56]. Both bipolar and ganglion cells show additional temporal tuning that develops beyond the photoreceptor signal and depends on a combination of synaptic interactions and intrinsic membrane properties, including voltagegated channels [29].

Author Manuscript

Lateral inhibition yields functional advantages for visual circuits. Visual signals tend to correlate over space and time, so that neighboring photoreceptor signals report redundant information. The concept of predictive coding posits that each photoreceptor could predict some amount of its current response from its neighbors' responses and from its own previous responses. By subtracting from each photoreceptor the signal of its neighbors in space or its own signal in time, a cell can report deviations from that predicted signal [47,57]. This feature removes correlations between signals adjacent in time or space and is referred to as decorrelation.

Author Manuscript

The optimal shape of the inhibitory receptive field depends on both time-scales and lengthscales of luminance correlations, as well as on the level of noise with which signals are measured [47]. In general, conditions with high noise (for example, dim light) require more integration over time or space, while conditions with low noise require less integration. Hence, strong lateral inhibition and fast temporal integration are observed under brighter conditions with relatively high signal-to-noise [34,47,58–61]. Accordingly, and in spite of the different phototransduction cascades, both insect and vertebrate photoreceptors respond to light more quickly at higher luminance levels (Figure 3C; reviewed in [45]) [27,34,62– 65].

ON–OFF Motifs Split the Visual Circuit into Two Broad Pathways

Author Manuscript

The photoreceptor signal divides into ON and OFF channels at the first synapse, forming a fundamental organizing principle across species [66,67]. ON and OFF channels split visual information into two pathways: the ON channel responds to contrast increments and the OFF channel responds ments and the OFF channel responds to contrast decrements. In flies, the ON channel follows the photoreceptor response, whereas the OFF channel is inverted; the opposite is true in mouse retina. The inversion of one channel's response is not mathematically interesting (at least, to us). More critical is the fact that each channel responds — depolarizes and releases neurotransmitter — to its preferred contrast change but suppresses responses to the non-preferred contrast change a process referred to as ‘half-wave rectification’. Specifically, each channel rectifies its response so that the ON channel reports information about contrast increments but loses information about contrast decrements, while the OFF channel reports information about contrast decrements but not increments (Figure 4A). In vertebrate retina, the rectification is commonly explained by a low baseline output (low basal neurotransmitter release rate) that cannot go negative [68–70]. Thus, ON and OFF channels report complementary information and one is active when the other is not.

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 6

Author Manuscript

In flies, the ON–OFF split occurs immediately after the photoreceptors, with the neuron types L1 and L2 leading into the ON and OFF pathways, respectively [52,71]. Both L1 and L2 receive inhibitory inputs from photoreceptors, so that they hyperpolarize to light increments [24,49,52,72]. L1's signal is inverted again at its synapse with post-synaptic partners, while L2's signal is not [73,74]. In both L1 and L2 pathways, rectification appears to occur at the synapse from L1 and L2 onto their postsynaptic partners (Figure 4B,C) [52,73], and is enhanced in those postsynaptic cells in their transformation from voltage to calcium [75]. Signals in the ON pathway appear to be both slower and less rectified than those in the OFF pathway [73]. The ON and OFF pathway circuits constitute distinct layers within the fly's second optic ganglion, the medulla [74]. The split at L1 and L2 is not perfectly segregated, as silencing experiments have shown that L1, along with a third neuron, L3, contributes activity to OFF pathway elements and dark edge motion detection [51,76].

Author Manuscript

In mouse retina, the ON–OFF split occurs at the photoreceptor synapse and depends on the postsynaptic glutamate receptors: OFF bipolar cells express ionotropic receptors, whereas ON bipolar cells express metabotropic receptors. Consequently, following photoreceptor glutamate release to light dimming, OFF bipolar cells depolarize whereas ON bipolar cells hyperpolarize [66]. The rectification step occurs at the synaptic release of the ON and OFF bipolar cells onto postsynaptic cells [69,70,77]. Like the fly's ON pathway, the ON pathway signals in mammalian retina are typically less rectified than the OFF pathway signals [77– 80]. ON and OFF bipolar cells project to different layers in the retina and synapse onto largely distinct sets of amacrine and retinal ganglion cells, which themselves are typically classified as ON or OFF cells (or else combined ON–OFF cells), depending on their response properties [81].

Author Manuscript

What purpose does this ON–OFF motif serve for these two visual systems? It may serve a metabolic advantage [82]: if one wished to code for light and dark with a single neural channel, rather than two, then for an average portion of the image, that cell would be forced to remain in the middle of its dynamic range, presumably expending energy to remain there (for example, to release neurotransmitter at a high rate continuously) [83]. With the ON– OFF motif, one channel is active while the other is quiescent, and stimuli near the mean luminance of the scene would cause both channels to become quiescent. This might save energy for the circuit, but one would also have to consider the cost of maintaining two parallel sets of circuitry.

Author Manuscript

A more rigorous version of this argument asked how much information could be transmitted by a pair of ON–OFF channels compared to a pair of ON–ON channels [84]. The analysis showed that an ON–OFF channel pair could transmit more information than an ON–ON channel pair when the mean firing of the combined channels was constrained [84]. Assuming that mean firing rate is a reasonable measure of on-going metabolic cost, the ONOFF pairing apparently outperforms either ON–ON or OFF–OFF pairings. The study found that the metabolic benefit of the ON–OFF pairing becomes even more pronounced for skewed distributions of light inputs, which includes naturalistic contrast distributions [85,86].

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 7

Author Manuscript

Other arguments relate to specific functionality. For instance, the paired ON–OFF channels generate signals that are easier to decode compared to ON–ON channels [84]. Specifically related to motion computation, algorithms that include ON and OFF channels can take advantage of light/dark asymmetries in natural scenes to improve motion signals over computations that treat light and dark symmetrically [87].

Simple Models Give Intuition for Elementary Motion Detection

Author Manuscript

For a circuit to generate a direction-selective signal there are three fundamental requirements [88]. First, the circuit must incorporate information about the contrast at two or more points in space, so that displacements may be detected. Second, the circuit must have a short-term memory, so that inputs can be compared over time. In practice, this means that there must be differential temporal processing at different points in space. Third, to obtain a directionselective average signal, there must be a combination step that nonlinearly integrates visual signals from different points in space and time. If the combination step is linear rather than nonlinear, then the circuit cannot produce a direction-selective average signal. In the case of linear combination, a direction-selective read-out of the circuit may be obtained by applying a nonlinear operator to the output of the linear combination step (for example, a measure of response modulation depth rather than an average over time).

Author Manuscript

In fly and mouse retina, two primary models have been proposed to account for early motion estimation. Both models have two arms that receive visual information from different locations in space and delay those signals differently before they feed into a nonlinear combination step (Figure 5). In the Hassenstein–Reichardt correlator [2], which has been widely applied in insects, the delayed signal and non-delayed signal are multiplied together to enhance the model output when there is motion in the preferred direction. The Hassenstein–Reichardt correlator computes the difference between two of these multiplication steps with opposite preferred directions, resulting in an opponent signal that responds positively to motion in the preferred direction and negatively to motion in the null direction. There are several proposed variants of the Hassenstein– Reichardt correlator that use more complex nonlinear operations [73,89,90], but these models all enhance signals when there is motion in the preferred direction. The Barlow–Levick model was first proposed to explain direction selectivity in rabbit retina [4] and was formulated in abstract logical language. More mechanistic interpretations of the model subtract a delayed signal from an adjacent non-delayed signal and then rectify the difference to suppress responses when there is motion in the circuit's null direction [91].

Author Manuscript

In both flies and mice, direction-selective signals are computed independently in the ON pathway and in the OFF pathway. In neither circuit is it clear whether a Hassenstein– Reichardt correlator-like, Barlow–Levick-like, or some other model best explains the direction-selective signal. In flies, the earliest ON and OFF motion signals are computed in the neurons T4 and T5 [92–94]. These two cell types may be derived from a single ancient neural motion pathway [95]. The two motion detectors receive inputs from non-directionselective ON and OFF rectified interneurons with different temporal processing properties

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 8

Author Manuscript

[73,90]. In the case of T4, these input neurons, according to EM reconstructions, should have slightly displaced spatial receptive fields relative to one another [96]. Both T4 and T5 appear to show signal enhancements to motion in their preferred directions [97]. The input synapses are cholinergic in the case of T5 [98] and may be at least partially cholinergic in the case of T4 [99]. The molecular and biophysical mechanisms for the earliest direction selectivity remain unknown. Following the elementary motion computations in T4 and T5, the horizontal system (HS) and vertical system (VS) neurons pool motion signals from both T4 and T5 over space, receiving opposing inputs from neurons with oppositely-tuned direction-selectivity [100,101].

Author Manuscript

In mice, the earliest ON and OFF motion signals are computed in the neurites that radiate out from the cell bodies of the ON and OFF starburst amacrine cells [102,103]. Starburst amacrine cells receive glutamatergic input from ON and OFF bipolar cells, which are not tuned to direction [103–107]. The direction-selective signals in starburst amacrine cell neurites appear to amplify centrifugal motion (radially outwards) signals cell-autonomously but also show suppression to centripetal motion (radially inwards) signals due to mutually inhibitory GABAergic connections between starburst amacrine cells [102,1108–111]. There may also be a direction-selective contribution to starburst amacrine cell tuning mediated by the temporal order of sustained and transient bipolar cell synapses, at least in the case of OFF starburst amacrine cells [112–114]. The final step in computing direction selectivity occurs in the connections between starburst amacrine cells and direction-selective retinal ganglion cells. The direction-selective ganglion cells collect inhibitory (GABAergic) inputs selectively from starburst amacrine cell neurites that point mostly in one direction (for example, leftward), yielding a direction-selective tuning for the ganglion cell to the opposite direction (for example, rightward) [115–117].

Author Manuscript

Spatiotemporal Correlations in Visual Scenes Are Signatures of Motion

Author Manuscript

Visual motion creates many spatiotemporal correlations across the retina. Motion-estimating algorithms detect these correlations, which act as signatures of the motion [7,118]. The simplest motion-dependent correlations exist between pairs of points in time and space. The Hassenstein–Reichardt correlator model estimates motion by computing these pairwise correlations with its multiplication step. The Barlow–Levick model estimates motion by computing correlations through its nonlinearity. To see this, one can expand the Barlow– Levick nonlinearity to isolate polynomial combinations of inputs (Figure 5C; Box 1) [119]. It becomes clear that, to lowest order, the Barlow–Levick model is sensitive to pairwise correlations with the opposite orientation from a Hassenstein–Reichardt correlator multiplier with the same preferred-direction. In fact, if only the first direction-selective term of the Barlow–Levick model is retained, the Barlow–Levick model resembles just the opponent half of the full Hassenstein–Reichardt correlator's two opponent multipliers. In natural scenes, motion can contain correlations beyond those computed by only multiplying or squaring inputs [120]. These arise from spatial structures like moving edges, from contrast asymmetries, and from moving objects that occlude a background. An optimal observer may take advantage of these higher order correlations to enhance motion estimates relative to simply detecting pairwise correlations [7,118]. In particular, triplet correlations —

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 9

Author Manuscript

the products between contrasts at three points in space and time — can improve motion estimates and can be computed by choosing nonlinearities other than pure multiplication in a detector [87]. Such a computation is visible, for instance, when the expansion in Figure 5C is continued to include third-degree polynomial terms. Thus, nonlinearities can be tuned to detect specific sets of spatiotemporal correlations [87]. Both flies and humans perceive motion in triplet correlations in the absence of pairwise correlations [121,122], showing that both species employ motion estimation algorithms that are sensitive to these correlations. Indeed, the fly's behavioral response to triplet correlations matches the responses of models tuned to perform well with natural scene inputs [87]. One way for a system to gain access to these higher order correlations is to treat positive and negative contrasts differently, as implemented by ON and OFF channels [87,121,123].

Author Manuscript

Motion Signals Are Structured Similarly in Flies and Mice Motion signals are similarly structured in both the fly visual system and the mouse retina: both systems compute motion using four cell types tuned to the four cardinal directions; both systems immediately add together ON and OFF motion signals to generate composite motion signals; and both systems create early motion signals that appear closely tuned to behaviors that cancel or minimize slip of images across the retina. Motion Signals Are Tuned to Four Cardinal Directions

Author Manuscript

Both mice and flies generate early motion signals that are tuned to the four cardinal directions. In flies, at each point in space, there exist four T4 neurons and four T5 neurons, whose axons stratify to four layers of the lobula plate that are tuned to the four cardinal directions [93,124,125]. In mouse retina, the ON-OFF direction-selective retinal ganglion cells comprise four functional types, one tuned for motion in each of the cardinal directions [109]. Each type forms a mosaic of cells that covers the retina completely [126,127].

Author Manuscript

The tuning need not be organized into cardinal directions. In flies, the columnar arrangement preceding motion computation is hexagonal, not square, so computing cardinal direction motion represents a significant transformation of basis. In the vertebrate brain downstream of the retina, there exist neurons tuned for many different motion directions [128–130]. In the starburst amacrine cells themselves, the release sites are near the tips of the neurites; and each neurite is direction-selective according to its orientation relative to the soma (that is, rightward-pointing neurites prefer rightward motion on the retina) [102,103]. There are ∼20 starburst amacrine cell neurites at each point in visual space, so motion is initially represented by neurites with ∼20 preferred directions before the cardinal directions are computed by averaging over direction and space [91,115]. The tuning of one class of direction-selective ganglion cell (the ON–OFF type) for four cardinal directions may be explained by their functional role in guiding eye movements. For example, the four preferred directions apparently align with the four rectus eye muscles [131]. Three groups of ON direction-selective cells with ∼120° separation in preferred directions were hypothesized to align with the semicircular canals of the vestibular system [109].

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 10

Author Manuscript

For both visual systems, representing two-dimensional motion vectors could be performed with only two neurons. For example, one neuron could prefer rightward motion and a second neuron could prefer upward motion. Each neuron would represent its preferred direction with positive deviations from baseline and its null (opposite) direction with negative deviations. Instead, both visual systems evolved to encode a single direction with two neurons. In particular, rightward motion drives a positive signal in a rightward-sensing neuron and an absence of signal in a leftward-sensing one. This complementary encoding is similar in principle to the ON–OFF pathways, in which one pathway is active when the other is not. This organization may be useful for downstream computations [132,133]. In particular, these direction-selective neurons may be organized to accurately encode direction information in the presence of noise [134,135]. Some Motion Signals Recombine ON and OFF Signals

Author Manuscript

In both flies and vertebrate retina,ONand OFF motion signals are recombined in neurons immediately postsynaptic to the motion computation. In flies, T4 and T5 form excitatory synapses directly onto the same population of horizontal and vertical system cells, while also providing oppositely-tuned inhibitory input to the same cells through inhibitory interneurons [100]. These horizontal and vertical system cells integrate the T4 and T5 signals over space but appear to get roughly equal contributions from the two cell types [93]. Some motion sensitive behaviors, like optomotor turning, appear to receive balanced light and dark edge inputs [52]. However, other motion-sensitive behaviors, like slowing and visual loom-dependent behaviors, appear to depend differently on light and dark moving edges [76,136–139].

Author Manuscript

In mice, there are several reported types of direction-selective retinal ganglion cells. The ON–OFF direction-selective retinal ganglion cell, mentioned above (Figure 1C), integrates inputs from both ON and OFF starburst amacrine cells [109]. Other types exist that are either ON or OFF selective [132,140,141]. In the case of recombined signals in fly and mouse, one might ask whether there was a functional purpose to computing the signals separately before the recombination, rather than employing a direction-selective mechanism without ON and OFF pathways, like many models of direction-selectivity [2,3]. For example, the separation of ON and OFF pathways before recombination could generate motion signals sensitive to higher order correlations [87]. In addition to mere separation, the ON and OFF motion detectors could be differently tuned to improve motion estimation under natural conditions. Such tuning differences exist in flies and may relate to natural input statistics [123].

Author Manuscript

Motion Signals Are Tuned to Compensatory Behaviors Both mice and flies exhibit compensatory behaviors when their body moves with respect to the world. Mouse eyes show a vestibulo-ocular response, in which the eyes move smoothly to maintain position as the head is rotated. Flies show an optomotor turning response, in which they turn in the direction of the motion [142,143]. Both behaviors minimize retinal slip of the visual scene and act to stabilize images on the retina. Both the mouse retina and the fly visual system also appear to compute early visual motion signals that are directly tuned to those behaviors.

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 11

Author Manuscript

In flies, neurons directly downstream of the elementary motion detectors perform a weighted sum of different vectorial motion components over space [144,145]. These signals in the fly's horizontal and vertical system cells weight input motion to create matched filters for specific rotations of the fly in space. The horizontal system cells are strongly tuned to detect rotation about the yaw axis, rotations that also elicit strong behavioral optomotor turning responses. Other cell types possess matched filters for rotation about combinations of pitch and roll axes. Such early motion signals, tuned to specific rotations, could speed the reflexive compensatory behaviors that stabilize flight.

Author Manuscript

In mice, one class of ON direction-selective retinal ganglion cells senses light edges [146]. These cells form three subtypes sensitive to motion in three directions spaced at approximately 120°, which are thought to align with the orientation of semicircular canals in the vestibular system [109,147]. Such early motion signals would allow the easy integration of visual and vestibular information in generating compensatory behavior and interpreting visual motion signals.

Outlook

Author Manuscript

In the motion computations performed by fly eye and mouse retina, the mathematical processing steps are similar, in spite of differences in gross anatomy, circuitry, and neurotransmitters. The processing steps for motion detection could represent convergent evolution on the algorithmic level. Alternatively, the observed processing steps could potentially share a common origin and have remained similar over hundreds of millions of years, while components in each system were swapped and modified (see for instance [148]). Whichever the case, there appears to be a relatively narrow space of algorithms that perform the motion computation well given the regularities of the natural world and the constraints imposed by biology. The different mechanisms that underlie the algorithm in these two systems show that even if the algorithm is highly constrained, its implementation is not. Natural selection apparently acts on the mathematical processing in these circuits, and different implementations may perform equally well.

Author Manuscript

The advantages of some retinal computations can be explained. In particular, at early stages of both fly and mouse retina systems, the gain control and spatiotemporal tuning have principled explanations in terms of coding efficiency [35,47]. Downstream, however, the benefits of the ON–OFF split are only starting to be understood in terms of coding efficiency and their use in downstream computations. Relatively little is known about the functional benefits of the two main classes of motion estimation algorithms (the Hassenstein–Reichardt correlator versus the Barlow–Levick model). Still less is understood about how ON and OFF motion signals might be combined for behavioral or functional purposes. It is interesting to note that paired, complementary signals appear in both the ON–OFF split and in the cardinal direction encoding. Early gain control is typically not included in models of motion computation (for example, see [149]). This appears appropriate when low contrast stimuli are considered, but naturalistic stimuli with higher contrast seem likely to recruit nonlinear effects at the circuit inputs. To understand how the system functions under natural conditions, it will be necessary to probe the system using naturalistic stimuli, and may be necessary to include front-end nonlinearities in models of motion detection.

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 12

Author Manuscript

As fly and mouse retina computations are further dissected, a continued comparison may prove fruitful. For instance, mammalian retina commonly employs cross-over inhibition, where the ON directly inhibits the OFF pathway (or vice versa), whereas this operation has not been observed in the fly eye. Cross-over inhibition enhances an anti-correlation between ON and OFF pathways, which can tune a variety of computational properties, and might be found widely across species [79,150–155]. In general, the comparison of the highly diverged circuits in fly and mouse can tell us about solutions to common problems, as well was how each circuit is tuned to the different stimuli encountered by two very different animals.

Acknowledgments D.A.C. is supported by the Smith Family Foundation, a Searle Scholar Award, a Sloan Research Fellowship, NSF IOS1558103, and NIH R01EY026555. J.B.D. is supported by NIH R01EY014454.

Author Manuscript

References

Author Manuscript Author Manuscript

1. Marr, D.; Poggio, T. AI Memo. Massachussetts Institute of Technology; 1976. From understanding computation to understanding neural circuitry. 2. Hassenstein B, Reichardt W. Systemtheoretische analyse der zeit-, reihenfolgen-und vorzeichenauswertung bei der bewegung-sperzeption des rüsselkäfers Chlorophanus. Zeits Naturforsch. 1956; 11:513–524. 3. Adelson E, Bergen J. Spatiotemporal energy models for the perception of motion. J Opt Am A. 1985; 2:284–299. 4. Barlow H, Levick WR. The mechanism of directionally selective units in rabbit's retina. J Physiol. 1965; 178:477. [PubMed: 5827909] 5. Rust NC, Mante V, Simoncelli EP, Movshon JA. How MT cells analyze the motion of visual patterns. Nat Neurosci. 2006; 9:1421–1431. [PubMed: 17041595] 6. Poggio T, Reichardt W. Considerations on models of movement detection. Biol Cybern. 1973; 13:223–227. 7. Potters M, Bialek W. Statistical mechanics and visual signal processing. J Physique. 1994; 4:1755– 1775. 8. Sanes JR, Zipursky SL. Design principles of insect and vertebrate visual systems. Neuron. 2010; 66:15. [PubMed: 20399726] 9. Borst A, Helmstaedter M. Common circuit design in fly and mammalian motion vision. Nat Neurosci. 2015; 18:1067–1076. [PubMed: 26120965] 10. Borst A, Euler T. Seeing things in motion: models, circuits, and mechanisms. Neuron. 2011; 71:974–994. [PubMed: 21943597] 11. Peterson KJ, Lyons JB, Nowak KS, Takacs CM, Wargo MJ, McPeek MA. Estimating metazoan divergence times with a molecular clock. Proc Natl Acad Sci USA. 2004; 101:6536–6541. [PubMed: 15084738] 12. Goldsmith TH. Optimization, constraint, and history in the evolution of eyes. Q Rev Biol. 1990:281–322. [PubMed: 2146698] 13. Porter ML, Blasic JR, Bok MJ, Cameron EG, Pringle T, Cronin TW, Robinson PR. Shedding new light on opsin evolution. Proc R Soc Lond B. 2012; 279:3–14. 14. Lamb TD. Evolution of phototransduction, vertebrate photoreceptors and retina. Prog Ret Eye Res. 2013; 36:52–119. 15. Hardie RC, Juusola M. Phototransduction in Drosophila. Curr Opin Neurobiol. 2015; 34:37–45. [PubMed: 25638280] 16. Van Hateren J. Processing of natural time series of intensities by the visual system of the blowfly. Vision Res. 1997; 37:3407–3416. [PubMed: 9425553] 17. van Hateren JH, van der Schaaf A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc R Soc Lond B. 1998; 265:359–366.

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 13

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

18. Rieke F, Rudd ME. The challenges natural images pose for visual adaptation. Neuron. 2009; 64:605–616. [PubMed: 20005818] 19. Choi SY, Borghuis B, Rea R, Levitan ES, Sterling P, Kramer RH. Encoding light intensity by the cone photoreceptor synapse. Neuron. 2005; 48:555–562. [PubMed: 16301173] 20. Sheng Z, Choi SY, Dharia A, Li J, Sterling P, Kramer RH. Synaptic Ca2+ in darkness is lower in rods than cones, causing slower tonic release of vesicles. J Neurosci. 2007; 27:5033–5042. [PubMed: 17494689] 21. Franceschini, N. Information Processing in the Visual Systems of Anthropods. Springer; 1972. Pupil and pseudopupil in the compound eye of Drosophila; p. 75-82. 22. Pennesi ME, Lyubarsky AL, Pugh EN. Extreme responsiveness of the pupil of the dark-adapted mouse to steady retinal illumination. Invest Ophthalmol Visual Sci. 1998; 39:2148–2156. [PubMed: 9761294] 23. Boschek GB. On the fine structure of the peripheral retina and lamina ganglionaris of the fly, Musca domestica. Zeit Zellforsch Mikroskop Anatom. 1971; 118:369–409. 24. Laughlin SB, Hardie RC. Common strategies for light adaptation in the peripheral visual systems of fly and dragonfly. J Comp Physiol. 1978; 128:319–340. 25. Burkhardt DA. Light adaptation and photopigment bleaching in cone photoreceptors in situ in the retina of the turtle. J Neurosci. 1994; 14:1091–1105. [PubMed: 8120614] 26. Nikonov SS, Kholodenko R, Lem J, Pugh EN. Physiological features of the S-and M-cone photoreceptors of wild-type mice from single-cell recordings. J Gen Physiol. 2006; 127:359–374. [PubMed: 16567464] 27. Juusola M, Kouvalainen E, Järvilehto M, Weckström M. Contrast gain, signal-to-noise ratio, and linearity in light-adapted blowfly photoreceptors. J Gen Physiol. 1994; 104:593–621. [PubMed: 7807062] 28. Shapley R, Enroth-Cugell C. Visual adaptation and retinal gain controls. Prog Ret Res. 1984; 3:263–346. 29. Demb JB, Singer JH. Functional circuitry of the retina. Annu Rev Vis Sci. 2015; 1:263–289. 30. Van Hateren J, Snippe H. Phototransduction in primate cones and blowfly photoreceptors: different mechanisms, different algorithms, similar response. J Comp Physiol A. 2006; 192:187–197. 31. Baccus S, Meister M. Fast and slow contrast adaptation in retinal circuitry. Neuron. 2002; 36:909– 919. [PubMed: 12467594] 32. Kim K, Rieke F. Temporal contrast adaptation in the input and output signals of salamander retinal ganglion cells. J Neurosci. 2001; 21:287–299. [PubMed: 11150346] 33. Demb JB. Functional circuitry of visual adaptation in the retina. J Physiol. 2008; 586:4377–4384. [PubMed: 18617564] 34. Dunn FA, Lankheet MJ, Rieke F. Light adaptation in cone vision involves switching between receptor and post-receptor sites. Nature. 2007; 449:603–606. [PubMed: 17851533] 35. Laughlin SB. A simple coding procedure enhances a neuron's information capacity. Z Naturforsch. 1981; 36:51. 36. Ruderman DL, Bialek W. Statistics of natural images: Scaling in the woods. Phys Rev Let. 1994; 73:814–817. [PubMed: 10057546] 37. Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annu Rev Neurosci. 2001; 24:1193–1216. [PubMed: 11520932] 38. Hartline H, Ratliff F. Inhibitory interaction of receptor units in the eye of Limulus. J Gen Physiol. 1957; 40:357–376. [PubMed: 13398569] 39. Werblin FS, Dowling JE. Organization of the retina of the mudpuppy, Necturus maculosus. II. Intracellular recording. J Neurophysiol. 1969; 32:339–355. [PubMed: 4306897] 40. Kaneko A. Physiological and morphological identification of horizontal, bipolar and amacrine cells in goldfish retina. J Physiol. 1970; 207:623–633. [PubMed: 5499739] 41. Dacey D, Packer OS, Diller L, Brainard D, Peterson B, Lee B. Center surround receptive field structure of cone bipolar cells in primate retina. Vision Res. 2000; 40:1801–1811. [PubMed: 10837827]

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 14

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

42. Wilson RI, Laurent G. Role of GABAergic inhibition in shaping odor-evoked spatiotemporal patterns in the Drosophila antennal lobe. J Neurosci. 2005; 25:9069–9079. [PubMed: 16207866] 43. Knudsen EI, Konishi M. Center-surround organization of auditory receptive fields in the owl. Science. 1978; 202:778–780. [PubMed: 715444] 44. Sachdev RN, Krause MR, Mazer JA. Surround suppression and sparse coding in visual and barrel cortices. Front Neur Circuits. 2012; 6:43. 45. Clark DA, Benichou R, Meister M, da Silveira RA. Dynamical adaptation in photoreceptors. PLoS Comp Biol. 2013; 9:e1003289. 46. Freifeld L, Clark DA, Schnitzer MJ, Horowitz MA, Clandinin TR. GABAergic lateral interactions tune the early stages of visual processing in Drosophila. Neuron. 2013; 78:1075–1089. [PubMed: 23791198] 47. Srinivasan M, Laughlin S, Dubs A. Predictive coding: a fresh view of inhibition in the retina. Proc R Soc Lond Ser B. 1982; 216:427. [PubMed: 6129637] 48. Hateren J. Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation. J Comp Physiol A. 1992; 171:157–170. 49. Dubs A. The spatial integration of signals in the retina and lamina of the fly compound eye under different conditions of luminance. J Comp Physiol. 1982; 146:321–343. 50. Meier M, Serbe E, Maisak MS, Haag J, Dickson BJ, Borst A. Neural circuit components of the Drosophila OFF motion vision pathway. Curr Biol. 2014; 24:385–392. [PubMed: 24508173] 51. Fisher YE, Leong JC, Sporar K, Ketkar MD, Gohl DM, Clandinin TR, Silies M. A class of visual neurons with wide-field properties is required for local motion detection. Curr Biol. 2015; 25:3178–3189. [PubMed: 26670999] 52. Clark DA, Bursztyn L, Horowitz MA, Schnitzer MJ, Clandinin TR. Defining the computational structure of the motion detector in Drosophila. Neuron. 2011; 70:1165–1177. [PubMed: 21689602] 53. Juusola M, Uusitalo R, Weckström M. Transfer of graded potentials at the photoreceptorinterneuron synapse. J Gen Physiol. 1995; 105:117. [PubMed: 7537323] 54. Thoreson WB, Mangel SC. Lateral interactions in the outer retina. Prog Ret Eye Res. 2012; 31:407–441. 55. Jackman SL, Choi SY, Thoreson WB, Rabl K, Bartoletti TM, Kramer RH. Role of the synaptic ribbon in transmitting the cone light response. Nat Neurosci. 2009; 12:303–310. [PubMed: 19219039] 56. Armstrong-Gold CE, Rieke F. Bandpass filtering at the rod to second-order cell synapse in salamander (Ambystoma tigrinum) retina. J Neurosci. 2003; 23:3796–3806. [PubMed: 12736350] 57. Barlow HB. Possible principles underlying the transformation of sensory messages. Sens Comm. 1961:217–234. 58. Van Hateren J. Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation. J Comp Physiol A. 1992; 171:157–170. 59. Barlow H, Fitzhugh R, Kuffler S. Change of organization in the receptive fields of the cat's retina during dark adaptation. J Physiol. 1957; 137:338. [PubMed: 13463771] 60. Wang YV, Weick M, Demb JB. Spectral and temporal sensitivity of cone-mediated responses in mouse retinal ganglion cells. J Neurosci. 2011; 31:7670–7681. [PubMed: 21613480] 61. Farrow K, Teixeira M, Szikra T, Viney TJ, Balint K, Yonehara K, Roska B. Ambient illumination toggles a neuronal circuit switch in the retina and visual perception at cone threshold. Neuron. 2013; 78:325–338. [PubMed: 23541902] 62. Howard J, Dubs A, Payne R. The dynamics of phototransduction in insects. J Comp Physiol A. 1984; 154:707–718. 63. Daly SJ, Normann RA. Temporal information processing in cones: effects of light adaptation on temporal summation and modulation. Vision Res. 1985; 25:1197–1206. [PubMed: 4071999] 64. Baylor D, Hodgkin A, Lamb T. The electrical response of turtle cones to flashes and steps of light. J Physiol. 1974; 242:685–727. [PubMed: 4449052] 65. Baylor D, Hodgkin A. Changes in time scale and sensitivity in turtle photoreceptors. J Physiol. 1974; 242:729–758. [PubMed: 4449053]

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 15

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

66. Euler T, Haverkamp S, Schubert T, Baden T. Retinal bipolar cells: elementary building blocks of vision. Nat Rev Neurosci. 2014; 15:507–519. [PubMed: 25158357] 67. Westheimer G. The ON-OFF dichotomy in visual processing: From receptors to perception. Prog Ret Eye Res. 2007; 26:636–648. 68. Baccus SA, Ölveczky BP, Manu M, Meister M. A retinal circuit that computes object motion. J Neurosci. 2008; 28:6807–6817. [PubMed: 18596156] 69. Schwartz G, Rieke F. Nonlinear spatial encoding by retinal ganglion cells: when 1+ 1 ≠ 2. J Gen Physiol. 2011; 138:283–290. [PubMed: 21875977] 70. Demb JB, Zaghloul K, Haarsma L, Sterling P. Bipolar cells contribute to nonlinear spatial summation in the brisk-transient (Y) ganglion cell in mammalian retina. J Neurosci. 2001; 21:7447–7454. [PubMed: 11567034] 71. Joesch M, Schnell B, Raghu S, Reiff D, Borst A. ON and OFF pathways in Drosophila motion vision. Nature. 2010; 468:300–304. [PubMed: 21068841] 72. Hardie RC. A histamine-activated chloride channel involved in neurotransmission at a photoreceptor synapse. Nature. 1989; 339:704–706. [PubMed: 2472552] 73. Behnia R, Clark DA, Carter AG, Clandinin TR, Desplan C. Processing properties of ON and OFF pathways for Drosophila motion detection. Nature. 2014; 512:427–430. [PubMed: 25043016] 74. Strother JA, Nern A, Reiser MB. Direct observation of ON and OFF pathways in the Drosophila visual system. Curr Biol. 2014; 24:976–983. [PubMed: 24704075] 75. Yang HH, St-Pierre F, Sun X, Ding X, Lin MZ, Clandinin TR. Subcellular imaging of voltage and calcium signals reveals neural processing in vivo. Cell. 2016; 166:245–257. [PubMed: 27264607] 76. Silies M, Gohl DM, Fisher YE, Freifeld L, Clark DA, Clandinin TR. Modular use of peripheral input channels tunes motion-detecting circuitry. Neuron. 2013; 79:111–127. [PubMed: 23849199] 77. Borghuis BG, Marvin JS, Looger LL, Demb JB. Two-photon imaging of nonlinear glutamate release dynamics at bipolar cell synapses in the mouse Retina. J Neurosci. 2013; 33:10972–10985. [PubMed: 23825403] 78. Zaghloul KA, Boahen K, Demb JB. Different circuits for ON and OFF retinal ganglion cells cause different contrast sensitivities. J Neurosci. 2003; 23:2645–2654. [PubMed: 12684450] 79. Liang Z, Freed MA. The ON pathway rectifies the OFF pathway of the mammalian retina. J Neurosci. 2010; 30:5533–5543. [PubMed: 20410107] 80. Chichilnisky E, Kalmar R. Functional asymmetries in ON and OFF ganglion cells of primate retina. J Neurosci. 2002; 22:2737. [PubMed: 11923439] 81. Baden T, Berens P, Franke K, Rosón MR, Bethge M, Euler T. The functional diversity of retinal ganglion cells in the mouse. Nature. 2016; 529:345–350. [PubMed: 26735013] 82. Sterling P, Freed M. How robust is a neural circuit? Vis Neurosci. 2007; 24:563–571. [PubMed: 17711602] 83. Laughlin SB, van Steveninck RRdR, Anderson JC. The metabolic cost of neural information. Nat Neurosci. 1998; 1:36–41. [PubMed: 10195106] 84. Gjorgjieva J, Sompolinsky H, Meister M. Benefits of pathway splitting in sensory coding. J Neurosci. 2014; 34:12127–12144. [PubMed: 25186757] 85. Ratliff CP, Borghuis BG, Kao YH, Sterling P, Balasubramanian V. Retina is structured to process an excess of darkness in natural scenes. Proc Natl Acad Sci USA. 2010; 107:17368–17373. [PubMed: 20855627] 86. Geisler WS. Visual perception and the statistical properties of natural scenes. Annu Rev Psychol. 2008; 59:167–192. [PubMed: 17705683] 87. Fitzgerald JE, Clark DA. Nonlinear circuits for naturalistic visual motion estimation. eLife. 2015:e09123. [PubMed: 26499494] 88. Borst A, Egelhaaf M. Principles of visual motion detection. Trends Neurosci. 1989; 12:297–306. [PubMed: 2475948] 89. Eichner H, Joesch M, Schnell B, Reiff DF, Borst A. Internal structure of the fly elementary motion detector. Neuron. 2011; 70:1155–1164. [PubMed: 21689601]

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 16

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

90. Serbe E, Meier M, Leonhardt A, Borst A. Comprehensive characterization of the major presynaptic elements to the Drosophila OFF motion detector. Neuron. 2016; 89:829–841. [PubMed: 26853306] 91. Demb JB. Cellular mechanisms for direction selectivity in the retina. Neuron. 2007; 55:179–186. [PubMed: 17640521] 92. Schnell B, Raghu SV, Nern A, Borst A. Columnar cells necessary for motion responses of widefield visual interneurons in Drosophila. J Comp Physiol A. 2012; 198:389–395. 93. Maisak MS, Haag J, Ammer G, Serbe E, Meier M, Leonhardt A, Schilling T, Bahl A, Rubin GM, Nern A, et al. Adirectional tuning map of Drosophila elementary motion detectors. Nature. 2013; 500:212–216. [PubMed: 23925246] 94. Bahl A, Ammer G, Schilling T, Borst A. Object tracking in motion-blind flies. Nat Neurosci. 2013; 16:730–738. [PubMed: 23624513] 95. Shinomiya K, Takemura Sy, Rivlin PK, Plaza SM, Scheffer LK, Meinertzhagen IA. A common evolutionary origin for the ON-and OFF-edge motion detection pathways of the Drosophila visual system. Front Neur Circuits. 2015; 9:33. 96. Takemura, Sy; Bharioke, A.; Lu, Z.; Nern, A.; Vitaladevuni, S.; Rivlin, PK.; Katz, WT.; Olbris, DJ.; Plaza, SM.; Winston, P. A visual motion detection circuit suggested by Drosophila connectomics. Nature. 2013; 500:175–181. [PubMed: 23925240] 97. Fisher YE, Silies M, Clandinin TR. Orientation selectivity sharpens motion detection in drosophila. Neuron. 2015; 88:390–402. [PubMed: 26456048] 98. Shinomiya K, Karuppudurai T, Lin TY, Lu Z, Lee CH, Meinertzhagen IA. Candidate neural substrates for off-edge motion detection in Drosophila. Curr Biol. 2014; 24:1062–1070. [PubMed: 24768048] 99. Hasegawa E, Kitada Y, Kaido M, Takayama R, Awasaki T, Tabata T, Sato M. Concentric zones, cell migration and neuronal circuits in the Drosophila visual center. Development. 2011; 138:983– 993. [PubMed: 21303851] 100. Mauss AS, Pankova K, Arenz A, Nern A, Rubin GM, Borst A. Neural circuit to integrate opposing motions in the visual field. Cell. 2015; 162:351–362. [PubMed: 26186189] 101. Brotz TM, Borst A. Cholinergic and GABAergic receptors on fly tangential cells and their role in visual motion detection. J Neurophysiol. 1996; 76:1786–1799. [PubMed: 8890292] 102. Euler T, Detwiler PB, Denk W. Directionally selective calcium signals in dendrites of starburst amacrine cells. Nature. 2002; 418:845–852. [PubMed: 12192402] 103. Yonehara K, Farrow K, Ghanem A, Hillier D, Balint K, Teixeira M, Jüttner J, Noda M, Neve RL, Conzelmann KK. The first stage of cardinal direction selectivity is localized to the dendrites of retinal ganglion cells. Neuron. 2013; 79:1078–1085. [PubMed: 23973208] 104. Park SJ, Kim IJ, Looger LL, Demb JB, Borghuis BG. Excitatory synaptic inputs to mouse on-off direction-selective retinal ganglion cells lack direction tuning. J Neurosci. 2014; 34:3976–3981. [PubMed: 24623775] 105. Poleg-Polsky A, Diamond JS. NMDA receptors multiplicatively scale visual signals and enhance directional motion discrimination in retinal ganglion cells. Neuron. 2016; 89:1277–1290. [PubMed: 26948896] 106. Poleg-Polsky A, Diamond JS. Retinal circuitry balances contrast tuning of excitation and inhibition to enable reliable computation of direction selectivity. J Neurosci. 2016; 36:5861– 5876. [PubMed: 27225774] 107. Chen M, Lee S, Park SJ, Looger LL, Zhou ZJ. Receptive field properties of bipolar cell axon terminals in direction-selective sublaminas of the mouse retina. J Neurophysiol. 2014; 112:1950– 1962. [PubMed: 25031256] 108. Lee S, Zhou ZJ. The synaptic mechanism of direction selectivity in distal processes of starburst amacrine cells. Neuron. 2006; 51:787–799. [PubMed: 16982423] 109. Vaney DI, Sivyer B, Taylor WR. Direction selectivity in the retina: symmetry and asymmetry in structure and function. Nat Rev Neurosci. 2012; 13:194–208. [PubMed: 22314444] 110. Vlasits AL, Morrie RD, Tran-Van-Minh A, Bleckert A, Gainer CF, DiGregorio DA, Feller MB. A role for synaptic input distribution in a dendritic computation of motion direction in the retina. Neuron. 2016; 89:1317–1330. [PubMed: 26985724] Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 17

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

111. Sethuramanujam S, McLaughlin AJ, Hoggarth A, Schwab DJ, Awatramani GB. A central role for mixed acetylcholine/GABA transmission in direction coding in the retina. Neuron. 2016; 90:1– 14. [PubMed: 27054611] 112. Kim JS, Greene MJ, Zlateski A, Lee K, Richardson M, Turaga SC, Purcaro M, Balkam M, Robinson A, Behabadi BF. Space-time wiring specificity supports direction selectivity in the retina. Nature. 2014; 509:331–336. [PubMed: 24805243] 113. Greene MJ, Kim JS, Seung HS. Analogous convergence of sustained and transient inputs in parallel on and off pathways for retinal motion computation. Cell Rep. 2016; 14:1892–1900. [PubMed: 26904938] 114. Stincic T, Smith RG, Taylor WR. Time-course of EPSCs in on-type starburst amacrine cells is independent of dendritic location. J Physiol. 2016 epub ahead of print. 115. Briggman KL, Helmstaedter M, Denk W. Wiring specificity in the direction-selectivity circuit of the retina. Nature. 2011; 471:183–188. [PubMed: 21390125] 116. Fried SI, Münch TA, Werblin FS. Mechanisms and circuitry underlying directional selectivity in the retina. Nature. 2002; 420:411–414. [PubMed: 12459782] 117. Lee S, Kim K, Zhou ZJ. Role of ACh-GABA cotransmission in detecting image motion and motion direction. Neuron. 2010; 68:1159–1172. [PubMed: 21172616] 118. Fitzgerald JE, Katsov AY, Clandinin TR, Schnitzer MJ. Symmetries in stimulus statistics shape the form of visual motion estimators. Proc Natl Acad Sci USA. 2011; 108:12909–12914. [PubMed: 21768376] 119. Poggio T, Reichardt W. On the representation of multi-input systems: computational properties of polynomial algorithms. Biol Cybern. 1980; 37:167–186. [PubMed: 7417600] 120. Nitzany EI, Victor JD. The statistics of local motion signals in naturalistic movies. J Vis. 2014; 14:10. 121. Clark DA, Fitzgerald JE, Ales JM, Gohl DM, Silies M, Norcia AM, Clandinin TR. Flies and humans share a motion estimation strategy that exploits natural scene statistics. Nat Neurosci. 2014; 17:296–303. [PubMed: 24390225] 122. Hu Q, Victor JD. A set of high-order spatiotemporal stimuli that elicit motion and reverse-phi percepts. J Vis. 2010; 10:9, 1–16. 123. Leonhardt A, Ammer G, Meier M, Serbe E, Bahl A, Borst A. Asymmetry of Drosophila ON and OFF motion detectors enhances real-world velocity estimation. Nat Neurosci. 2016; 19:706–715. [PubMed: 26928063] 124. Buchner E, Buchner S, Bülthoff I. Deoxyglucose mapping of nervous activity induced inDrosophila brain by visual movement. J Comp Physiol A. 1984; 155:471–483. 125. Fischbach K, Dittrich A. The optic lobe of Drosophila melanogaster. I. A Golgi analysis of wildtype structure. Cell Tissue Res. 1989; 258:441–475. 126. Huberman AD, Wei W, Elstrott J, Stafford BK, Feller MB, Barres BA. Genetic identification of an On-Off direction-selective retinal ganglion cell subtype reveals a layer-specific subcortical map of posterior motion. Neuron. 2009; 62:327–334. [PubMed: 19447089] 127. Kay JN, De la Huerta I, Kim IJ, Zhang Y, Yamagata M, Chu MW, Meister M, Sanes JR. Retinal ganglion cells with distinct directional preferences differ in molecular identity, structure, and central projections. J Neurosci. 2011; 31:7753–7762. [PubMed: 21613488] 128. Niell CM, Stryker MP. Highly selective receptive fields in mouse visual cortex. J Neurosci. 2008; 28:7520–7536. [PubMed: 18650330] 129. Marshel JH, Garrett ME, Nauhaus I, Callaway EM. Functional specialization of seven mouse visual cortical areas. Neuron. 2011; 72:1040–1054. [PubMed: 22196338] 130. Sun W, Tan Z, Mensh BD, Ji N. Thalamus provides layer 4 of primary visual cortex with orientation-and direction-tuned inputs. Nat Neurosci. 2016; 19:308–315. [PubMed: 26691829] 131. Oyster C. The analysis of image motion by the rabbit retina. J Physiol. 1968; 199:613. [PubMed: 5710424] 132. Gauvain G, Murphy GJ. Projection-specific characteristics of retinal input to the brain. J Neurosci. 2015; 35:6575–6583. [PubMed: 25904807]

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 18

Author Manuscript Author Manuscript Author Manuscript Author Manuscript

133. Cruz-Martín A, El-Danaf RN, Osakada F, Sriram B, Dhande OS, Nguyen PL, Callaway EM, Ghosh A, Huberman AD. A dedicated circuit links direction-selective retinal ganglion cells to the primary visual cortex. Nature. 2014; 507:358–361. [PubMed: 24572358] 134. Franke F, Fiscella M, Sevelev M, Roska B, Hierlemann A, da Silveira RA. Structures of neural correlation and how they favor coding. Neuron. 2016; 89:409–422. [PubMed: 26796692] 135. Zylberberg J, Cafaro J, Turner MH, Shea-Brown E, Rieke F. Direction-selective circuits shape noise to ensure a precise population code. Neuron. 2016; 89:369–383. [PubMed: 26796691] 136. Katsov A, Clandinin T. Motion processing streams in Drosophila are behaviorally specialized. Neuron. 2008; 59:322–335. [PubMed: 18667159] 137. Holmqvist MH, Srinivasan MV. A visually evoked escape response of the housefly. J Comp Physiol A. 1991; 169:451–459. [PubMed: 1779418] 138. de Vries SE, Clandinin TR. Loom-sensitive neurons link computation to action in the Drosophila visual system. Curr Biol. 2012; 22:353–362. [PubMed: 22305754] 139. Card G, Dickinson MH. Visually mediated motor planning in the escape response of Drosophila. Curr Biol. 2008; 18:1300–1307. [PubMed: 18760606] 140. Kim IJ, Zhang Y, Yamagata M, Meister M, Sanes JR. Molecular identification of a retinal cell type that responds to upward motion. Nature. 2008; 452:478–482. [PubMed: 18368118] 141. Rousso DL, Qiao M, Kagan RD, Yamagata M, Palmiter RD, Sanes JR. Two pairs of ON and OFF retinal ganglion cells are defined by intersectional patterns of transcription factor expression. Cell Rep. 2016; 15:1930–1944. [PubMed: 27210758] 142. Götz K. Optomotorische untersuchung des visuellen systems einiger augenmutanten der fruchtfliege. Drosophila Biol Cybern. 1964; 2:77–92. 143. Götz K, Wenking H. Visual control of locomotion in the walking fruitfly Drosophila. J Comp Physiol A. 1973; 85:235–266. 144. Franz MO, Krapp HG. Wide-field, motion-sensitive neurons and matched filters for optic flow fields. Biol Cybern. 2000; 83:185–197. [PubMed: 11007295] 145. Krapp HG, Hengstenberg R. Estimation of self-motion by optic flow processing in single visual interneurons. Nature. 1996; 384:463–466. [PubMed: 8945473] 146. Sun W, Deng Q, Levick WR, He S. ON direction-selective ganglion cells in the mouse retina. J Physiol. 2006; 576:197–202. [PubMed: 16901944] 147. Dhande OS, Estevez ME, Quattrochi LE, El-Danaf RN, Nguyen PL, Berson DM, Huberman AD. Genetic dissection of retinal inputs to brainstem nuclei controlling image stabilization. J Neurosci. 2013; 33:17797–17813. [PubMed: 24198370] 148. Shubin N, Tabin C, Carroll S. Deep homology and the origins of evolutionary novelty. Nature. 2009; 457:818–823. [PubMed: 19212399] 149. Simoncelli EP, Heeger DJ. A model of neuronal responses in visual area MT. Vision Res. 1998; 38:743–761. [PubMed: 9604103] 150. Werblin FS. Six different roles for crossover inhibition in the retina: correcting the nonlinearities of synaptic transmission. Vis Neurosci. 2010; 27:1–8. [PubMed: 20392301] 151. Roska B, Werblin F. Vertical interactions across ten parallel, stacked representations in the mammalian retina. Nature. 2001; 410:583–587. [PubMed: 11279496] 152. Murphy GJ, Rieke F. Signals and noise in an inhibitory interneuron diverge to control activity in nearby retinal ganglion cells. Nat Neurosci. 2008; 11:318–326. [PubMed: 18223648] 153. Manookin MB, Beaudoin DL, Ernst ZR, Flagel LJ, Demb JB. Disinhibition combines with excitation to extend the operating range of the OFF visual pathway in daylight. J Neurosci. 2008; 28:4136–4150. [PubMed: 18417693] 154. Borghuis BG, Looger LL, Tomita S, Demb JB. Kainate receptors mediate signaling in both transient and sustained OFF bipolar cell pathways in mouse retina. J Neurosci. 2014; 34:6128– 6139. [PubMed: 24790183] 155. Rosa JM, Ruehle S, Ding H, Lagnado L. Crossover inhibition generates sustained visual responses in the inner retina. Neuron. 2016; 90:308–319. [PubMed: 27068790] 156. Payne R, Howard J. Response of an insect photoreceptor: a simple log-normal model. Nature. 1981; 290:415–416.

Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 19

Author Manuscript

Box 1 Expanding a Barlow–Levick Model

Author Manuscript

Following Figure 5, the output of a Barlow–Levick model can be written as the nonlinearity, g, acting on the difference A – B′, where A is the signal from location a, and B′ is the delayed signal from location b. Thus, the response is r = g(A – B′). The nonlinearity g(x) can be expanded as a polynomial in small x (by means of a Taylor expansion about 0 in this case). In general, g(x) ≈ g0 + g1x + g2x2 + …, where the polynomial coefficients gi are determined by derivatives of the function. Into this, one may substitute x = A – B′ to find r ≈ g0 + g1 (A – B′) + g2 (A2 + B′2 − 2AB′) + …. The first direction-selective term in this series is the one that multiplies A and B′, which has the form of the opponent term in the Hassenstein–Reichardt correlator (Figure 5A) when g2 > 0. This condition is met by typical nonlinearities applied in the Barlow–Levick model. The linear and quadratic polynomial terms are illustrated in Figure 5C, without scaling coefficients. This account may be made more rigorous by including different temporal filters for each arm, rather than a perfect delay and non-delay line. Moreover, sensitivity to higher order spatiotemporal correlations are revealed by the cubic and higher terms in this series [87].

Author Manuscript Author Manuscript Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 20

Author Manuscript Figure 1.

Author Manuscript Author Manuscript

Overview of computations and circuits in fly and mouse motion detection pathways. (A) Outline of the parallel processing steps involved in motion detection in mouse retina and fly eye. Initial gain control and lateral inhibition transform absolute light levels into fractional changes in light level, or contrast. Positive and negative contrasts are divided into ON and OFF pathways, motion is computed in each, and the signals are recombined. (B) Circuit outline for the fly visual system. Photoreceptors (PRs) detect light, pass information to the neurons L1, L2, and L3, which feed into a variety of different identified relay cells before motion is computed in the neurons T4 and T5. Horizontal and vertical system (HS and VS) cells recombine the motion signals from T4 and T5. (C) Circuit outline for the mouse retina. Cone photoreceptors detect light, and their signals are passed to ON and OFF bipolar cells, which act as relays to the ON and OFF starburst amacrine cells, where motion signals are computed. Direction-selective ganglion cells integrate non-direction-selective synapses from bipolar cells and direction-selective synapses from starburst cells. For example, a rightwardpreferring ganglion cell (in retinal coordinates) will collect inhibitory (GABAergic) synapses from leftward-preferring starburst neurites (that is, those that point leftward).

Author Manuscript Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 21

Author Manuscript Author Manuscript Figure 2.

Author Manuscript

Gain control in photoreceptors. (A) Sensitivity adjusts to match the range of intensities in the visual environment. Intensity distributions may be dim (dark red curve, top) or bright (light red curve, top). Photoreceptors adjust their sensitivity to match the mean intensity and encode the intensity distribution efficiently. (B) Once adapted to a mean intensity, the photoreceptor response curve can be adjusted to best represent the intensities about that mean. The black curve (bottom) is the cumulative distribution function of the input intensities, and best encodes these intensities. The solid gray curve would not encode the tails of the distribution well, while the dashed gray curve would not make use of the full dynamic range of the response.

Author Manuscript Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 22

Author Manuscript Author Manuscript Figure 3.

Author Manuscript

Spatial and temporal lateral inhibition. (A) The center of a spatial receptive field is antagonized by the surround region. (B) A biphasic temporal receptive field is excited by one polarity in the immediate past and antagonized by the opposite polarity in the distant past. For both spatial and temporal receptive fields, the combined positive and negative weightings tune the signals and make them sensitive to contrast transitions in space and time. The representations illustrate feed-forward mechanisms, but recurrent networks or adaptation mechanisms could also contribute to receptive field properties. (C) Tuning can change with adaptation state, notably in photoreceptors, which respond to pulses of light more quickly in bright backgrounds than in dim backgrounds. Traces show model insect photoreceptor (top) and vertebrate photoreceptor (bottom) responses to a brief flash in bright and dark backgrounds. Models of photoreceptor responses are from [156] and [45].

Author Manuscript Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 23

Author Manuscript Figure 4.

Author Manuscript

ON–OFF pathway motifs. (A) The input signal is high-pass filtered to emphasize changes in contrast, and one copy (OFF) is inverted relative to the other (ON). In the ON pathway, rectification enhances contrast increment signals; whereas in the OFF pathway, rectification enhances contrast decrement signals. (B,C) Neurons at the input to the ON and OFF pathways in the fly (B) and the mouse retina (C). In the fly, the inversion between the two pathways occurs at the second synapse after the photoreceptor. In mouse retina, the inversion of the pathways occurs at the first synapse after the photoreceptor.

Author Manuscript Author Manuscript Curr Biol. Author manuscript; available in PMC 2017 October 24.

Clark and Demb

Page 24

Author Manuscript

Figure 5.

Author Manuscript Author Manuscript

Models of elementary motion detection. (A) A diagram of the Hassenstein–Reichardt correlator with a rightward preferred direction. Signals from photoreceptors are split, delayed, and then multiplied, before an anti-symmetric subtraction. An object moving rightward first activates the left photoreceptor a, followed by the right photoreceptor b. The delay on photoreceptor a ensures that the two signals coincide at the multiplication step (in green). The multiplication amplifies the signal. Motion in the opposite direction generates an oppositely signed signal derived from the purple multiplier. (B) A diagram of a Barlow– Levick type model with rightward preferred direction. An object moving rightward first activates photoreceptor a, which passes a positive signal through. When the motion is leftward, the delayed negative signal from photoreceptor b coincides with the positive signal from photoreceptor a and suppresses the output. (C) By expanding the nonlinearity in the Barlow–Levick model in a Taylor series (Box 1), one can rewrite the model as a series of operations with different polynomial orders. The constant term is omitted here, as are the different weightings of each expansion term. The first expansion term is just the difference between inputs, which itself is not direction-selective (DS). The next term squares the inputs before they are added; this term is also not direction-selective, since there is no nonlinear interaction in space and time. The next term multiplies the two inputs and is direction selective; this term is identical to one of the two opponent multipliers in the Hassenstein– Reichardt correlator model. The neglected terms indicated by the ellipsis contain polynomial terms of higher than second degree.

Author Manuscript Curr Biol. Author manuscript; available in PMC 2017 October 24.