Neural Networks 51 (2014) 26–38

Contents lists available at ScienceDirect

Neural Networks journal homepage: www.elsevier.com/locate/neunet

A generalized analog implementation of piecewise linear neuron models using CCII building blocks Hamid Soleimani a , Arash Ahmadi a,∗ , Mohammad Bavandpour b , Ozra Sharifipoor a a

Electrical Engineering Department, Razi University, Kermanshah, Iran

b

Faculty of Electrical Engineering, Sharif University of Technology, Tehran, Iran

article

info

Article history: Received 8 December 2012 Received in revised form 2 November 2013 Accepted 4 December 2013 Keywords: Spiking neural network Piecewise linear model CCII Programmable analog circuit Bifurcation

abstract This paper presents a set of reconfigurable analog implementations of piecewise linear spiking neuron models using second generation current conveyor (CCII) building blocks. With the same topology and circuit elements, without W/L modification which is impossible after circuit fabrication, these circuits can produce different behaviors, similar to the biological neurons, both for a single neuron as well as a network of neurons just by tuning reference current and voltage sources. The models are investigated, in terms of analog implementation feasibility and costs, targeting large scale hardware implementations. Results show that, in order to gain the best performance, area and accuracy; these models can be compromised. Simulation results are presented for different neuron behaviors with CMOS 350 nm technology. © 2013 Elsevier Ltd. All rights reserved.

1. Introduction Spiking neural networks (SNNs) have received considerable attention and an increasing research interest in developing artificial neural networks during the past few years (Gerstner & Kistler, 2002; Hodgkin & Huxley, 1952; Izhikevich, 2001, 2003, 2007), due to their behavioral resemblance to biological neurons. Motivated by biological discoveries, pulse-coupled neural networks with spike-timing are considered as an essential component in biological information processing systems, such as the implementation of both high-level and low-level features of the brain like performing complex pattern recognition, motor control, autonomous learning, adaptability, robustness against noise and fault tolerance. Implementation of these models, targeting different platforms, has been the subject of studies in terms of efficiency and large scale simulations (Andreou, Meitzler, Strohbehn, & Boahen, 1995; Arthur & Boahen, 2011; CamuñasMesa, Acosta-Jiménez, Zamarreño-Ramos, Serrano-Gotarredona, & Linares-Barranco, 2011; Davies, Galluppi, Rast, & Furber, 2012; Indiveri, Chicca, & Douglas, 2006; Serrano-Gotarredona, SerranoGotarredona, Acosta-Jiménez, & Linares-Barranco, 2006; Sharifipour & Ahmadi, 2012; Soleimani, Ahmadi, & Bavandpour, 2012; van Schaik, 2001; Wijekoon & Dudek, 2008; Yamashita & Torikai, 2012). Although digitally implemented simulators are found to be convenient and practical for behavioral study of neural networks,

∗

Corresponding author. Tel.: +98 8314283261; fax: +98 8314283261. E-mail addresses: [email protected], [email protected] (A. Ahmadi).

0893-6080/$ – see front matter © 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.neunet.2013.12.004

they are not suitable for actual biologically plausible systems, or detailed real-time and large-scale simulations of neural systems. Custom digital systems that exploit parallel graphical processing units (GPUs) (Ahmadi & Soleimani, 2011) or field programmable gate arrays (FPGAs) (Soleimani et al., 2012) may offer such capabilities in future, but it is not clear how such systems could be able to approach the density, energy efficiency and resilience of the neurons and synapses which they model. The observation that the brain operates based on analog principles of the physics of neural computation that are fundamentally different from digital principles in traditional computing, has initiated the investigations in the field of analog implementation of neuro-systems. Motivated by these reasons, utilizing well developed electronic components and analog circuits to mimic neurological behaviors, is considered as the main choice for direct implementation of neuro-systems. If one can provide a suitable reconfigurable platform to implement neural structures, very large scale integration (VLSI) implementation can be used for prototyping of the neural models, neural dynamics, network structures, and learning mechanisms to test different theories. In terms of VLSI implementation, analog implementations can replicate neural dynamics down to the ion channels in the neural membrane and are fast and efficient; but they are inflexible and require a long development time. As a midpoint in the design space, reconfigurable platforms can provide compact and flexible solutions for biologically plausible neuro-system designers. Many different neuron models are described by nonlinear Ordinary Differential Equations (ODEs) such as the Hodgkin–Huxley model (Hodgkin & Huxley, 1952) or nonlinear ODEs with

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

state-dependent resets (Gerstner & Kistler, 2002; Izhikevich, 2001, 2003, 2007). These models are based on the bio-chemical inspection of the neuron structure and mostly are expressed in the form of differential equations. Although detailed neuron models (Hodgkin & Huxley, 1952), can imitate most experimental measurements to a high degree of accuracy, they are mostly complicated and difficult to physically implement. Izhikevich (2003), has developed a class of models for spiking neurons, which balances the computational efficiency of integrate and fire (IF) models with the biological plausibility and versatility of Hodgkin–Huxley type models (Hodgkin & Huxley, 1952). An analog circuit for implementation of the Izhikevich neuron model has been reported in the literature (Wijekoon & Dudek, 2008). Although this model cannot exactly implement the standard neuron model responses (Izhikevich, 2003), it allows the implementation of some cortical neuron behaviors. The spiking shapes produced by the circuit are biologically plausible and some of the spiking patterns can be obtained by changing two out of the four parameters in the neuron model (Vc and Vd ). The circuit has a low reconfigurability, because the variable parameters after implementation are only two out of the four and the other parameters are fixed. Recently, a novel reconfigurable analog circuit implementation of the neuron has been presented using a second generation current conveyor (CCII) (Sharifipour & Ahmadi, 2012). This circuit is based on a new piecewise linear (PWL) modification of the Izhikevich model, which can reproduce different dynamic behaviors of the cortical neurons. Due to the regular structure of the circuit using standard building blocks, it has the capability to be realized as an application specific reconfigurable analog device, targeting neural networks. This can be considered as a step towards programmable analog neural integrated circuits. Moreover, the resistant structure against noise and impedance matching in the input and output, make this model a suitable choice for analog implementation of large scale neural networks. In this paper, a generalized analog implementation of piecewise linear neuron models consisting of implementation of third and fourth piecewise linear models using reconfigurable CCII building blocks is proposed.

2. The PWL spiking neuron models In Izhikevich (2003) proposed a model of two coupled differential equations as:

 dv   = 0.04v 2 + 5v + 140 − u + I dt

  du = a(bv − u)

(1)

with the auxiliary reset equations:

 v←c u←u+d

2.1. Second order piecewise linear model The second order piecewise (2PWL) model approximates the quadratic part of the Izhikevich model with two crossed lines. This approximation can be formulated as:

 dv   = k1 |v + k2 | − k3 − u + I dt

  du = a(bv − u).

(3)

dt

This approximation provides three degrees of freedom to achieve the closest behavior to the original model. 2.2. Third order piecewise linear model For the third order piecewise (3PWL) approximation the following function is presented:

 dv   = k1 (|v + k2 | + |v − k2 |) − k3 − u + I dt

  du = a(bv − u).

(4)

dt

This approximation provides three degrees of freedom to achieve the closest behavior to the original model. In terms of implementation, the 3PWL approximation is more expensive compared to the 2PWL, but the behavior of the 3PWL model can be closer to the original model by appropriate choice of the coefficients. 2.3. Fourth order piecewise linear model The proposed fourth order piecewise (4PWL) approximation is formulated as:

 dv   = k2 (|v + k3 | + |v − k3 |) + k1 |v + k4 | − k5 − u + I dt

  du = a(bv − u)

(5)

dt

where k1 , k2 , k3, k4 , k5 similar to the other PWL models, are constant values. This approximation provides five degrees of freedom for achieving the closest behavior to the original model. This model requires more complex circuit implementation compared to the other PWL models, but has a very close behavior compared to the other models. 3. Bifurcation analysis of PWL models

dt

v ≥ 30 mV then

27

(2)

where v represents the membrane potential of the neuron, u represents a membrane recovery variable, which accounts for the activation of K+ ionic currents and inactivation of Na+ ionic currents and it provides negative feedback to v . Parameters a, b, c , d are constant values, describing neuron type. After the spike reaches its apex (30 mV), the membrane voltage and the recovery variable are reset according to the equations above. To improve the computational efficiency of the Izhikevich model, three piecewise linear approximations have been proposed in Soleimani et al. (2012).

This section investigates the qualitative bifurcation analysis of the 2 and 4PWL neuron models and explains their relations to a standard biological neuron model (Izhikevich, 2003) based on the procedure introduced in Yamashita and Torikai (2012). Due to similarity of the analyses across all three models, as a midpoint we have waived the analysis of the 3PWL model. 3.1. Basic neuron responses Resting states: The eigenvalues of the PWL dynamical systems have a negative real part, called the nodal sink, in the states of Fig. 1(a1–a2) therefore the intersection point of the borders attracts any nearby point. This phenomenon is also referred to as a stable resting state (Izhikevich, 2003). Moreover, if both eigenvalues have positive real part, the intersection point repels any nearby point named nodal source in Fig. 1(b1–b2). Such a phenomenon is known as an unstable resting state.

28

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

(2)

(3)

(4)

Fig. 1. Basic behaviors of the 2 and 4PWL spiking neuron models. (a1–a2) Stable resting state for 2 and 4PWL models respectively. (b1–b2) Unstable resting state and stable periodic subthreshold oscillation for 2 and 4PWL models respectively. (c1–c2) Bistability of stable resting state and stable tonic spiking for 2 and 4PWL models respectively.

Subthreshold oscillations: The membrane potential in Fig. 1(b1– b2), continues to oscillate periodically under the spiking threshold. This phenomenon is named a periodic subthreshold oscillation (Izhikevich, 2003). If the periodic subthreshold oscillation attracts any nearby point, it is named a stable periodic subthreshold oscillation. If the periodic subthreshold oscillation repels any nearby point, this is known as an unstable periodic subthreshold oscillation. Tonic spikings: The membrane potential in Fig. 1(c1–c2), continues to fire periodically. This phenomenon is named a tonic spiking (Izhikevich, 2003). If the tonic spiking orbit attracts any nearby point, it is named a stable tonic spiking. If the tonic spiking orbit repels any nearby point, this is known as an unstable tonic spiking. Bistability: Two stable conditions (e.g., Stable Resting State and Stable Tonic State) can be seen in Fig. 1(c1–c2), and also the PWL models exhibit that one of them depends on the initial state. This phenomenon is known as a bistability (Izhikevich, 2003). 3.2. Bifurcation analysis (1) Saddle-node off invariant circle bifurcation: In Fig. 2(a1–a2), the PWL models have a stable resting state and a saddle (in the PWL dynamical system, one eigenvalues is positive and one is negative). In Fig. 2(b1–b2), the stimulation input is increased compared to Fig. 2(a1–a2), and the stable resting state and the saddle collide with each other. In Fig. 2(c1–c2), the stimulation input is further increased, and the stable resting state and the saddle point vanish in the intersection of the borders. This

(5)

(6)

change of phenomena is named the saddle-node off invariant circle bifurcation (Izhikevich, 2003). Hence, we refer to the change of phenomena from Fig. 2(a1–a2) to Fig. 2(c1–c2) as a saddle-node off invariant circle bifurcation. Saddle homoclinic orbit bifurcation: In Fig. 2(f1–f2), the PWL models exhibit a bistability of a stable resting state and a stable tonic spiking, and the models also have an unstable resting state. In Fig. 2(e1–e2), the stimulation input is decreased compared to Fig. 2(f1–f2), and the stable tonic spiking and the unstable resting state collide with each other. In Fig. 2(d1–d2), the stimulation input is further decreased, and the stable tonic spiking vanishes when it is absorbed by the attraction domain of the stable resting state. This change of phenomena is known as the saddle homoclinic orbit bifurcation (Izhikevich, 2003). Hence, we refer to the change of phenomena from Fig. 2(d1–d2) to (f1–f2) as a saddle homoclinic orbit bifurcation. Saddle-node on invariant circle bifurcation: In Fig. 3(c1–c2), there exists a stable tonic spiking. In Fig. 3(b1–b2), the stimulation input is decreased compared to Fig. 3(c1–c2), and the stable tonic spiking includes the break point of the border that touches the border. In Fig. 3(a1–a2), the stimulation input is further decreased, and the stable tonic spiking vanishes when it is absorbed by the attraction domain of the stable resting state. This change of phenomena is named the saddle-node on invariant circle bifurcation (Izhikevich, 2003). Hence, we refer to the change of phenomena from Fig. 3(a1–a2) to (c1–c2) as a saddle-node on invariant circle bifurcation. Supercritical Hopf bifurcation: In Fig. 3(d1–d2), there is a stable fixed point and the PWL models exhibit a corresponding stable resting state. In Fig. 3(e1–e2), the stimulation input is increased. Then, it is seen that the stable fixed point is changed into an unstable fixed point and the stable resting state is changed into an unstable resting state. In Fig. 3(f1–f2), the stimulation input is further increased. Then, it is seen that, the phase plane also has stable periodic points and the PWL models exhibit a stable periodic subthreshold oscillation. This change of phenomena is named the supercritical Hopf bifurcation (Izhikevich, 2003). Hence, we refer to the change of phenomena from Fig. 3(d1–d2) to (f1–f2) as a supercritical Hopf bifurcation. Fold limit cycle type BC bifurcation: In Fig. 4(c1–c2), the PWL models exhibit a bistability of a stable resting state and a stable tonic spiking. In Fig. 4(b1–b2), the stimulation input is decreased compared to Fig. 4(c1–c2), and the attraction domain of the stable resting state touches the stable tonic spiking. In Fig. 4(a1–a2), the stimulation input is further decreased, and the stable tonic spiking vanishes when it is absorbed by the attraction domain of the stable resting state. This change of phenomena is referred to as the fold limit cycle bifurcation (Izhikevich, 2003). Hence, we refer to the change of phenomena from Fig. 4(a1–a2) to (c1–c2) as a fold limit cycle bifurcation. Spiking-threshold-reaching border-collision bifurcation: In Fig. 4(d1–d2), the PWL models exhibit a stable subthreshold oscillation. In Fig. 4(e1–e2), the stimulation input is increased compared to Fig. 4(d1–d2), and the stable sub-threshold oscillation touches the spiking threshold. In Fig. 4(f1–f2), the stimulation input is further increased, and the PWL models exhibits a stable tonic spiking. This change of phenomena is named a typical border-collision bifurcation (Izhikevich, 2003). Hence, we refer to the change of phenomena from Fig. 4(d1–d2) to (f1–f2) as a spiking-threshold-reaching border-collision bifurcation.

4. Time domain error assessment Since the correct dynamic behavior of a mathematical neuron model depends on both the qualitative analysis (bifurcation analysis) as well as error assessment in the time domain, in this

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

29

Fig. 2. Saddle-node off invariant circle and saddle homoclinic orbit bifurcations. (a1–a2) Stable resting state and unstable resting state in 2 and 4PWL respectively. (b1–b2) Saddle-node off invariant circle bifurcation in 2 and 4PWL respectively. (c1–c2) The state resting state and the unstable resting state vanish and the PWL models exhibit a stable tonic spiking in 2 and 4PWL respectively. (d1–d2) Stable resting state. (e1–e2) Saddle-node on invariant circle bifurcation. (f1–f2) Stable tonic spiking.

section we present an error criterion and investigate the accuracy of the models in the time domain. To do this, one needs to understand the relationship between neuron behavior and its equilibrium locus. The refractory phase in membrane potential response strongly depends on the slope of the quadratic part of the original model (the refractory phase is the amount of time it takes for an excitable membrane to be ready for a second stimulus once it returns to its resting state following an excitation). Since in PWL models this quadratic part is replaced with linear approximations, the refractory response of the PWL models must be examined for any affection. Another point, which needs to be taken into account, is the return path of the membrane potential to the u steady state line after neuron excitation. During this phase, the membrane potential rises up to the peak point. The curvature and smoothness of this path affects the excitation form of the membrane potential response. Moreover, the length of this path affects the firing rate frequency in the dynamic behavior. It means that the smaller

the path, the higher the firing rate will be. Based on these initial reviews error values are defined as follows: ERRS : This error is defined as the difference between the slope of the main curve and the PWL models. This error affects the refractory phase response, where the bigger deference makes the refractory phase response slower. This error can be formulated as:

      du  du   ERRS =  − .  dt Original model dt PWL models 

(6)

ERRS for different PWL models are presented in Table 1. Since this error is more important at the points where neuron moves along the excitation path, the slope errors in Table 1 are presented as functions of v . ERRP : This error is defined as the difference between the main curve and PWL models at the lowest point of the curves. The PWL models may shift peaks at the bottom of the curves. This

30

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

Fig. 3. Saddle-node on invariant circle and supercritical Hopf type border-collision bifurcations. (a1–a2) Existence of stable resting state and unstable resting state. (b1–b2) Saddle homoclinic orbit bifurcation. (c1–c2) Bistability of state resting state and stable tonic spiking. (d1–d2) Stable resting state. (e1–e2) Supercritical Hopf bifurcation. (f1–f2) Existence of unstable resting state and stable periodic subthreshold oscillation.

determines the initial excitation for the neuron. This implies, when this error is bigger the neuron requires bigger input stimulus (I) to put the curve in a suitable place for excitation in the u–v plane. In addition, this error has a strong effect on the excitation form of the membrane potential (v). ERRP for different PWL models are presented in Table 1. Moreover, the optimized k coefficients for PWL models based on these time domain error assessments are presented in Table 2. 5. Neuron implementation In general, current mode analog circuit design is attractive because of wide bandwidth, low power consumption, high slew rate and simple circuitry. The second-generation current conveyor (CCII) proposed by Sedra and Smith (1970) has been proven to be functionally flexible and versatile, and can act as an operational amplifier in the voltage mode circuit design. Because of its wide bandwidth and low power consumption, the current conveyor

Table 1 Error formulations for PWL models.

ERRS ERRP

2PWL

3PWL

4PWL

|0.08v + 5 − k1 | |k2 − 16.25|

|0.08v + 5 − 2k1 | |k1 k2 (2−k3 )−16.25|

|0.08v + 5 − (2k2 + k1 )| |2k2 k3 − 16.25|

continues to attract more attention in analog design (Alzaher, Elwan, & Ismail, 2003; Elwan & Soliman, 1996; Gaudet & Gulak, 1997; Grigorescu, 2008; Hwang, Liu, Tu, & Chen, 2009; Premont, Grisel, Abouchi, & Chante, 1998). CCIIs in conjunction with other components can implement several analog basic functions such as integrator, amplifier, multiplier, filter and many other signal processing operations. This generality has made them a serious candidate for programmable analog arrays (Gaudet & Gulak, 1997; Grigorescu, 2008; Premont et al., 1998). The operational model of the CCII± is a three port component (namely X , Y and Z terminals) with the matrix representation of

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

31

Fig. 4. Fold limit cycle and spiking-threshold-reaching border-collision bifurcations. (a1–a2) Stable resting state. (b1–b2) Fold limit cycle bifurcation. (c1–c2) Bistability of stable resting state and stable tonic spiking. (d1–d2) Stable periodic subthreshold oscillation. (e1–e2) Spiking-threshold-reaching border-collision bifurcation. (f1–f2) Stable tonic spiking.

the terminal voltages and currents as: Iy Vx Iz

 

0 1 0

 =

0 0 ±1

0 0 0

Vy Ix . Vz

 

(7)

For more realistic analysis of the proposed circuit, second order effects of CCII on the circuit behavior have been considered. Considering CCII non-idealities, its terminal relationships can be rewritten as: Iy = 0 Vx = (1 + εv )Vy Iz = (1 + εi )Ix



(8)

where εv , εi (εv , εi ≪ 1) denote voltage and current errors, respectively and the positive and negative signs of the current Iz denote the non-inverting CCII+ and inverting CCII−, respectively. The current gain between ports X and Z can be ±1 depending on the type

Fig. 5. Block diagram of CCII±. The current gain between ports X and Z is ±1.

of the current conveyor (CCII+ or CCII−type). A block diagram of the CCII is shown in Fig. 5. The current I is conveyed to output terminal Z such that terminal Z has the characteristics of a current source of value I,

32

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

Table 2 The optimized k coefficients for PWL models based on the error assessment procedure. Neuron type

Tonic spiking Phasic spiking Tonic bursting Phasic bursting Mixed mode Spike frequency adaptation Class 1 Class 2 Spike latency Subthreshold oscillations Resonator Integrator Rebound spike Rebound burst Threshold variability Bistability Depolarizing after-potential Accommodation Inhibition-induced spiking Inhibition-induced bursting Mean error %

2PWL

3PWL

4PWL

ERRp

ERRs

K1

K2

ERRp

ERRs

K1

K2

K3

ERRp

ERRs

K1

K2

K3

3.75 1.75 3.75 3.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 1.75 5.75 1.75 1.75 1.75 1.75 2.25

8.5 7.4 10.8 9.2 8.2 7.8 8.3 10.1 8.3 9.2 9.8 8.7 7.5 8.1 9.9 10.9 7.4 10.8 7.9 8.5 8.865

0.75 0.5 0.625 0.5 0.5 0.375 0.375 0.625 0.625 0.875 0.875 0.875 0.875 0.375 0.375 2 0.625 0.625 0.625 0.625 –

20 18 20 20 18 18 18 18 18 18 18 18 18 18 18 22 18 18 18 18 –

0.3 0.3 0.3 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 1.75 0.5 0.5 0.5 0.5 0.5325

6.6 6.4 6.2 5.1 4.9 5.1 5.2 4.4 6.1 5.5 4.9 6.3 3.4 5.1 5.8 6.1 6.1 4.4 3.2 5.1 5.295

0.625 0.625 0.625 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 1.25 0.5 0.5 0.5 0.5 –

5.8 5.8 5.8 7 7 7 7 7 7 7 7 7 7 7 7 12 7 7 7 7 –

6.4 6.4 6.4 6.5 6.5 6.5 6.5 6.5 6.5 6.5 6.5 6.5 6.5 6.5 6.5 3 6.5 6.5 6.5 6.5 –

0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25

1.8 1.5 1.9 1.4 1.5 1.4 0.7 0.7 1.3 1.9 1.3 1.1 1.6 1.7 0.9 0.9 0.8 1.1 0.7 0.5 1.235

0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 0.375 –

0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 0.75 –

11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 –

Fig. 6. A CMOS realization of the CCII± block. The block constructed by 20 transistors, consists of positive/negative output.

with high output impedance. The potential of X is controlled by that of Y , and is independent of the current being forced into port X . A CMOS realization of the positive/negative CCII is shown in Fig. 6. In this study, the circuit of Fig. 6 is utilized and simulated using 350 nm CMOS technology for the current conveyors. 5.1. Second order piecewise linear neuron circuit The circuit is shown in Fig. 7, consisting of two parts: the membrane circuit and the auxiliary after-spike resting circuit. By tuning k2 and k3 parameters different neuron behaviors can be produced with this circuit. The membrane circuit consists of six resistors, two capacitors, two diodes and a CCII+ realizing Eq. (3). Two state variables v and u are represented by two voltages across capacitors Cv and Cu respectively. Cv is connected to the Y terminal in parallel with R1 , the current source and the input current (representing pre-synaptic inputs). Knowing the fact that the absolute function can be expanded as:

|v + k2 | =

 v + k2 −(v + k2 )

v > −k2 v ≤ −k2 .

Fig. 7. The proposed circuit for second order piecewise linear neuron model. This circuit consists of two parts: the membrane circuit (left part) and the auxiliary afterspike resting circuit identified with a dashed line (right part). The membrane circuit consists of six resistors, two capacitors, two diodes and a CCII+.

the Z terminal is equal to the sum of four currents (of R1 , input, k3 and Cv ). The voltage of the Y terminal is copied to the X terminal, therefore, the total input currents of the X and Z nodes considering the CCII non-idealities, are:

 (1 + εv )v (1 + εv )v + k2 (1 + εv )v − u   IX = + + R R4 R3 v 2   IZ = − I + k 3 + IC v

(10)

R1

where I is the input current, k3 is the tuning current source and IC v is the input current to Cv . According to the functional model of the CCII+, the current of the Z terminal is equal to that of the X terminal resulting in:

(9)

Eq. (9) is implemented using two diodes and the voltage source k2 , where k2 is one of the tuning parameters. If the −k2 voltage falls lower than the membrane voltage, D2 will be open circuit and D1 is short circuit. Writing KCL, the current of the X terminal is equal to the sum of three currents (of R2 , R4 and R3 ) and the current of

R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εv ∼ =εi

−−−→ +

dv dt

εi

Cv

=

1 R4 Cv

(v + k2 ) −

(I − k3 + IC v ).

1 Cv R3

u−

k3 Cv

+

I Cv

(11)

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

33

If −k2 goes higher than membrane voltage (VC v ) then D2 will be short circuit and D1 open circuit. Therefore, the current of the X terminal is equal to two currents (of R2 and R3 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , K3 , R1 ε and R6 . Besides, the C i (I − k3 + IC v ) term is considered, because v εi is divided by Cv , which is a small number in comparison with εi , so the whole term cannot be neglected, and this procedure is done for the rest of the formulas in the paper. Nonetheless, this term just acts as an input current for the model and is omitted by tuning the input current. In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v − u (1 + εv )v  IX = + R2

R3

(12)

v + k2 v  IZ = + − I + k3 + IC v . R1

R6

Fig. 8. The proposed circuit for the third order piecewise linear neuron model. The circuit consists of two parts: the membrane circuit (left part) and the auxiliary resting circuit identified with a dashed line (right part). The membrane circuit consists of eight resistors, two capacitors, four diodes and a CCII.

Since Iz = Ix we will have: R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εv ∼ =εi

−−−→

dv dt

=

−1

1

k3

I

(v + k2 ) − u− + R 6 Cv Cv R3 Cv C  v εi v k2 + I − k3 + IC v − − Cv

R6

(13)

R3 du dt du dt

= =

R3 + R5 R 3 R 5 Cu R3 + R5 R 3 R 5 Cu

u

= + ICu R  5  R5 εv v εv ∼ =0 v−u + −−→ R3 + R5 R3   R5 v−u . R3 + R5

(14)

Comparing Eqs. (3) and (14) indicates that a and b parameters are represented by circuit values as: a=

R3 + R5 R3 R5 Cu

b=

R5 R3 + R5

.

2v |v + k2 | + |v − k2 | = 2k2 −2v



R6

assuming R6 = R4 , then Eq. (3) can be realized by Eqs. (11) and (13). On the other hand, the total input current to the Cu is:

(1 + εv )v − u

Knowing the fact that absolute function in this model can be expanded as:

(15)

To implement Eq. (2) a comparator is required. The comparator is shown in the dashed part of the circuit in Fig. 7. The membrane potential voltage v is applied to the gate of M5 and the firing threshold voltage is applied to the gate of M6 . When Vin is higher than the firing threshold voltage then current flows through R1 from right to left. The input capacitances M1 , M2 will be charged and discharged respectively resulting in a low O2 output voltage and high O1 output voltage. Similarly, when Vin falls lower than the firing threshold voltage, the current flow will be reversed and M1 , M2 will be off. Consequently, voltages O1 and O2 return to their resting voltage level.

(16)

Eq. (16) is implemented using four diodes and two voltage sources of ±k2 , where k2 is one of the tuning parameters. If the k2 voltage falls lower than the membrane voltage, D2 and D4 will be open circuit and D1 and D3 are short circuit. Writing KCL, the current of the X terminal is equal to the sum of three currents (of R2 , R3 , R4 and R7 ) and the current of the Z terminal is equal to the sum of four currents (of R1 , input, k3 and Cv ). The voltage of the Y terminal is copied to the X terminal, therefore, the total input currents of the X and Z nodes considering the CCII non-idealities, are:

 (1 + εv )v (1 + εv )v + k2  +  IX =   R R4 2  (1 + εv )v − k2 (1 + εv )v − u + +  R7 R3   v   IZ = − I + k3 + IC v

(17)

R1

where I is the input current, k3 is the tuning current source and IC v is the input current to Cv , assuming R4 = R7 = R6 = R8 . According to the functional model of the CCII, the current of the Z terminal is equal to that of the X terminal resulting in: R1=R2∥R3

IX = (1 + εi )IZ −−−−−→ εv ≈εi

−−−→

dv dt

=

2v

u

k3

I

− − + R4 Cv C R Cv Cv  v 3  εi 2 v + + I − k3 − IC v . Cv

5.2. Third order piecewise linear neuron circuit The circuit is shown in Fig. 8. The structure of this circuit is much like the second order pricewise linear circuit which consists of two parts: the membrane circuit and the auxiliary resting circuit. Tuning k2 and k3 parameters different neuron behaviors can be produced by this circuit. The membrane circuit consists of eight resistors, two capacitors, four diodes and a CCII realizing Eq. (3). Two state variables v and u are represented by two voltages across capacitors Cv and Cu respectively the same as the previous model. Cv is connected to the Y terminal in parallel with R1 , the current source and the input current (representing pre-synaptic inputs).

v ≥ k2 −k2 < v < k2 v < −k2 .

(18)

R4

If the membrane voltage stays between k2 and −k2 then D2 and D3 will be short circuit and D1 and D4 open circuit. Therefore, the current of the X terminal is equal to three currents (of R2 , R3 and R4 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , k3 , R1 and R6 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v (1 + εv )v − u (1 + εv )v + k2   IX = + + R2

R3

v v − k2   IZ = + − I + k 3 + IC v . R1

R6

R7

(19)

34

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

So we will have: R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εi ≈εv

−−−→

dv dt

=

2k2 R6 Cv

+

I Cv

− +

u Cv R 3

εi

−

k3 (20)

Cv

 I − k3 + IC v +

Cv

k2



R6

.

If −k2 goes higher than membrane voltage (VC v ) then D2 and D4 will be short circuit and D1 and D3 open circuit. Therefore, the current of the X terminal is equal to two currents (of R2 and R3 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , k3 , R1 and R6 and R8 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v − u (1 + εv )v   +  IX = R2

v + k2 v    IZ = + + R1

R3 v − k2

R6

R8

(21)

− I + k3 + IC v .

to the functional model of the CCII, the current of the Z terminal is equal to that of the X terminal resulting in:

So we will have: R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εi ≈εv

−−−→

dv dt

=

−2v

u

k3

Cv

R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→

I

− − + R6 Cv Cv R 3 Cv Cv   εi 2v + I − k3 + IC v − .

(22)

εv ≈εi

−−−→

dv dt

=

R6

Eq. (4) can be realized by Eqs. (18), (20) and (22). The second term in Eq. (4) can be analyzed the same as the second order piecewise model. 5.3. Fourth order piecewise linear neuron circuit The proposed circuit is shown in Fig. 9. Tuning the k3 , k4 , and k5 parameters allows different neuron parameters to be produced by this circuit. The membrane circuit consists of ten resistors, two capacitors, six diodes and a CCII realizing Eq. (5). Two state variables v and u are represented by two voltages across capacitors Cv and Cu respectively. Cv is connected to the Y terminal in parallel with R1 , the current source and the input current (representing pre-synaptic inputs). Knowing the fact that absolute function can be expanded as: k2 (|v + k3 | + |v − k3 |) + k1 |v + k4 |

2k v + k (v + k ) 2 1 4    2k2 v − k1 (v + k4 )   2k2 k3 + k1 (v + k4 ) = 2k2 k3 − k1 (v + k4 )     −2k2 v + k1 (v + k4 ) −2k2 v − k1 (v + k4 )

Fig. 9. The proposed circuit for the fourth order piecewise linear neuron model. The circuit consists of two parts: the membrane circuit (left part) and the auxiliary resting circuit identified with a dashed line (right part). The membrane circuit consists of ten resistors, two capacitors, six diodes and a CCII.

v ≥ k3 && v ≥ −k4 v ≥ k3 && v < −k4 −k3 < v < k3 && v ≥ −k4 −k3 < v < k3 && v < −k4 v < −k3 && v ≥ −k4 v < −k3 && v < −k4 .

v + k4 u k5 I + − − + R 4 Cv R9 Cv R 3 C Cv  v v εi 2v + + + I − k5 . Cv

R4

(25)

R9

If the membrane voltage will be higher than k3 and lower than −k4 then D2 , D4 and D5 will be short circuit and D1 , D3 and D6 open circuit. Therefore, the current of the X terminal is equal to four currents (of R2 , R3 , R4 and R7 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , k3 , R1 and R10 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v − u (1 + εv )v + k3 (1 + εv )v   + +  IX =  R2 R3 R7   (1 + εv )v − k3 +  R4   v + k4 v   I = + −I +k +I . Z

R1

Cv

5

R10

(26)

So we will have: R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ (23)

Eq. (23) is implemented using six diodes and three voltage sources k3 , k4 and k5 . If the k3 and −k4 voltages fall lower than the membrane voltage, D2 , D4 and D6 will be open circuit and D1 , D3 and D5 are short circuit. Writing KCL, the current of the X and Z terminals results in:

 (1 + εv )v (1 + εv )v + k4 (1 + εv )v − k3  + + IX =    R R R4 2 9  (1 + εv )v + k3 (1 + εv )v − u + +  R7 R3   v   IZ = − I + k3 + IC v

2v

(24)

R1

where I is the input current, k5 is the tuning current source and IC v is the input current to Cv , assuming R4 = R6 = R7 = R8 . According

εv ≈εi

−−−→

dv dt

=

2v

(v + k4 ) u I k5 − − + − R10 Cv Cv R3 Cv Cv   εi 2v v + k4 + − + I − k5 + IC v . R 4 Cv

Cv

R4

(27)

R10

If the membrane voltage stays between −k3 and k3 and higher than −k4 then D2 , D3 and D5 will be short circuit and D1 , D4 and D6 open circuit. Therefore, the current of the X terminal is equal to four currents (of R2 , R3 , R7 and R9) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , K3 , R1 and R6 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v − u (1 + εv )v + k3 (1 + εv )v  IX = + +    R R3 R7 2   (1 + εv )v + k4 + R9      IZ = v + v − k3 − I + k5 + IC v . R1

R6

(28)

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

So we will have:

Eq. (5) can be realized by Eqs. (29), (31), (33) and (35). The second term in Eq. (5) can be analyzed the same as the second order piecewise model.

R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εv ≈εi

−−−→

dv dt

=

v + k4 u k5 I + − − + R 9 Cv Cv R 3 Cv Cv   v εi k3 + + I − k5 + IC v . + 2k3

(29)

R 7 Cv

Cv

R7

R9

If the membrane voltage stays between −k3 and k3 and lower than −k4 then D2 , D3 and D6 will be short circuit and D1 , D4 and D5 open circuit. Therefore, the current of the X terminal is equal to three currents (of R2 , R3 and R7 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , K3 , R1 , R6 and R10 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v (1 + εv )v − u (1 + εv )v + k3   + + IX = R2

R3

R7

(30)

v v − k3 v + k4   IZ = + + − I + k5 + IC v . R1

R6

R10

So we will have: R1=R2∥R3

IX = (1 + εi )IZ −−−−−→ εv ≈εi

−−−→

dv dt

=

v + k4 u k5 I − − − + R10 Cv Cv R 3 Cv Cv   εi k3 + + I − k 5 − IC v . 2k3

(31)

R 7 Cv

Cv

R6

If the membrane voltage is lower than −k3 and higher than −k4 then D2 , D4 and D5 will be short circuit and D1 , D3 and D6 open circuit. Therefore, the current of the X terminal is equal to three currents (of R2 , R3 and R9 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , k3 , R1 , R6 and R8 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v − u (1 + εv )v + k4 (1 + εv )v   + + IX = R2

R3

R9

(32)

v v − k3 v + k3   IZ = + + − I + k5 + IC v . R1

R6

R8

So we will have: R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εv ≈εi

−−−→

dv dt

=

−2v

v + k4 u k5 I + − − + R 9 Cv Cv R 3 Cv Cv   εi −2v v + + − k 5 + I − IC v .

(33)

R 6 Cv

Cv

R6

R9

If the membrane voltage is lower than −k3 and −k4 then D2 , D4 and D6 will be short circuit and D1 , D3 and D5 open circuit. Therefore, the current of the X terminal is equal to four currents (of R2 , R3 ) and the current of the Z terminal is equal to the currents of the input source (I ), Cv , k3 , R1 , R6 , R8 and R10 . In this case, the currents of the X and Z nodes are given by:

 (1 + εv )v (1 + εv )v − u   + IX = R2

R3

(34)

v v − k3 v + k3 v + k4   IZ = + + + − I + k5 + IC v . R1

R6

R8

R10

So we will have: R1 =R2 ∥R3

IX = (1 + εi )IZ −−−−−→ εv ≈εi

−−−→

dv dt

=

35

−2v R 6 Cv

+

I Cv

−

v + k4

+

εi

R10 Cv

Cv



−

−2v R6

u Cv R 3

−

−

k5

v + k4 R10

(35)

Cv



− k5 + I − IC v .

6. Results In this section simulation results for sample neurons are presented. Simulations are performed using HSPICE for 350 nm, CMOS technology. In the second order piecewise linear model circuit k1 is fixed and specified by resistors as: R1 = 4.5k, R2 = 9k, R3 = 33k, R5 = 11k, R4 = 16k, R6 = 16k, Cv = 0.33pf and Cu = 2.13pf. The type of the neuron depends on the tuning parameters (k2 and k3 ) also in the third order piecewise linear model circuit k1 is fixed and specified by resistors as: R1 = 4.5k, R2 = 9k, R3 = 38k, R5 = 13k, R4 = 16k, R6 = 16k, R7 = 16k, R8 = 16k, Cv = 0.1pf and Cu = 1.45pf and in the fourth order piecewise linear model circuit k1 and k2 are fixed and specified by resistors as: R1 = 4.5k, R2 = 9k, R3 = 41k, R5 = 13k, R4 = 16k, R6 = 16k, R7 = 16k, R8 = 16k, R9 = 80k, R10 = 80k, Cv = 0.15pf and Cu = 1.32pf. Values for the k parameter, c and d, corresponding to the different neuron behaviors for all modified models, are presented in Table 3. Different output spiking waveforms of the model neuron circuit are depicted in Fig. 10. As is attainable from the results, this circuit can produce a similar behavior of the neuronal dynamics as presented by Izhikevich (2003). It is also notable that k2 and k3 , k4 , k5 are voltage and current sources (see Figs. 7–9) by which various spiking behaviors are produced without any change in the circuit layout, such as transistor resizing. Comparison between the shapes of the output spiking signals of the modified circuits shows that the 4PWL model has the nearest output to the original model but is more complex. As can be seen, overall, this method offers low cost accurate options for reconfigurable analog implementation of the neural dynamics. Table 4 presents a comparison between the proposed circuits and the previously published works. Generally, this method offers a low cost accurate option for reconfigurable analog implementation of the neural dynamics. The variation and mismatch in the fabrication process of transistors in submicron scale design are major concerns, which can become problematic in large scale analog CMOS neuromorphic circuits. In order to minimize the effect of variations and mismatches, the analog VLSI signal processing guidelines proposed by Vittoz (1985) can be adopted. It should be noted that a complete elimination of these mismatches and variations is not possible. Our proposed circuits, as analog CMOS neuromorphic devices, are also susceptible to process variations. In order to show that the proposed design is robust against process variation and transistor mismatch, 1000 Monte Carlo (MC) runs were performed on the proposed circuits. In this way, all transistors in the proposed circuit were exposed to a variation, which changes their absolute parameter values in the typical model. The process parameters that were selected to perform these variations are the transistor threshold voltage and gate oxide thickness (TOX) which are the most important process parameters. These variations were assigned in a random fashion using a Gaussian distribution with appropriate values for the means and standard deviations. The amount of injected variability was determined by the standard deviation of the Gaussian distribution, defined as a percentage of the nominal value of the parameter which has been extracted from the current Berkeley Predictive Technology Model (PTM) (Predictive Technology Model (PTM), 2013) of the N- and P-type MOSFETs in the target technologies. The threshold voltages and gate oxide thickness of the PMOS and NMOS transistors varied according to the sigma of the Gaussian distribution, which can change the threshold voltages by up to 50 mV, and gate oxide thickness by up to about 1 nm. Under such a circumstance,

36

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

Fig. 10. Simulation results for different neuron behaviors produced by implemented circuit. Table 3 The optimized k, c and d parameters for different neuron responses in PWL models. Neuron type

IB CH1 CH2 FS1 FS2 FS3 RS1 RS2 RS3

2PWL

3PWL

4PWL

k2 (V)

k3 (µA)

c (V)

d (V)

k2 (V)

k3 (µA)

c (V)

d (V)

k3 (V)

k4 (V)

k5 (µA)

c (V)

d (V)

0.8 0.69 0.5 0.67 0.74 0.8 0.68 0.66 0.78

75 65 59 83 74 81 73 73 77

−0.75 −0.69 −0.7 −0.87 −0.79 −0.79 −0.83 −0.82 −0.79

2.4 2 2.17 1.9 1.87 1.6 2.4 2.3 2.1

0.66 0.62 0.56 0.65 0.72 0.78 0.65 0.61 0.60

73 67 61 84 76 74 73 75 78

−0.83 −0.66 −0.68 −0.86 −0.79 −0.77 −0.81 −0.80 −0.78

2.4 2.05 2.1 1.92 1.86 1.71 2.3 2.2 2.15

0.66 0.61 0.57 0.67 0.73 0.79 0.66 0.66 0.65

0.6 0.53 0.58 0.55 0.57 0.61 0.6 0.55 0.50

80 67 61 83 78 75 72 76 81

−0.83 −0.65 −0.67 −0.85 −0.8 −0.74 −0.83 −0.82 −0.81

2.2 1.9 2.1 1.91 1.88 1.75 2.2 2.2 2.1

Table 4 The proposed circuits compared with the previous works. Model

No. of transistors

Output types

Spike shape

Energy (J)/spike

Re-configurability

Reference

Integrate and fire FitzHugh–Nagumo Hindmarsh–Rose Morris–Lecar Hindmarsh–Rose Izhikevich 2PWL 3PWL 4PWL

18–20 21 22 20 90 14 36 42 48

Simple spiking Oscillatory Oscillatory Oscillatory Bursting CH Some types All types All types All types

Fair Envelope Envelope Envelope Pulse Good Good Excellent Excellent

3–15 n – – – – 9p 48 p 51 p 56 p

– Weak Weak Weak Weak Mediocre Good Good Good

Indiveri (2003) Linares-Barranco et al. (1991) Patel and DeWeerth (1997) Nakada et al. (2005) Lee et al. (2004) Wijekoon and Dudek (2008) Sharifipour and Ahmadi (2012) This work This work

1000 MC runs were performed on the proposed circuits for PWL models. The simulation results for PWL models are shown in Fig. 11. The process variations lead to slight variations in the frequency of neuron signals but not to spike rejection, which can easily be adjusted by retuning the input currents (I ) even after circuit fabrication. This robustness suggests that the physical implementation of the proposed design would be also robust and work within the expected design boundaries.

The second analysis was performed under similar process variation conditions to the first case, but this time, 1000 MC runs were performed and the resulting Mean Square Error (MSE) was computed for frequency changes. The MSE% is calculated using the following equation: N 1 

(fcenter − fvariation )2 × 100 (36) N i=1 where fcenter is the output frequency without the presence of variation, fvariation is the output frequency with the presence MSE% =

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

37

Fig. 11. Simulation results for 1000 Monte Carlo runs for variation analysis on the proposed circuits. The figure shows that the expected process variation leads to slight variations in the frequency of neuron signals but not to spike rejection, which can easily be adjusted by retuning the input currents (I ) even after circuit fabrication.

Table 5 MSE% results for 1000 MC runs on the proposed PWL models.

References

Neuron type

2PWL

3PWL

4PWL

CH RS FS Mean

3.46% 4.31% 2.94% 3.57%

2.52% 3.68% 1.63% 2.61%

2.87% 3.15% 1.71% 2.57%

of variation and N is the number of samples which have been performed. Simulation results are shown in Table 5. The presented results show that significant variations in the value of transistor parameters result in a need to adjust the circuit input current bias, and despite these deviations in the MSE values for output frequency under process variations, they are easily treatable by retuning the input bias currents. This is an optimization process of the input bias current which results in reaching a minimum MSE value and so the closest possible frequency to the ideal case. Both Fig. 11 and quantitative analysis in Table 5 support the robustness and the controllability of the design in the presence of physical variations. Hence, despite the fact that the proposed design has some susceptibility to process variations, a post-fabrication calibration is possible through retuning the bias currents of the design to achieve a minimal MSE of frequency changes to faithfully reproduce the needed learning behavior.

7. Conclusion This paper presents a generalized analog implementation of piecewise linear neuron models using CCII building blocks. Findings show that these circuits are capable of generating all different neural behaviors. Circuits are suitable for CCII-based field programmable analog arrays. To change the neuron type, the circuit does not need W/L modification, which is impossible after circuit fabrication. Also the resistant structure against noise and impedance matching in the input and output make this model a suitable choice for large scale neural networks. This can be considered as a step towards programmable analog neural integrated circuits.

Ahmadi, A., & Soleimani, H. (2011). A GPU based simulation of multilayer spiking neural networks. In 19th Iranian conference on electrical engineering, ICEE (pp. 1–5). IEEE. Alzaher, H. A., Elwan, H., & Ismail, M. (2003). A CMOS fully balanced secondgeneration current conveyor. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 50(5), 278–287. Andreou, A. G., Meitzler, R. C., Strohbehn, K., & Boahen, K. A. (1995). Analog VLSI neuromorphic image acquisition and pre-processing systems. Neural Networks, 8(6), 1323–1347. Arthur, J. V., & Boahen, K. A. (2011). Silicon-neuron design: a dynamical systems approach. IEEE Transactions on Circuits and Systems I: Regular Papers, 58(4), 1034–1043. Camuñas-Mesa, L., Acosta-Jiménez, A., Zamarreño-Ramos, C., Serrano-Gotarredona, T., & Linares-Barranco, B. (2011). A 32 × 32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 Meps throughput. IEEE Transactions on Circuits and Systems I: Regular Papers, 58(3), 777–790. Davies, S., Galluppi, F., Rast, A. D., & Furber, S. B. (2012). A forecast-based STDP rule suitable for neuromorphic implementation. Neural Networks, 32, 3–14. Elwan, H. O., & Soliman, A. M. (1996). A novel CMOS current conveyor realization with an electronically tunable current mode filter suitable for VLSI. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 43(8), 663–670. Gaudet, V. C., & Gulak, P. G. (1997). CMOS implementation of a current conveyor-based field-programmable analog array. In Conference record of the thirty-first asilomar conference on signals, systems & computers, 1997. Vol. 2 (pp. 1156–1159). IEEE. Gerstner, W., & Kistler, W. M. (2002). Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press. Grigorescu, L. (2008). Amplifiers built with current-conveyors. Romanian Journal of Physics, 53(1–2), 105–109. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of Physiology, 117(3), 500. Hwang, Y. S., Liu, W. H., Tu, S. H., & Chen, J. J. (2009). New building block: multiplication-mode current conveyor. IET Circuits, Devices & Systems, 3(1), 41–48. Indiveri, G. (2003). A low-power adaptive integrate-and-fire neuron circuit. In Proceedings of the 2003 international symposium on circuits and systems, 2003. ISCAS’03 (Vol. 4, pp. IV–820). IEEE. Indiveri, G., Chicca, E., & Douglas, R. (2006). A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Transactions on Neural Networks, 17(1), 211–221. Izhikevich, E. M. (2001). Resonate-and-fire neurons. Neural Networks, 14(5), 883–894. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(5), 1569–1572. Izhikevich, E. (2007). Dynamical systems in neuroscience: the geometry of excitability and bursting. The MIT Press. Lee, Y. J., Lee, J., Kim, Y. B., Ayers, J., Volkovskii, A., Selverston, A., Abarbanel, H., & Rabinovich, M. (2004). Low power real time electronic neuron VLSI design using subthreshold technique. In Proceedings of the 2004 international symposium on circuits and systems, 2004. ISCAS’04 (Vol. 4, pp. IV–744). IEEE. Linares-Barranco, B. E. R. N. A. B. E., Sánchez-Sinencio, E., Rodríguez-Vázquez, A., & Huertas, J. L. (1991). A CMOS implementation of FitzHugh–Nagumo neuron model. IEEE Journal of Solid-State Circuits, 26(7), 956–965.

38

H. Soleimani et al. / Neural Networks 51 (2014) 26–38

Nakada, K., Asai, T., & Hayashi, H. (2005). A silicon Resonate-and-fire neuron based on the volterra system. In Int. symp. on nonlinear theory and its applications (pp. 82–85). Patel, G. N., & DeWeerth, S. P. (1997). Analogue VLSI morris-lecar neuron. Electronics Letters, 33(12), 997–998. Predictive Technology Model (PTM). (2013). Website: http://ptm.asu.edu/. Premont, C., Grisel, R., Abouchi, N., & Chante, J. P. (1998). A current conveyor based field programmable analog array. Analog Integrated Circuits and Signal Processing, 17(1–2), 105–124. Sedra, A., & Smith, K. (1970). A second-generation current conveyor and its applications. IEEE Transactions on Circuits System, 17, 132–134. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jiménez, A., & LinaresBarranco, B. (2006). A neuromorphic cortical-layer microchip for spike-based event processing vision systems. IEEE Transactions on Circuits and Systems I: Regular Papers, 53(11), 2548–2566.

Sharifipour, O., & Ahmadi, A. (2012). An analog implementation of biologically plausible neurons using CCII building blocks. Neural Networks, 36(1), 129–135. Soleimani, H., Ahmadi, A., & Bavandpour, M. (2012). Biologically inspired spiking neurons: piecewise linear models and digital implementation. IEEE Transactions on Circuits and Systems I: Regular Papers, 99(1), 2991–3004. van Schaik, A. (2001). Building blocks for electronic spiking neural networks. Neural Networks, 14(5), 617–628. Vittoz, E. A. (1985). The design of high-performance analog circuits on digital CMOS chips. IEEE Journal of Solid-State Circuits, 20(2), 657–665. Wijekoon, J. H., & Dudek, P. (2008). Compact silicon neuron circuit with spiking and bursting behaviour. Neural Networks, 21(2), 524–534. Yamashita, Y., & Torikai, H. (2012). A novel PWC spiking neuron model: neuron-like bifurcation scenarios and responses. IEEE Transactions on Circuits and Systems I: Regular Papers, 59(10), 2678–2691.