Home
Search
Collections
Journals
About
Contact us
My IOPscience
Long-term decoding stability of local field potentials from silicon arrays in primate motor cortex during a 2D center out task
This content has been downloaded from IOPscience. Please scroll down to see the full text. 2014 J. Neural Eng. 11 036009 (http://iopscience.iop.org/1741-2552/11/3/036009) View the table of contents for this issue, or go to the journal homepage for more
Download details: IP Address: 134.151.40.2 This content was downloaded on 07/09/2014 at 08:11
Please note that terms and conditions apply.
Journal of Neural Engineering J. Neural Eng. 11 (2014) 049501 (2pp)
doi:10.1088/1741-2560/11/4/049501
Corrigendum: Long-term decoding stability of local field potentials from silicon arrays in primate motor cortex during a 2D center out task (2014 J. Neural Eng. 11 036009) Dong Wang1,2,6, Qiaosheng Zhang1,3, Yue Li1,2, Yiwen Wang1,3, Junming Zhu1,4, Shaomin Zhang1,3,5 and Xiaoxiang Zheng1,2,3 1
Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou 310027, People’s Republic of China 2 College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, People’s Republic of China 3 Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, People’s Republic of China 4 Department of Neurosurgery, Second Affiliated Hospital School of Medicine, Zhejiang University, Hangzhou 310009, People’s Republic of China 5 Institute for Brain Science, Brown University, Providence, RI 02912, USA E-mail: [email protected]
(Some figures may appear in colour only in the online journal) In the supplementary figure 3, the third plot in the left column, which describes the relation between the tuning number
6
of LFP and the decoding performance of x-velocity, is incorrect. The correct plot is given in the figure supplied here.
Author to whom any correspondence should be addressed.
1741-2560/14/049501+02$33.00
1
© 2014 IOP Publishing Ltd Printed in the UK
D Wang et al
J. Neural Eng. 11 (2014) 049501
Figure 3. Stability of signal features for three monkeys. (a) Mean firing rates of SUA and MUA across channels with observed spikes in the first recording session in our study. The number of channels with observed spikes in the first session was 70, 37, 49 for B04, B01 and B03, respectively. (b) The mean power in each frequency band and the mean LMP amplitude averaged across channels in 100 ms time window over time. Each marker in the figures represents the result of one dataset. Dashed lines indicate the discontinuity in the time which is also labeled in the x-axis. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (* p < 0.05, ** p < 0.01).
2
Journal of Neural Engineering J. Neural Eng. 11 (2014) 036009 (16pp)
doi:10.1088/1741-2560/11/3/036009
Long-term decoding stability of local field potentials from silicon arrays in primate motor cortex during a 2D center out task Dong Wang 1,2 , Qiaosheng Zhang 1,3 , Yue Li 1,2 , Yiwen Wang 1,3 , Junming Zhu 1,4 , Shaomin Zhang 1,3,5 and Xiaoxiang Zheng 1,2,3 1 Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou 310027, People’s Republic of China 2 College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, People’s Republic of China 3 Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, People’s Republic of China 4 Department of Neurosurgery, Second Affiliated Hospital School of Medicine, Zhejiang University, Hangzhou 310009, People’s Republic of China 5 Institute for Brain Science, Brown University, Providence, RI 02912, USA
E-mail: [email protected] Received 6 November 2013, revised 16 February 2014 Accepted for publication 20 February 2014 Published 8 May 2014 Abstract
Objective. Many serious concerns exist in the long-term stability of brain–machine interfaces (BMIs) based on spike signals (single unit activity, SUA; multi unit activity, MUA). Some studies showed local field potentials (LFPs) could offer a stable decoding performance. However, the decoding stability of LFPs was examined only when high quality spike signals were recorded. Here we aim to examine the long-term decoding stability of LFPs over a larger time scale when the quality of spike signals was from good to poor or even no spike was recorded. Approach. Neural signals were collected from motor cortex of three monkeys via silicon arrays over 230, 290 and 690 days post-implantation when they performed 2D center out task. To compare long-term stability between LFPs and spike signals, we examined them in neural signals characteristics, directional tuning properties and offline decoding performance, respectively. Main results. We observed slow decreasing trends in the number of LFP channels recorded and mean LFP power in different frequency bands when spike signals quality decayed over time. The number of significantly directional tuning LFP channels decreased more slowly than that of tuning SUA and MUA. The variable preferred directions for the same signal features across sessions indicated non-stationarity of neural activity. We also found that LFPs achieved better decoding performance than SUA and MUA in retrained decoder when the quality of spike signals seriously decayed. Especially, when no spike was recorded in one monkey after 671 days post-implantation, LFPs still provided some kinematic information. In addition, LFPs outperformed MUA in long-term decoding stability in a static decoder. Significance. Our results suggested that LFPs were more durable and could provide better decoding performance when spike signals quality seriously decayed. It might be due to their resistance to recording degradation and their high redundancy among channels. Keywords: Long-term stability, local field potential, motor cortex, brain–machine interface S Online supplementary data available from stacks.iop.org/JNE/11/036009/mmedia (Some figures may appear in colour only in the online journal)
1741-2560/14/036009+16$33.00
1
© 2014 IOP Publishing Ltd
Printed in the UK
D Wang et al
J. Neural Eng. 11 (2014) 036009
matter whether SUA was present on the same electrodes or not [23]. However, the quality of neural recording did not show much degradation throughout their experiments because the decoding performance of SUA did not significantly decrease across sessions in their study. Therefore, it is still unclear whether LFPs could contain much movement information when SUA seriously decayed or was totally lost. Furthermore, there are few studies about how the decoding performance of LFPs changed when spike signals gradually declined in chronic recording and how much information remained in LFPs when spike signals were totally lost. Therefore, further studies are needed to systematically evaluate the long-term decoding stability of LFPs over larger time scale, compared with that of SUA and MUA. In this study, both degradation of neural recording and non-stationarity of neural activities were taken into account when we evaluated the long-term decoding stability of LFPs, compared with that of SUA and MUA. Neural data were collected from motor cortex of three well-trained monkeys with Blackrock silicon microelectrode arrays over different time ranges (from 115 to 692 days post-implantation) and offline analysis was used to exclude the learning factors involved in online experiments. First, we analyzed the basic characteristics of LFPs, SUA and MUA across sessions to show the degradation of neural recording. Second, direction tuning analysis was performed to assess the effect of neural recording degradation and neural non-stationarity on directional tuning respectively. Finally, we examined the offline long-term decoding performance of LFPs, SUA and MUA for hand movement direction, position and velocity. The retrained decoder was used to assess the long-term decoding performance under the effect of the neural recording degradation. The static decoder was used to evaluate the longterm decoding performance under the joint effects of these two factors. In our experiments, no SUA or MUA could be recorded from one monkey (B03) after 671 days postimplantation. To our knowledge, this is the first study to present the decoding performance of LFPs after no spiking activity could be recorded from the array.
1. Introduction Brain–machine interfaces (BMIs) could extract kinematic information about limbs from motor cortex activities and help paralyzed people restore their lost motor functions [1–6]. With the development of BMIs, rich movement information could be decoded from various neural signals: single unit activity (SUA) [7–18], multi unit activity (MUA) [18–22] and local field potentials (LFPs) [18, 21–30]. With high spatial–temporal resolution, SUA has been widely used to control multi degrees of freedom prosthesis, such as moving a cursor [10–14] or a robot arm [7, 8, 15, 16], or to restore muscle activity [9, 17]. However, there are still many serious concerns about the longevity of SUA recording. Many previous studies suggested that the decline of decoding performance based on SUA over time was closely related to the degradation of SUA recording, which might result from biological responses, material changes and engineering failures occurring at the electrode recording site post-implantation [31, 32]. In addition to the degradation of neural recording, the non-stationarity of neural activities across various behavioral conditions or attention states in chronic recording could also lead to fluctuations in decoding performance over time [33–35]. Although some studies demonstrated that the decoding performance of BMI systems could be improved by adaptive decoders in online experiments [34, 36–38], these studies were carried out in short-term periods when no significant signal degradation was observed. Furthermore, online experiments might introduce some learning factors which could compensate the degradation of decoding performance and improve the stability of BMI systems. Considering the learning factors, it was difficult to accurately evaluate the effect of neural signals degradation on decoding performance in chronic recording. MUA was another type of neural signal extracted from spiking activities and defined as threshold-crossing but unsorted spikes here. Previous studies showed that many kinematics could be well decoded from MUA [18–22] and MUA might provide much more durable and stable decoding performance than SUA [19, 22]. Chestek et al found that offline decoding performance of MUA was not well correlated with amplitude of MUA in three of four arrays. However, only one metric, average action potential amplitude across electrodes, was used for evaluating the quality of MUA and it might not be enough to present the degradation of neural recording [19]. Recently, Flint et al showed that online BMIs using MUA could retain stable over six months [22]. However, if brain plasticity is not taken into account, it is still unclear whether MUA could retain stable decoding performance under the degradation of neural recording in chronic offline experiments. Recently LFPs, which are supposed to be synchronized membrane potentials at dendritic synapses [39], have been demonstrated to contain much information about kinematics [18, 21–30] and it is widely postulated that LFPs could contain movement information much longer than SUA because of their durable recording. Flint et al demonstrated that LFPs provided accurate decoding performance as well as SUA did even on the 290th day post-implantation. They also found that the decoding performance of LFPs was similar to that of SUA no
2. Methods 2.1. Behavior task
All experimental procedures conformed to the Guide for the Care and Use of Laboratory Animals (China Ministry of Health) and were approved by the animal care committee at Zhejiang University, China. Three male macaque monkeys (B01, B03 and B04) were trained to control a cursor to perform a radial-4 center out task by operating a joystick as described in our previous study [40]. In brief, a vertical computer screen was positioned in front of the monkey in a primate chair. The experiment began when two circles were presented on the screen, an orange circle representing the target and a blue circle controlled by a joystick. The joystick was free to be moved in 2D horizontal plane. Once the target circle was presented in the center of the screen, the monkey was requested to control the blue circle to hit the target circle by moving the joystick 2
D Wang et al
J. Neural Eng. 11 (2014) 036009
in a horizontal plane. The monkey had to hit the target circle within 2 s and hold for 300 ms to obtain rewards. Then, the center circle disappeared and a randomly chosen peripheral target appeared. The monkey had to hit the target circle within 2 s and hold for 300 ms to obtain rewards again.
including the amplitude of LMP and seven spectral powers of aforementioned frequency bands. The kinematic data of hand were down-sampled at 10 Hz. All kinematic data and signal features were normalized by z-score transformation for decoding.
2.2. Electrode implantation and neural recording
2.4. The characteristics analysis of neural signals
All monkeys were chronically implanted with 96-microelectrode arrays in a 10 × 10 configuration (Blackrock Microsystems Inc., 4.2 × 4.2 mm2). In monkeys B01 and B04, the arrays were implanted in the arm areas of the right primary motor cortex (M1) contralateral to the arm used to perform the task. In B03, the array was implanted in the arm area of the left dorsal premotor cortex (PMd) contralateral to the arm used. The details of surgical implantation procedures have been described in our previous study [40]. Neural signals were recorded with Cerebus multichannel data acquisition system (Blackrock Microsystems Inc., USA) at a sample rate of 30 kHz for each channel. Ninety-six channels of neural signals were recorded for monkeys B01 and B04, 64 channels of neural signals were recorded for monkey B03 due to the technical constraints. The joystick position indicating the hand position was measured with potentiometers and recorded synchronously with the neural signals by two analogue channels of Cerebus multichannel data acquisition system at 1 kHz sample rate.
We examined the characteristics of neural signals to show the degradation of neural recording over time. First, we examined the number of LFP channels, SUA and MUA recorded across sessions. The channels with obvious artifacts were excluded in our analysis. We further examined the variability of the signal features over time. For SUA and MUA, we calculated their mean firing rates across electrodes with observed spikes in the first recording session for each monkey in each session. For LFPs, we calculated the mean power of each frequency band and the mean amplitude of LMP in 100 ms window across channels in each session. 2.5. Directional tuning analysis
To study the variability of movement information contained in three types of neural signals over time, we examined directional tuning properties of their features. The signal features obtained from the radial-4 center-out task were fitted with a cosine directional tuning model [43],
2.3. Signal preprocessing and feature extraction
y = a + b × cos(x − φ),
To obtain spike signals, raw data were first filtered by Butterworth high-pass filter at 250 Hz and then processed with a spike-detection threshold that was 4.5 times the root mean square (RMS) of baseline signal. The threshold-crossing but unsorted spikes were defined as MUA and the count of unsorted spikes in each 100 ms time window was used as its feature. Commercial software (Offline Sorter, Plexon Inc., USA) was further used to sort the threshold-crossing spikes into different SUA. Firing number in each 100 ms time window was used as the SUA feature. To obtain LFPs, raw data were digitally filtered by a bandpass filter between 0.3 and 500 Hz and down-sampled at 2 kHz. The channels with obvious artifacts (determined by visual inspection) were excluded from analysis for each session. First, the LFP time series of an entire session were segmented into 300 ms width windows with 200 ms overlapped. Then multi-taper spectral estimation approach was employed to obtain the power spectrum in each window of all the channels [41]. Finally the frequency spectrum of LFPs (0.3–400 Hz) in each channel was further partitioned into the seven different frequency bands as described in [26]: δ (0.3–5 Hz), θ –α (5–15 Hz), β (15–30 Hz), γ 1 (30–50 Hz), γ 2 (50–100 Hz), γ 3 (100–200 Hz) and the broad high frequency band (bhfLFP: 200–400 Hz). For each frequency band, the spectrum power was obtained by summing up all the estimated power values within that band. In addition, local motor potential (LMP) [42], which was calculated as the moving average of time domain LFP signal within 100 ms window, was used as another LFP feature. Overall, there were eight types of LFP features
(1)
where x represents the direction of movement; y corresponds to signal features; a denotes the baseline response; φ is the preferred direction and b indicates the tuning depth. The p-value obtained from the regression test was used to determine whether the signal was significantly tuned to direction or not. Here, we examined the number of significantly directional tuning LFP channels, SUA and MUA over time. For each SUA or MUA, if the regression test yielded p < 0.01, it was considered to be significantly tuned to direction and we counted it as a significantly directional tuning SUA or MUA. For LFPs, if the regression test yielded p < 0.00125 for any of eight LFP features in a channel, we counted this channel as a significantly directional tuning LFP channel. We studied the number of significantly directional tuning LFP channels, SUA and MUA from all the electrodes recorded in the arrays in three monkeys to show the effect of neural recording degradation on directional tuning. To further examine the effect of neural non-stationarity on directional tuning, we also analyzed the directional tuning of neural signals in the channels where spike signals were recorded in both the first session and the last session within 120 days after the first session. Because there was clear neural degradation in B03 and late sessions in B01, we chose the sessions within 120 days after the first recording session in B04 and B01 to assess the effect of neural non-stationarity on directional tuning. In addition, we also examined the number of significantly directional tuning features in each LFP feature (see supplementary method, available from stacks.iop.org/JNE/11/036009/mmedia). 3
D Wang et al
J. Neural Eng. 11 (2014) 036009
each test session was divided into five segments and the five decoding results were averaged to get the final decoding result for this session. For each session, the five decoding results of LFPs were compared with those of MUA in static decoder by one-tailed paired Student test. In addition, chance level was calculated for discrete decoding by shuffling the target directions and repeating the decoding process above. The whole process was repeated 1000 times and the 50 highest accuracy values were averaged to get the chance performance with a 95% confidence interval. A box plot was used to exhibit the stability of decoding performance across sessions. On each box, the central mark was the median value of decoding performance across sessions and the edges of the box were the 25th and 75th percentiles.
2.6. Discrete decoding
To explore how the offline discrete decoding performance changed over time, we used support vector machines (SVMs) to decode the movement directions in each session. Here the open source package ‘libsvm-mat-3.11’ was used to implement the SVMs decoding [44]. The signal features in four bins from 200 ms before to 200 ms after the movement onset were selected as the decoder input and the decoder was retrained for each session. 2.7. Continuous decoding
To estimate the offline continuous decoding performance, a Kalman filter (KF) [45] was used here to decode the position and velocity of hand. The Kalman filter was composed of the following generative model and system model: zk = Ak xk + pk ,
(2)
xk+1 = Hk xk + qk ,
(3)
3. Results Neural activities and kinematic data were collected from three monkeys, which were recorded over 230, 290 and 690 days post-implantation for B04, B01 and B03, respectively. Figure 1 shows the temporal distribution of all recording sessions used in this study. Because the monkeys also performed other tasks during their recording periods, the data used in this study were collected only from sessions when they performed these radial4 center out tasks. The numbers of recording sessions were 17, 23 and 21 for monkeys B04, B01 and B03, respectively. Each session lasted for 25 min and contained more than 300 trials. These sessions spanned 111, 183 and 400 days for B04, B01 and B03, respectively. All monkeys were well-trained before implantation and the success rates for target acquisition in these sessions were 97% ± 4% (B04, mean ± SD), 97% ± 2% (B01, mean ± SD) and 96% ± 3% (B03, mean ± SD).
where xk = [x, y, vx , vy ]Tk represents the position and velocity of hand at time tk = kt (t = 100 ms in this study); zk denotes the signal features at time tk ; A is the observation matrix that linearly relates the hand state to the signal features; H is the state matrix that linearly relates the hand kinematics at time tk+1 to the state at time tk ; pk ∼ N(0, T) and qk ∼ N(0, W ) are the observation and state Gaussian noise respectively with covariance matrices T and W . In our study, both retrained decoder and static decoder were used to comprehensively examine the long-term decoding stability of neural signals. The retrained decoder was recalibrated for each session. We employed it to evaluate the effect of neural recording degradation when comparing the decoding performance of LFPs, SUA and MUA over time. On the other hand, the static decoder was built in the first session and used to decode the kinematics in the consequent sessions without any recalibration. It was used to evaluate the joint effects of neural recording degradation and non-stationarity of neural activities on decoding performance. Because it was hard to track the same SUA across sessions, only MUA and LFPs were examined in the static decoder. Considering that the static decoder was fixed across all sessions, if no MUA was detected in a channel through a session, we set the MUA feature in this channel to zero in the session. Likewise, if a channel contained obvious artifacts in a session, all LFP features in this channel were set to zero in LFP decoding in this session.
3.1. Neural signal recording stability
Figure 2 shows the number of LFP channels, SUA and MUA recorded from arrays across sessions in three monkeys. We found that many channels gradually lost their capability of recording SUA or MUA over time and that most of the curves show significantly decreasing trends. The results of the linear regression method showed that the number of SUA declined by 6.0% (B04, p < 0.01), 8.27% (B01, p < 0.01) and 7.38% (B03, p < 0.01) per month on average and the number of MUA declined by 6.99% (B04, p < 0.01), 7.50% (B01, p < 0.01) and 7.36% (B03, p < 0.01) per month on average. The number of LFP channels recorded also showed a decreasing trend over time in the three monkeys, but they decreased much more slowly than SUA and MUA with the declining ratios of 1.96% (B04, p < 0.01), 0.41% (B01, p < 0.05) and 1.15% (B03, p < 0.01) per month on average. It was consistent with Flint’s study that more and more channels were removed from decoders over time due to noise [22]. Figure 3 shows the variability of different signal features over time. We found that different signal features decreased with different ratios (table 1). Meanwhile, the LFP features exhibited gradually decreasing trends over time while the SUA and MUA features decreased much more quickly across sessions.
2.8. Performance evaluation
For discrete decoding, classification accuracy was used to evaluate the decoding performance. To evaluate the decoding performance of continuous kinematics, we chose the correlation coefficient (CC) as the metric. It measures the similarity between the decoded kinematics and the corresponding actual values. For the retrained decoders, fivefold cross-validation was used to get the final results [23]. For each session, the five decoding results from the five-fold crossvalidation of LFPs were compared with those from SUA and MUA by one-tailed paired Student test. For the static decoder, 4
D Wang et al
J. Neural Eng. 11 (2014) 036009
B04(M1)
B01(M1)
B03(PMd)
100
200
300 400 500 Time since implantation(days)
600
700
Figure 1. Time distribution of neural recording sessions. The x-axis denotes the number of days after the electrodes were implanted, the y-axis labels the three monkeys and their corresponding recording cortices in parentheses respectively. Each bar in the line represents one recording session.
80
60 40 20 0 120
LFP ** SUA** MUA **
100
60 40 20
140 160 180 200 220 240 Time since implantation(days)
LFP ** SUA** MUA**
80
LFP * SUA** MUA**
Number
80 Number
100
Number
100
60 40 20
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
Figure 2. The number of LFP channels, SUA and MUA recorded from three monkeys over time. Each marker in the figures represents the result of one dataset. The five-pointed star on the y-axis indicates the number of neural channels recorded for the monkey. Dashed lines indicate the discontinuity in time, which is also labeled on the x-axis. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01). Table 1. The decreasing rate per month for different neural features.
Monkey B04 B01 B03
LMP
δ
θ –α
β
1.75% 1.85% 1.98% 0.20% 4.50% 4.43% 4.80% 4.38% −0.11% −0.04% −0.03% −0.25%
γ1
γ2
γ3
bhfLFP
SUA
MUA
0.06% 2.57% 0.59%
0.49% 1.20% 1.25%
0.15% 0.42% 2.46%
0.76% 0.85% 2.82%
9.36% 12.60% 7.15%
9.31% 12.64% 7.12%
bhfLFP was significantly tuned in all 17 sessions. Similar results were also found in many other channels in the three monkeys. It is also noted that preferred directions of bhfLFP and MUA were variable across sessions. Furthermore, we examined the directional tuning stability of three types of neural signals by assessing the number of significantly directional tuning LFP channels, SUA and MUA across sessions (figure 5(a)). We found they all showed declining trends over time, but LFPs decreased much more slowly than the other two types of signals. Especially, it clearly shows that many LFP channels were still significantly tuned to direction even when no SUA or MUA was recorded from monkey B03 after 671 days post-implantation. We also found that many more LFP channels were tuned to direction than SUA and MUA across all sessions in the three animals (figure 5(a)) and the numbers of two LFP features with directional tuning (γ 3 and bhfLFP) were significantly higher than those of SUA and MUA across sessions (onetailed paired Student test, p < 0.05, supplementary figure S1, available from stacks.iop.org/JNE/11/036009/mmedia).
3.2. Directional tuning stability
By analyzing directional tuning properties of neural signals, we found that all three types of neural signals could be significantly tuned to movement directions, which was similar to the previous studies [18, 23, 27]. For example, figure 4(a) illustrates the directional tuning of LFPs, SUA and MUA recorded from channel 91 of monkey B04 on 144th day post-implantation. Six LFP features in this channel showed significantly directional tuning, but with different preferred directions and different tuning depths. MUA and SUA were tuned to the same direction because only one unit was found in this channel. We also found that multi LFP features from the same channel were tuned to different movement directions in many other channels, which suggested these directional tuning signal features could complement each other and be combined in neural decoding. Figure 4(b) further shows the tuning directions of bhfLFP and MUA across all 17 sessions in channel 91 of monkey B04. We found that MUA was significantly tuned in only four recording sessions while 5
D Wang et al
J. Neural Eng. 11 (2014) 036009
(a)
**
**
120 100 80 60 40 20 120
**
**
* ***
50 40 140 160 180 200 220 240 Time since implantaton(days)
5
** **
100 80 60 40
* ***
** **
120 100 80 60 40 20 280 300 320 560 580 600 620 640 660 680 700 Time since implantaton(days)
**
Mean amplitude
Mean amplitude
60
10
*** *
120
*
70
*** *
20 100 120 140 160 180 200 220 240 260 280 300 Time since implantaton(days)
140 160 180 200 220 240 Time since implantaton(days)
**
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
Mean pow power(10*log) ower(10*log)
**
5
70 60 50 40 100 120 140 160 180 200 220 240 260 280 300 Time since implantaton(days)
Mean amplitude
Mean power(10*log)
*
10
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
140 160 180 200 220 240 Time since implantation(days) *
Mean firing ra rrate(Hz) ate(Hz)
Mean firing ra rrate(Hz) ate(Hz)
0 120 0
Mean power(10*log)
Mean firing rate(Hz)
5
120
**
15
15
10
(b)
**
**
**
15
70 60 50 40 280 300 320 560 580 600 620 640 660 680 700 Time since implantaton(days)
Figure 3. Stability of signal features for three monkeys. (a) Mean firing rates of SUA and MUA across channels with observed spikes in the first recording session in our study. The number of channels with observed spikes in the first session was 70, 37, 49 for B04, B01 and B03, respectively. (b) The mean power in each frequency band and the mean LMP amplitude averaged across channels in 100 ms time window over time. Each marker in the figures represents the result of one dataset. Dashed lines indicate the discontinuity in the time which is also labeled in the x-axis. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01).
LFPs showed significantly higher performance in most late sessions from B01 and B03 (sessions after 240 days in B01 and after 620 days in B03, one-tailed paired Student test, p < 0.05). We further observed that the classification accuracy of LFPs maintained above 0.90 and that the decreasing trend was only found in the sessions after 600 days post-implantation in B03. Especially, it is worth noting that the decoding accuracy of LFPs was still higher than 0.53 even when no SUA or MUA was recorded from B03 array after 671 days postimplantation. When comparing the decoding performance among three types of neural signals across all sessions, we found that LFPs had higher mean values with less variance than SUA and MUA in both B01 and B03 (figure 6(b)). The decoding performances of three types of neural signals neither have significant declination nor show difference between each other in B04, which might due to the quality of neural signal not seriously decaying. In addition, we found that the decoding performances of SUA and MUA were very close in all sessions across three monkeys, which was consistent with the results from other studies [19, 20].
Figure 5(b) further shows the effect of neural non-stationarity on directional tuning in B04 and B01. In these chosen channels, we found that the number of directional tuning LFP channels was more stable than the other two signals across sessions. There were high values for SUA and MUA in the first and last sessions in figure 5(b), which might be due to no spike signals being recorded in some of these chosen channels in the middle sessions. The variability of the number of each directional tuning LFP feature in these chosen channels is shown in supplementary figure S2, available from stacks.iop.org/JNE/11/036009/mmedia. We also found that the number of directional tuning bhfLFP in these chosen channels was most stable among LFP features and more stable than that of SUA and MUA. All these results suggested that directional tuning information in LFPs remained stable for a long time. 3.3. Discrete decoding stability
We offline decoded movement directions using three types of neural signals in 2D center out task and compared their decoding performance in the long-term recording. Figure 6(a) shows that the decoding performance of three types of neural signals gradually declined over time in B01 and B03. We found that in early sessions the decoding performance of LFPs was comparable to those of SUA and MUA in three animals and
3.4. Continuous decoding stability
We offline decoded hand position and velocity to further compare the long-term decoding performance of LFPs, SUA 6
D Wang et al
J. Neural Eng. 11 (2014) 036009
(a)
10 Firing rate(Hz)
Frequency(Hz)
400
200
0
5
0 0 Time(s)
1
0
1 Time(s)
10 0 -10
(b)
Figure 4. Examples of directional tuning analysis from monkey B04. (a) Time-resolved power spectrum of a sample channel for different directions (left) and directional modulation of the firing rate of a sample unit (right). The data come from channel 91 recorded on 144th day post-implantation. The first dashed line indicates the movement onset and the second dashed line indicates the movement offset. The data in the left figures are normalized based on the mean of the data before movement onset. The circles in the center of figures showed the preferred directions and tuning depth of each signal feature. The stars represent the signal feature was significantly tuned to direction (p < 0.01). (b) Circles show the preferred directions of bhfLFP and MUA in channel 91 across sessions. The two red lines represent the preferred directions of bhfLFP and MUA 144 days post implantation respectively. Because there were 13 sessions in which no MUA was recorded in channel 91, only four preferred directions were shown in the figure for MUA.
direction were not significantly different (one-tailed paired Student test, p > 0.05). Similar results were also observed in B01. Compared with discrete decoding, the result from continuous decoding also showed bigger fluctuations and more serious degradation in B03, which might be relevant to the complexity of the decoding task. Figure 9 shows that LFPs achieved significantly better decoding performance than SUA and MUA did in most kinematics (one-tailed paired Student test, p < 0.05), which further demonstrated the advantages of LFPs in long-term decoding stability. In addition, we found the decoding performance of SUA was a little better than that of MUA with statistical significance (one-tailed paired Student test, p < 0.05). To study the joint effects of neural recording degradation and non-stationarity of neural activities on long-term decoding performance, we further used static decoder to assess decoding performance of MUA and LFPs across sessions. Because there was a break of over 200 days during the B03 recording, we only examined the performance of static decoder across sessions in B04 and B01 respectively. Figure 10 shows the results of LFPs and MUA obtained from both static and retrained decoder across sessions. We observed that the decoding
and MUA. The results from KF decoder showed both SUA and MUA significantly linearly declined in decoding performance across sessions in three monkeys (figure 7, linear regression, p < 0.01). In contrast, LFPs only showed significantly linearly declining decoding performance in B03 (linear regression, p < 0.01). Figure 8 shows some samples of Y-position decoding based on LFPs, SUA and MUA, respectively. All these types of neural signals achieved good decoding performance when good quality neural signals were recorded. From figure 7, we also found that the CCs obtained from LFPs in most late sessions from B01 and B03 were much higher than those of SUA and MUA. Especially when no spike signals was detected in B03 after 671 days post-implantation, kinematic information could still be extracted from LFPs (CC: from 0.15 to 0.33). Furthermore, we found that the continuous decoding performance of LFP outperformed those of SUA and MUA in some sessions where no significant difference was found in discrete decoding performance between them. For example, in most of the late sessions from B04 (after 160 days postimplantation), we found LFPs achieved significantly higher CCs than SUA and MUA did (one-tailed paired Student test, p < 0.05) while their classification accuracies of movement 7
D Wang et al
J. Neural Eng. 11 (2014) 036009
100
60
0 120
60 40 20
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
140 160 180 200 220 240 Time since implantation(days)
Directional tuning number
(b)
LFP SUA ** MUA**
50
Directional tuning number
20
LFP SUA ** MUA **
80
40 30 20 10 0 120
LFP SUA MUA 140 160 180 200 220 240 Time since implantation(days)
Directional tuning number
80
40
100
100 Directional tuning number
Directional tuning number
(a) LFP ** SUA ** MUA **
80 60 40 20
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
30
20
LFP SUA MUA
10
0 100
120 140 160 180 200 220 Time since implantation(days)
240
Figure 5. The number of significantly directional tuning LFP channels, SUA and MUA over time. (a) Each marker in the figures represents the result of one session. Dashed lines indicate the discontinuity in time, which is also labeled in the x-axis. (b) Only the channels with observed spikes in both first session and last session in the figures were examined for the directional tuning. Each marker in the figures represents the result of one session within 120 days after the first session. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01).
1
1
0.8
0.8
0.8
0.6 0.4
LFP SUA MUA
0.2
0.6 0.4
0.6 0.4
LFP ** SUA ** MUA **
0.2
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantaton(days)
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantaton(days)
1
1
0.8
0.8
0.8
0.6 0.4
0.6 0.4
LFP
SUA
MUA
0
0.6 0.4 0.2
0.2
0.2 0
Accuracy
1
Accuracy
Accuracy
LFP SUA ** MUA **
0.2
0 120 140 160 180 200 220 240 Time since implantaton(days)
(b)
Accuracy
1
Accuracy
Accuracy
(a)
LFP
SUA
MUA
0
LFP
SUA
MUA
Figure 6. Direction decoding performance based on LFPs, SUA and MUA using retrained decoder over time. (a) Direction decoding accuracy of LFPs, SUA and MUA over time. Each marker represents the result of one session. The downward-pointing triangle means that the decoding accuracy of LFPs was significantly better than that of SUA and MUA in that session (one-tailed paired Student test, p < 0.05). Dashed lines indicate the discontinuity in time. (b) The distribution of direction decoding accuracy for LFPs, SUA and MUA for three monkeys, respectively. On each box, the central red mark is the median, the edges of the box are the 25th and 75th percentiles and red ‘+’ labels the outliers. Dashed line indicates the chance level. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01).
performance obtained from static decoder was significantly worse than that from retrained decoder in both monkeys (onetailed paired Student test, p < 0.05). Although significantly decreasing trends in static decoding were found in both types of neural signals across sessions, LFPs provided significantly
better performance than MUA in static decoding in most late sessions, which were labeled with downward-pointing triangles in figure 10 (one-tailed paired Student test, p < 0.05). Figure 11 further shows that LFPs outperformed MUA in static decoder in long-term decoding stability in B01. The variances 8
D Wang et al
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4 0.2
0.2
LFP ** SUA ** MUA U **
1
1
0.8
0.8
0.8
0.6
0.6
0.6
LFP SUA ** MUA **
CC
0.4
0.4 0.2
CC
1
140 160 180 200 220 240 Time since implantation(days)
LFP SUA ** MUA **
0.4 0.2
LFP ** SUA ** MUA **
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4 0.2 0 120
LFP * SUA * MUA **
0.4 0.2
CC
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
140 160 180 200 220 240 Time since implantation(days)
LFP SUA ** MUA **
0.4 0.2
LFP ** SUA ** MUA **
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4 0.2 0 120
140 160 180 200 220 240 Time since implantation(days)
LFP * SUA ** MUA ** 140 160 180 200 220 240 Time since implantation(days)
0.4 0.2
CC
CC
X-velocity
0.4
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
0.2
CC
0.2
*
LFP SUA ** MUA **
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
0 120
Y-velocity
0.4
*
CC
CC
Y-position
0 120
LFP SUA MUA
CC
1
CC
1
CC
CC
X-position
J. Neural Eng. 11 (2014) 036009
LFP SUA ** MUA **
0.4 0.2
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
LFP ** SUA ** MUA **
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
Figure 7. Continuous decoding performance using retrained decoder over time. The figures show the decoding performance for position and
velocity based on LFPs, SUA and MUA respectively over time for three monkeys. The results were displayed by correlation coefficient (CC). Each marker indicates one dataset. The downward-pointing triangle means that the decoding performance of LFPs was significantly better than that of SUA and MUA for that kinematics in that session (one-tailed paired Student test, p < 0.05). Dashed lines indicate the discontinuity in time. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases over time (∗ p < 0.05, ∗∗ p < 0.01).
could cause the decreasing decoding performance when the quality of neural recording seriously decayed. As the number of directional tuning signals decreased over time, the decoding performance showed different decreasing trends in discrete and continuous decoding. We found that many more directional tuning signals were needed to keep a high level in continuous decoding, compared with the discrete decoding. In discrete decoding, we found that the decoding performance of LFPs remained at a high level and did not drop sharply until the number of tuned LFP channels decreased to less than about 25 in B03 (figure 12). Similar results were observed when the number of tuned SUA and MUA decreased to less than 20 in both B01 and B03. It was also noted from figure 5(a) that the number of tuned LFP channels was still higher than 25 when the number of tuned SUA or MUA decreased to less than 20 during recording degradation.
of decoding performance were close among LFPs and MUA in static decoder in B04, it might be relevant that the quality of neural recording did not decay seriously in short time and the advantage of long-term stability of LFPs was not shown. All these results suggested that LFPs were more stable than MUA in long-term decoding. 3.5. Correlation analysis
We further examined the correlation between the number of directional tuning signals and decoding performance. The results are shown in figure 12 and supplementary figure S3, available from stacks.iop.org/JNE/11/036009/mmedia. From the results, we found that the number of directional tuning signals was nonlinearly correlated with decoding performance and the decline in the number of directional tuning signals 9
D Wang et al
J. Neural Eng. 11 (2014) 036009
LFP decoding
actual trajectory decoded trajectory
5
0 CC=0.71 -5
0
20
40
60
80
100 120 time(s) SUA decoding
140
160
180
200
actual trajectory decoded trajectory
4 2 0 -2
CC=0.79 -4
0
20
40
60
80
100 120 time(s) MUA decoding
140
160
180
200
actual trajectory decoded trajectory
4 2 0 -2
CC=0.78 -4
0
20
40
60
80
100 time(s)
120
140
160
180
200
Figure 8. Examples of decoding performance based on LFPs, SUA and MUA for Y-position. This dataset was recorded 115 days
post-implantation from monkey B01. The predicted trajectory was close to the actual trajectory and most of the peaks could be decoded correctly using these three types of neural signals.
* *
*
*
* *
*
*
1
*
0.8
0.8
0.6
0.6 CC
CC
*
0.4 0.2 0 X-position Y-position X-velocity
*
*
*
*
*
*
0.8
*
* * *
*
*
*
* *
*
0.6
0.4 0.2
Y-velocity
*
1
*
* *
CC
1
0.4 0.2
0 X-position Y-position X-velocity
Y-velocity
0 X-position Y-position X-velocity
Y-velocity
Figure 9. The distribution of continuous decoding performance using retrained decoder for LFPs, SUA and MUA. The results were displayed by correlation coefficient (CC) for position and velocity for each monkey. On each box, the central red mark is the median, the edges of the box are the 25th and 75th percentiles and red ‘+’ labels the outliers. The asterisk above LFPs and SUA/MUA boxes means that the decoding performance of LFPs was significantly better than that of SUA/MUA. The asterisk above SUA and MUA boxes means that the decoding performance of SUA was significantly better than that of MUA. One-tailed paired Student test was performed here (p < 0.05). 10
D Wang et al
J. Neural Eng. 11 (2014) 036009
1 CC
CC
X-position
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
1 CC
CC
Y-position
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
1 CC
CC
X-velocity
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
1 CC
CC
Y-velocity
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
Figure 10. Continuous decoding performance using static decoder over time. The static decoder prediction results of LFPs and MUA were
obtained by using a session to train the decoder and decoding the following several sessions selected respectively. The decoding performance using retrained decoder for LFPs and MUA are shown here for comparison. The results are displayed by CC. Each marker indicates one dataset. The downward-pointing triangle means that the decoding performance of LFPs based on static decoder was significantly better than that of MUA for the kinematics in that session (one-tailed paired Student test, p < 0.05).
properties and offline decoding performance. Two factors were considered here: the degradation of neural recording and the non-stationarity of neural activities. The results showed that the number of LFP channels, SUA and MUA recorded decreased over time, but LFPs decreased more slowly than the other two signals. The mean power in each frequency band in LFPs decreased over time along with decreasing mean firing rates of spikes. The number of directional tuning LFP channels also decreased much more slowly than that of SUA and MUA. The variable preferred directions for the same signal features showed the effect of neural non-stationarity on directional tuning. When the quality of neural signals seriously decayed in chronic recording, LFPs achieved better decoding performance than SUA and MUA did in retrained
Therefore, sharply dropping decoding performance in LFPs appeared later than for SUA/MUA, which was also found in continuous decoding analysis (supplementary figure S3, available from stacks.iop.org/JNE/11/036009/mmedia). These results suggested that the less decoding performance decline of LFPs in chronic recording was relevant to LFPs retaining directional tuning information more durably than SUA and MUA. 4. Discussion In this study, we examined the long-term stability of LFPs and compared it with that of spike signals (SUA and MUA) in the characteristics of neural signals, directional tuning 11
D Wang et al
J. Neural Eng. 11 (2014) 036009
1
1 *
0.8
0.8
0.6
0.6 CC
CC
*
0.4
*
0.4 0.2
0.2 0 X-position
Y-position
X-velocity
0 X-position
Y-velocity
Y-position
X-velocity
Y-velocity
Figure 11. The distribution of continuous decoding performance using static decoder for LFPs and MUA. The results were displayed by
1
1
0.8
0.8
0.8
0.6 0.4 0.2 0 100
B04 B01 B03 80 60 40 20 Directional al tuning number
0.6 0.4 0.2
0
Accuracy
1
Accuracy
Accuracy
correlation coefficient (CC) for position and velocity for each monkey. On each box, the central red mark is the median, the edges of the box are the 25th and 75th percentiles and red ‘+’ labels the outliers. The asterisk above boxes means that the decoding performance of LFPs was significantly better than that of MUA using static decoder (one-tailed paired Student test, p < 0.05).
0 80
B04 B01 B03
0.6 0.4 0.2
60 40 20 Directional Di ti l ttuning i number b
0
0 80
B04 B01 B03 60 40 20 Directional tuning t ning number n mb
0
Figure 12. The correlation between the number of directional tuning signals and their direction decoding accuracies. For LFPs, we plot the
scatter figure for the number of directional tuning LFP channels versus decoding accuracy (left). For SUA/MUA, we plot the figure for the number of directional tuning SUA/MUA versus decoding accuracy (middle/right). Dot marker represents individual session result. Note that the decoding accuracy remained stable at a high level when the directional tuning number was more than 25 for LFPs and 20 for SUA and MUA. It also shows when the directional tuning number was less than 25 for LFPs and 20 for SUA and MUA, poorer decoding accuracy was obtained with fewer directional tuning signals. The decoding performances here were significantly correlated with the number of directional tuning signals (spearman correlation, p < 0.05).
Furthermore, we examined the effect of neural degradation on LFPs recording quality over time. To our knowledge, this is the first study to systematically examine the degradation of LFPs in chronic recording. Declining trends were found in both the number of LFP channels recorded and mean power in each frequency band. The number of directional tuning LFP channels also decreased due to the degradation of neural recording. The close correlation between the number of directional tuning signals and decoding performance further suggested the degradation of neural recording was one reason causing the decreasing decoding performance in chronic recording. The results of correlation analysis also indicated that the number of directional tuning signals could be used as a good measure of BMI performance compared with the amplitude used in a previous study [19]. Our results in this study showed that both LFPs and spike signals decayed over time in the signal quality under the effect of neural recording degradation, which might be related to similar reasons, such as tissue reaction at the recording site, engineering or material failures, micromotion and so on [50–56]. However, multiple mechanisms underlying degradation of neural recording were still unclear. The long-term tissue reaction or material examination needs to be further explored
decoder and outperformed MUA in static decoder. Especially, we provided strong evidence that LFPs could still be used to decode kinematics even when neither SUA nor MUA could be recorded from arrays. All results suggested that LFPs could provide better performance in durability and stability. 4.1. The impact of degradation of neural recording on long-term decoding performance
Previous studies have shown that the recording quality of spike signals (SUA and MUA) decayed over time and it was hard to record them over several years except for several groups [19, 32, 46, 47]. Williams et al showed large declines in SUA using wire microelectrode arrays over months [48]. Vetter et al reported that mean signal-to-noise ratio (SNR) of SUA decreased slowly over time using Michigan microelectrode arrays [49]. Chestek and Gilja et al found that the peak-to-peak voltage of action potentials decreased over months and was significantly correlated with changes in offline performance using all three metrics in one array which was recorded over three years [19, 31]. Here we also got declining quality of spike signals over time, which was consistent with the aforementioned studies. 12
D Wang et al
J. Neural Eng. 11 (2014) 036009
in chronic recording to develop new technology for practical BMI application.
declining trends in decoding performance over time, but LFPs decreased much more slowly and still got significantly better decoding performance than spikes in most recording sessions where the quality of spike signals was poor. The difference in decoding stability between them might be due to the signal characteristics they represented. Spike signals are extracellular action potentials of neurons recorded close to electrode tip, which have high spatial selectivity. LFPs are low frequency signals (
Search
Collections
Journals
About
Contact us
My IOPscience
Long-term decoding stability of local field potentials from silicon arrays in primate motor cortex during a 2D center out task
This content has been downloaded from IOPscience. Please scroll down to see the full text. 2014 J. Neural Eng. 11 036009 (http://iopscience.iop.org/1741-2552/11/3/036009) View the table of contents for this issue, or go to the journal homepage for more
Download details: IP Address: 134.151.40.2 This content was downloaded on 07/09/2014 at 08:11
Please note that terms and conditions apply.
Journal of Neural Engineering J. Neural Eng. 11 (2014) 049501 (2pp)
doi:10.1088/1741-2560/11/4/049501
Corrigendum: Long-term decoding stability of local field potentials from silicon arrays in primate motor cortex during a 2D center out task (2014 J. Neural Eng. 11 036009) Dong Wang1,2,6, Qiaosheng Zhang1,3, Yue Li1,2, Yiwen Wang1,3, Junming Zhu1,4, Shaomin Zhang1,3,5 and Xiaoxiang Zheng1,2,3 1
Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou 310027, People’s Republic of China 2 College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, People’s Republic of China 3 Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, People’s Republic of China 4 Department of Neurosurgery, Second Affiliated Hospital School of Medicine, Zhejiang University, Hangzhou 310009, People’s Republic of China 5 Institute for Brain Science, Brown University, Providence, RI 02912, USA E-mail: [email protected]
(Some figures may appear in colour only in the online journal) In the supplementary figure 3, the third plot in the left column, which describes the relation between the tuning number
6
of LFP and the decoding performance of x-velocity, is incorrect. The correct plot is given in the figure supplied here.
Author to whom any correspondence should be addressed.
1741-2560/14/049501+02$33.00
1
© 2014 IOP Publishing Ltd Printed in the UK
D Wang et al
J. Neural Eng. 11 (2014) 049501
Figure 3. Stability of signal features for three monkeys. (a) Mean firing rates of SUA and MUA across channels with observed spikes in the first recording session in our study. The number of channels with observed spikes in the first session was 70, 37, 49 for B04, B01 and B03, respectively. (b) The mean power in each frequency band and the mean LMP amplitude averaged across channels in 100 ms time window over time. Each marker in the figures represents the result of one dataset. Dashed lines indicate the discontinuity in the time which is also labeled in the x-axis. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (* p < 0.05, ** p < 0.01).
2
Journal of Neural Engineering J. Neural Eng. 11 (2014) 036009 (16pp)
doi:10.1088/1741-2560/11/3/036009
Long-term decoding stability of local field potentials from silicon arrays in primate motor cortex during a 2D center out task Dong Wang 1,2 , Qiaosheng Zhang 1,3 , Yue Li 1,2 , Yiwen Wang 1,3 , Junming Zhu 1,4 , Shaomin Zhang 1,3,5 and Xiaoxiang Zheng 1,2,3 1 Qiushi Academy for Advanced Studies, Zhejiang University, Hangzhou 310027, People’s Republic of China 2 College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou 310027, People’s Republic of China 3 Key Laboratory of Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, People’s Republic of China 4 Department of Neurosurgery, Second Affiliated Hospital School of Medicine, Zhejiang University, Hangzhou 310009, People’s Republic of China 5 Institute for Brain Science, Brown University, Providence, RI 02912, USA
E-mail: [email protected] Received 6 November 2013, revised 16 February 2014 Accepted for publication 20 February 2014 Published 8 May 2014 Abstract
Objective. Many serious concerns exist in the long-term stability of brain–machine interfaces (BMIs) based on spike signals (single unit activity, SUA; multi unit activity, MUA). Some studies showed local field potentials (LFPs) could offer a stable decoding performance. However, the decoding stability of LFPs was examined only when high quality spike signals were recorded. Here we aim to examine the long-term decoding stability of LFPs over a larger time scale when the quality of spike signals was from good to poor or even no spike was recorded. Approach. Neural signals were collected from motor cortex of three monkeys via silicon arrays over 230, 290 and 690 days post-implantation when they performed 2D center out task. To compare long-term stability between LFPs and spike signals, we examined them in neural signals characteristics, directional tuning properties and offline decoding performance, respectively. Main results. We observed slow decreasing trends in the number of LFP channels recorded and mean LFP power in different frequency bands when spike signals quality decayed over time. The number of significantly directional tuning LFP channels decreased more slowly than that of tuning SUA and MUA. The variable preferred directions for the same signal features across sessions indicated non-stationarity of neural activity. We also found that LFPs achieved better decoding performance than SUA and MUA in retrained decoder when the quality of spike signals seriously decayed. Especially, when no spike was recorded in one monkey after 671 days post-implantation, LFPs still provided some kinematic information. In addition, LFPs outperformed MUA in long-term decoding stability in a static decoder. Significance. Our results suggested that LFPs were more durable and could provide better decoding performance when spike signals quality seriously decayed. It might be due to their resistance to recording degradation and their high redundancy among channels. Keywords: Long-term stability, local field potential, motor cortex, brain–machine interface S Online supplementary data available from stacks.iop.org/JNE/11/036009/mmedia (Some figures may appear in colour only in the online journal)
1741-2560/14/036009+16$33.00
1
© 2014 IOP Publishing Ltd
Printed in the UK
D Wang et al
J. Neural Eng. 11 (2014) 036009
matter whether SUA was present on the same electrodes or not [23]. However, the quality of neural recording did not show much degradation throughout their experiments because the decoding performance of SUA did not significantly decrease across sessions in their study. Therefore, it is still unclear whether LFPs could contain much movement information when SUA seriously decayed or was totally lost. Furthermore, there are few studies about how the decoding performance of LFPs changed when spike signals gradually declined in chronic recording and how much information remained in LFPs when spike signals were totally lost. Therefore, further studies are needed to systematically evaluate the long-term decoding stability of LFPs over larger time scale, compared with that of SUA and MUA. In this study, both degradation of neural recording and non-stationarity of neural activities were taken into account when we evaluated the long-term decoding stability of LFPs, compared with that of SUA and MUA. Neural data were collected from motor cortex of three well-trained monkeys with Blackrock silicon microelectrode arrays over different time ranges (from 115 to 692 days post-implantation) and offline analysis was used to exclude the learning factors involved in online experiments. First, we analyzed the basic characteristics of LFPs, SUA and MUA across sessions to show the degradation of neural recording. Second, direction tuning analysis was performed to assess the effect of neural recording degradation and neural non-stationarity on directional tuning respectively. Finally, we examined the offline long-term decoding performance of LFPs, SUA and MUA for hand movement direction, position and velocity. The retrained decoder was used to assess the long-term decoding performance under the effect of the neural recording degradation. The static decoder was used to evaluate the longterm decoding performance under the joint effects of these two factors. In our experiments, no SUA or MUA could be recorded from one monkey (B03) after 671 days postimplantation. To our knowledge, this is the first study to present the decoding performance of LFPs after no spiking activity could be recorded from the array.
1. Introduction Brain–machine interfaces (BMIs) could extract kinematic information about limbs from motor cortex activities and help paralyzed people restore their lost motor functions [1–6]. With the development of BMIs, rich movement information could be decoded from various neural signals: single unit activity (SUA) [7–18], multi unit activity (MUA) [18–22] and local field potentials (LFPs) [18, 21–30]. With high spatial–temporal resolution, SUA has been widely used to control multi degrees of freedom prosthesis, such as moving a cursor [10–14] or a robot arm [7, 8, 15, 16], or to restore muscle activity [9, 17]. However, there are still many serious concerns about the longevity of SUA recording. Many previous studies suggested that the decline of decoding performance based on SUA over time was closely related to the degradation of SUA recording, which might result from biological responses, material changes and engineering failures occurring at the electrode recording site post-implantation [31, 32]. In addition to the degradation of neural recording, the non-stationarity of neural activities across various behavioral conditions or attention states in chronic recording could also lead to fluctuations in decoding performance over time [33–35]. Although some studies demonstrated that the decoding performance of BMI systems could be improved by adaptive decoders in online experiments [34, 36–38], these studies were carried out in short-term periods when no significant signal degradation was observed. Furthermore, online experiments might introduce some learning factors which could compensate the degradation of decoding performance and improve the stability of BMI systems. Considering the learning factors, it was difficult to accurately evaluate the effect of neural signals degradation on decoding performance in chronic recording. MUA was another type of neural signal extracted from spiking activities and defined as threshold-crossing but unsorted spikes here. Previous studies showed that many kinematics could be well decoded from MUA [18–22] and MUA might provide much more durable and stable decoding performance than SUA [19, 22]. Chestek et al found that offline decoding performance of MUA was not well correlated with amplitude of MUA in three of four arrays. However, only one metric, average action potential amplitude across electrodes, was used for evaluating the quality of MUA and it might not be enough to present the degradation of neural recording [19]. Recently, Flint et al showed that online BMIs using MUA could retain stable over six months [22]. However, if brain plasticity is not taken into account, it is still unclear whether MUA could retain stable decoding performance under the degradation of neural recording in chronic offline experiments. Recently LFPs, which are supposed to be synchronized membrane potentials at dendritic synapses [39], have been demonstrated to contain much information about kinematics [18, 21–30] and it is widely postulated that LFPs could contain movement information much longer than SUA because of their durable recording. Flint et al demonstrated that LFPs provided accurate decoding performance as well as SUA did even on the 290th day post-implantation. They also found that the decoding performance of LFPs was similar to that of SUA no
2. Methods 2.1. Behavior task
All experimental procedures conformed to the Guide for the Care and Use of Laboratory Animals (China Ministry of Health) and were approved by the animal care committee at Zhejiang University, China. Three male macaque monkeys (B01, B03 and B04) were trained to control a cursor to perform a radial-4 center out task by operating a joystick as described in our previous study [40]. In brief, a vertical computer screen was positioned in front of the monkey in a primate chair. The experiment began when two circles were presented on the screen, an orange circle representing the target and a blue circle controlled by a joystick. The joystick was free to be moved in 2D horizontal plane. Once the target circle was presented in the center of the screen, the monkey was requested to control the blue circle to hit the target circle by moving the joystick 2
D Wang et al
J. Neural Eng. 11 (2014) 036009
in a horizontal plane. The monkey had to hit the target circle within 2 s and hold for 300 ms to obtain rewards. Then, the center circle disappeared and a randomly chosen peripheral target appeared. The monkey had to hit the target circle within 2 s and hold for 300 ms to obtain rewards again.
including the amplitude of LMP and seven spectral powers of aforementioned frequency bands. The kinematic data of hand were down-sampled at 10 Hz. All kinematic data and signal features were normalized by z-score transformation for decoding.
2.2. Electrode implantation and neural recording
2.4. The characteristics analysis of neural signals
All monkeys were chronically implanted with 96-microelectrode arrays in a 10 × 10 configuration (Blackrock Microsystems Inc., 4.2 × 4.2 mm2). In monkeys B01 and B04, the arrays were implanted in the arm areas of the right primary motor cortex (M1) contralateral to the arm used to perform the task. In B03, the array was implanted in the arm area of the left dorsal premotor cortex (PMd) contralateral to the arm used. The details of surgical implantation procedures have been described in our previous study [40]. Neural signals were recorded with Cerebus multichannel data acquisition system (Blackrock Microsystems Inc., USA) at a sample rate of 30 kHz for each channel. Ninety-six channels of neural signals were recorded for monkeys B01 and B04, 64 channels of neural signals were recorded for monkey B03 due to the technical constraints. The joystick position indicating the hand position was measured with potentiometers and recorded synchronously with the neural signals by two analogue channels of Cerebus multichannel data acquisition system at 1 kHz sample rate.
We examined the characteristics of neural signals to show the degradation of neural recording over time. First, we examined the number of LFP channels, SUA and MUA recorded across sessions. The channels with obvious artifacts were excluded in our analysis. We further examined the variability of the signal features over time. For SUA and MUA, we calculated their mean firing rates across electrodes with observed spikes in the first recording session for each monkey in each session. For LFPs, we calculated the mean power of each frequency band and the mean amplitude of LMP in 100 ms window across channels in each session. 2.5. Directional tuning analysis
To study the variability of movement information contained in three types of neural signals over time, we examined directional tuning properties of their features. The signal features obtained from the radial-4 center-out task were fitted with a cosine directional tuning model [43],
2.3. Signal preprocessing and feature extraction
y = a + b × cos(x − φ),
To obtain spike signals, raw data were first filtered by Butterworth high-pass filter at 250 Hz and then processed with a spike-detection threshold that was 4.5 times the root mean square (RMS) of baseline signal. The threshold-crossing but unsorted spikes were defined as MUA and the count of unsorted spikes in each 100 ms time window was used as its feature. Commercial software (Offline Sorter, Plexon Inc., USA) was further used to sort the threshold-crossing spikes into different SUA. Firing number in each 100 ms time window was used as the SUA feature. To obtain LFPs, raw data were digitally filtered by a bandpass filter between 0.3 and 500 Hz and down-sampled at 2 kHz. The channels with obvious artifacts (determined by visual inspection) were excluded from analysis for each session. First, the LFP time series of an entire session were segmented into 300 ms width windows with 200 ms overlapped. Then multi-taper spectral estimation approach was employed to obtain the power spectrum in each window of all the channels [41]. Finally the frequency spectrum of LFPs (0.3–400 Hz) in each channel was further partitioned into the seven different frequency bands as described in [26]: δ (0.3–5 Hz), θ –α (5–15 Hz), β (15–30 Hz), γ 1 (30–50 Hz), γ 2 (50–100 Hz), γ 3 (100–200 Hz) and the broad high frequency band (bhfLFP: 200–400 Hz). For each frequency band, the spectrum power was obtained by summing up all the estimated power values within that band. In addition, local motor potential (LMP) [42], which was calculated as the moving average of time domain LFP signal within 100 ms window, was used as another LFP feature. Overall, there were eight types of LFP features
(1)
where x represents the direction of movement; y corresponds to signal features; a denotes the baseline response; φ is the preferred direction and b indicates the tuning depth. The p-value obtained from the regression test was used to determine whether the signal was significantly tuned to direction or not. Here, we examined the number of significantly directional tuning LFP channels, SUA and MUA over time. For each SUA or MUA, if the regression test yielded p < 0.01, it was considered to be significantly tuned to direction and we counted it as a significantly directional tuning SUA or MUA. For LFPs, if the regression test yielded p < 0.00125 for any of eight LFP features in a channel, we counted this channel as a significantly directional tuning LFP channel. We studied the number of significantly directional tuning LFP channels, SUA and MUA from all the electrodes recorded in the arrays in three monkeys to show the effect of neural recording degradation on directional tuning. To further examine the effect of neural non-stationarity on directional tuning, we also analyzed the directional tuning of neural signals in the channels where spike signals were recorded in both the first session and the last session within 120 days after the first session. Because there was clear neural degradation in B03 and late sessions in B01, we chose the sessions within 120 days after the first recording session in B04 and B01 to assess the effect of neural non-stationarity on directional tuning. In addition, we also examined the number of significantly directional tuning features in each LFP feature (see supplementary method, available from stacks.iop.org/JNE/11/036009/mmedia). 3
D Wang et al
J. Neural Eng. 11 (2014) 036009
each test session was divided into five segments and the five decoding results were averaged to get the final decoding result for this session. For each session, the five decoding results of LFPs were compared with those of MUA in static decoder by one-tailed paired Student test. In addition, chance level was calculated for discrete decoding by shuffling the target directions and repeating the decoding process above. The whole process was repeated 1000 times and the 50 highest accuracy values were averaged to get the chance performance with a 95% confidence interval. A box plot was used to exhibit the stability of decoding performance across sessions. On each box, the central mark was the median value of decoding performance across sessions and the edges of the box were the 25th and 75th percentiles.
2.6. Discrete decoding
To explore how the offline discrete decoding performance changed over time, we used support vector machines (SVMs) to decode the movement directions in each session. Here the open source package ‘libsvm-mat-3.11’ was used to implement the SVMs decoding [44]. The signal features in four bins from 200 ms before to 200 ms after the movement onset were selected as the decoder input and the decoder was retrained for each session. 2.7. Continuous decoding
To estimate the offline continuous decoding performance, a Kalman filter (KF) [45] was used here to decode the position and velocity of hand. The Kalman filter was composed of the following generative model and system model: zk = Ak xk + pk ,
(2)
xk+1 = Hk xk + qk ,
(3)
3. Results Neural activities and kinematic data were collected from three monkeys, which were recorded over 230, 290 and 690 days post-implantation for B04, B01 and B03, respectively. Figure 1 shows the temporal distribution of all recording sessions used in this study. Because the monkeys also performed other tasks during their recording periods, the data used in this study were collected only from sessions when they performed these radial4 center out tasks. The numbers of recording sessions were 17, 23 and 21 for monkeys B04, B01 and B03, respectively. Each session lasted for 25 min and contained more than 300 trials. These sessions spanned 111, 183 and 400 days for B04, B01 and B03, respectively. All monkeys were well-trained before implantation and the success rates for target acquisition in these sessions were 97% ± 4% (B04, mean ± SD), 97% ± 2% (B01, mean ± SD) and 96% ± 3% (B03, mean ± SD).
where xk = [x, y, vx , vy ]Tk represents the position and velocity of hand at time tk = kt (t = 100 ms in this study); zk denotes the signal features at time tk ; A is the observation matrix that linearly relates the hand state to the signal features; H is the state matrix that linearly relates the hand kinematics at time tk+1 to the state at time tk ; pk ∼ N(0, T) and qk ∼ N(0, W ) are the observation and state Gaussian noise respectively with covariance matrices T and W . In our study, both retrained decoder and static decoder were used to comprehensively examine the long-term decoding stability of neural signals. The retrained decoder was recalibrated for each session. We employed it to evaluate the effect of neural recording degradation when comparing the decoding performance of LFPs, SUA and MUA over time. On the other hand, the static decoder was built in the first session and used to decode the kinematics in the consequent sessions without any recalibration. It was used to evaluate the joint effects of neural recording degradation and non-stationarity of neural activities on decoding performance. Because it was hard to track the same SUA across sessions, only MUA and LFPs were examined in the static decoder. Considering that the static decoder was fixed across all sessions, if no MUA was detected in a channel through a session, we set the MUA feature in this channel to zero in the session. Likewise, if a channel contained obvious artifacts in a session, all LFP features in this channel were set to zero in LFP decoding in this session.
3.1. Neural signal recording stability
Figure 2 shows the number of LFP channels, SUA and MUA recorded from arrays across sessions in three monkeys. We found that many channels gradually lost their capability of recording SUA or MUA over time and that most of the curves show significantly decreasing trends. The results of the linear regression method showed that the number of SUA declined by 6.0% (B04, p < 0.01), 8.27% (B01, p < 0.01) and 7.38% (B03, p < 0.01) per month on average and the number of MUA declined by 6.99% (B04, p < 0.01), 7.50% (B01, p < 0.01) and 7.36% (B03, p < 0.01) per month on average. The number of LFP channels recorded also showed a decreasing trend over time in the three monkeys, but they decreased much more slowly than SUA and MUA with the declining ratios of 1.96% (B04, p < 0.01), 0.41% (B01, p < 0.05) and 1.15% (B03, p < 0.01) per month on average. It was consistent with Flint’s study that more and more channels were removed from decoders over time due to noise [22]. Figure 3 shows the variability of different signal features over time. We found that different signal features decreased with different ratios (table 1). Meanwhile, the LFP features exhibited gradually decreasing trends over time while the SUA and MUA features decreased much more quickly across sessions.
2.8. Performance evaluation
For discrete decoding, classification accuracy was used to evaluate the decoding performance. To evaluate the decoding performance of continuous kinematics, we chose the correlation coefficient (CC) as the metric. It measures the similarity between the decoded kinematics and the corresponding actual values. For the retrained decoders, fivefold cross-validation was used to get the final results [23]. For each session, the five decoding results from the five-fold crossvalidation of LFPs were compared with those from SUA and MUA by one-tailed paired Student test. For the static decoder, 4
D Wang et al
J. Neural Eng. 11 (2014) 036009
B04(M1)
B01(M1)
B03(PMd)
100
200
300 400 500 Time since implantation(days)
600
700
Figure 1. Time distribution of neural recording sessions. The x-axis denotes the number of days after the electrodes were implanted, the y-axis labels the three monkeys and their corresponding recording cortices in parentheses respectively. Each bar in the line represents one recording session.
80
60 40 20 0 120
LFP ** SUA** MUA **
100
60 40 20
140 160 180 200 220 240 Time since implantation(days)
LFP ** SUA** MUA**
80
LFP * SUA** MUA**
Number
80 Number
100
Number
100
60 40 20
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
Figure 2. The number of LFP channels, SUA and MUA recorded from three monkeys over time. Each marker in the figures represents the result of one dataset. The five-pointed star on the y-axis indicates the number of neural channels recorded for the monkey. Dashed lines indicate the discontinuity in time, which is also labeled on the x-axis. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01). Table 1. The decreasing rate per month for different neural features.
Monkey B04 B01 B03
LMP
δ
θ –α
β
1.75% 1.85% 1.98% 0.20% 4.50% 4.43% 4.80% 4.38% −0.11% −0.04% −0.03% −0.25%
γ1
γ2
γ3
bhfLFP
SUA
MUA
0.06% 2.57% 0.59%
0.49% 1.20% 1.25%
0.15% 0.42% 2.46%
0.76% 0.85% 2.82%
9.36% 12.60% 7.15%
9.31% 12.64% 7.12%
bhfLFP was significantly tuned in all 17 sessions. Similar results were also found in many other channels in the three monkeys. It is also noted that preferred directions of bhfLFP and MUA were variable across sessions. Furthermore, we examined the directional tuning stability of three types of neural signals by assessing the number of significantly directional tuning LFP channels, SUA and MUA across sessions (figure 5(a)). We found they all showed declining trends over time, but LFPs decreased much more slowly than the other two types of signals. Especially, it clearly shows that many LFP channels were still significantly tuned to direction even when no SUA or MUA was recorded from monkey B03 after 671 days post-implantation. We also found that many more LFP channels were tuned to direction than SUA and MUA across all sessions in the three animals (figure 5(a)) and the numbers of two LFP features with directional tuning (γ 3 and bhfLFP) were significantly higher than those of SUA and MUA across sessions (onetailed paired Student test, p < 0.05, supplementary figure S1, available from stacks.iop.org/JNE/11/036009/mmedia).
3.2. Directional tuning stability
By analyzing directional tuning properties of neural signals, we found that all three types of neural signals could be significantly tuned to movement directions, which was similar to the previous studies [18, 23, 27]. For example, figure 4(a) illustrates the directional tuning of LFPs, SUA and MUA recorded from channel 91 of monkey B04 on 144th day post-implantation. Six LFP features in this channel showed significantly directional tuning, but with different preferred directions and different tuning depths. MUA and SUA were tuned to the same direction because only one unit was found in this channel. We also found that multi LFP features from the same channel were tuned to different movement directions in many other channels, which suggested these directional tuning signal features could complement each other and be combined in neural decoding. Figure 4(b) further shows the tuning directions of bhfLFP and MUA across all 17 sessions in channel 91 of monkey B04. We found that MUA was significantly tuned in only four recording sessions while 5
D Wang et al
J. Neural Eng. 11 (2014) 036009
(a)
**
**
120 100 80 60 40 20 120
**
**
* ***
50 40 140 160 180 200 220 240 Time since implantaton(days)
5
** **
100 80 60 40
* ***
** **
120 100 80 60 40 20 280 300 320 560 580 600 620 640 660 680 700 Time since implantaton(days)
**
Mean amplitude
Mean amplitude
60
10
*** *
120
*
70
*** *
20 100 120 140 160 180 200 220 240 260 280 300 Time since implantaton(days)
140 160 180 200 220 240 Time since implantaton(days)
**
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
Mean pow power(10*log) ower(10*log)
**
5
70 60 50 40 100 120 140 160 180 200 220 240 260 280 300 Time since implantaton(days)
Mean amplitude
Mean power(10*log)
*
10
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
140 160 180 200 220 240 Time since implantation(days) *
Mean firing ra rrate(Hz) ate(Hz)
Mean firing ra rrate(Hz) ate(Hz)
0 120 0
Mean power(10*log)
Mean firing rate(Hz)
5
120
**
15
15
10
(b)
**
**
**
15
70 60 50 40 280 300 320 560 580 600 620 640 660 680 700 Time since implantaton(days)
Figure 3. Stability of signal features for three monkeys. (a) Mean firing rates of SUA and MUA across channels with observed spikes in the first recording session in our study. The number of channels with observed spikes in the first session was 70, 37, 49 for B04, B01 and B03, respectively. (b) The mean power in each frequency band and the mean LMP amplitude averaged across channels in 100 ms time window over time. Each marker in the figures represents the result of one dataset. Dashed lines indicate the discontinuity in the time which is also labeled in the x-axis. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01).
LFPs showed significantly higher performance in most late sessions from B01 and B03 (sessions after 240 days in B01 and after 620 days in B03, one-tailed paired Student test, p < 0.05). We further observed that the classification accuracy of LFPs maintained above 0.90 and that the decreasing trend was only found in the sessions after 600 days post-implantation in B03. Especially, it is worth noting that the decoding accuracy of LFPs was still higher than 0.53 even when no SUA or MUA was recorded from B03 array after 671 days postimplantation. When comparing the decoding performance among three types of neural signals across all sessions, we found that LFPs had higher mean values with less variance than SUA and MUA in both B01 and B03 (figure 6(b)). The decoding performances of three types of neural signals neither have significant declination nor show difference between each other in B04, which might due to the quality of neural signal not seriously decaying. In addition, we found that the decoding performances of SUA and MUA were very close in all sessions across three monkeys, which was consistent with the results from other studies [19, 20].
Figure 5(b) further shows the effect of neural non-stationarity on directional tuning in B04 and B01. In these chosen channels, we found that the number of directional tuning LFP channels was more stable than the other two signals across sessions. There were high values for SUA and MUA in the first and last sessions in figure 5(b), which might be due to no spike signals being recorded in some of these chosen channels in the middle sessions. The variability of the number of each directional tuning LFP feature in these chosen channels is shown in supplementary figure S2, available from stacks.iop.org/JNE/11/036009/mmedia. We also found that the number of directional tuning bhfLFP in these chosen channels was most stable among LFP features and more stable than that of SUA and MUA. All these results suggested that directional tuning information in LFPs remained stable for a long time. 3.3. Discrete decoding stability
We offline decoded movement directions using three types of neural signals in 2D center out task and compared their decoding performance in the long-term recording. Figure 6(a) shows that the decoding performance of three types of neural signals gradually declined over time in B01 and B03. We found that in early sessions the decoding performance of LFPs was comparable to those of SUA and MUA in three animals and
3.4. Continuous decoding stability
We offline decoded hand position and velocity to further compare the long-term decoding performance of LFPs, SUA 6
D Wang et al
J. Neural Eng. 11 (2014) 036009
(a)
10 Firing rate(Hz)
Frequency(Hz)
400
200
0
5
0 0 Time(s)
1
0
1 Time(s)
10 0 -10
(b)
Figure 4. Examples of directional tuning analysis from monkey B04. (a) Time-resolved power spectrum of a sample channel for different directions (left) and directional modulation of the firing rate of a sample unit (right). The data come from channel 91 recorded on 144th day post-implantation. The first dashed line indicates the movement onset and the second dashed line indicates the movement offset. The data in the left figures are normalized based on the mean of the data before movement onset. The circles in the center of figures showed the preferred directions and tuning depth of each signal feature. The stars represent the signal feature was significantly tuned to direction (p < 0.01). (b) Circles show the preferred directions of bhfLFP and MUA in channel 91 across sessions. The two red lines represent the preferred directions of bhfLFP and MUA 144 days post implantation respectively. Because there were 13 sessions in which no MUA was recorded in channel 91, only four preferred directions were shown in the figure for MUA.
direction were not significantly different (one-tailed paired Student test, p > 0.05). Similar results were also observed in B01. Compared with discrete decoding, the result from continuous decoding also showed bigger fluctuations and more serious degradation in B03, which might be relevant to the complexity of the decoding task. Figure 9 shows that LFPs achieved significantly better decoding performance than SUA and MUA did in most kinematics (one-tailed paired Student test, p < 0.05), which further demonstrated the advantages of LFPs in long-term decoding stability. In addition, we found the decoding performance of SUA was a little better than that of MUA with statistical significance (one-tailed paired Student test, p < 0.05). To study the joint effects of neural recording degradation and non-stationarity of neural activities on long-term decoding performance, we further used static decoder to assess decoding performance of MUA and LFPs across sessions. Because there was a break of over 200 days during the B03 recording, we only examined the performance of static decoder across sessions in B04 and B01 respectively. Figure 10 shows the results of LFPs and MUA obtained from both static and retrained decoder across sessions. We observed that the decoding
and MUA. The results from KF decoder showed both SUA and MUA significantly linearly declined in decoding performance across sessions in three monkeys (figure 7, linear regression, p < 0.01). In contrast, LFPs only showed significantly linearly declining decoding performance in B03 (linear regression, p < 0.01). Figure 8 shows some samples of Y-position decoding based on LFPs, SUA and MUA, respectively. All these types of neural signals achieved good decoding performance when good quality neural signals were recorded. From figure 7, we also found that the CCs obtained from LFPs in most late sessions from B01 and B03 were much higher than those of SUA and MUA. Especially when no spike signals was detected in B03 after 671 days post-implantation, kinematic information could still be extracted from LFPs (CC: from 0.15 to 0.33). Furthermore, we found that the continuous decoding performance of LFP outperformed those of SUA and MUA in some sessions where no significant difference was found in discrete decoding performance between them. For example, in most of the late sessions from B04 (after 160 days postimplantation), we found LFPs achieved significantly higher CCs than SUA and MUA did (one-tailed paired Student test, p < 0.05) while their classification accuracies of movement 7
D Wang et al
J. Neural Eng. 11 (2014) 036009
100
60
0 120
60 40 20
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
140 160 180 200 220 240 Time since implantation(days)
Directional tuning number
(b)
LFP SUA ** MUA**
50
Directional tuning number
20
LFP SUA ** MUA **
80
40 30 20 10 0 120
LFP SUA MUA 140 160 180 200 220 240 Time since implantation(days)
Directional tuning number
80
40
100
100 Directional tuning number
Directional tuning number
(a) LFP ** SUA ** MUA **
80 60 40 20
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
30
20
LFP SUA MUA
10
0 100
120 140 160 180 200 220 Time since implantation(days)
240
Figure 5. The number of significantly directional tuning LFP channels, SUA and MUA over time. (a) Each marker in the figures represents the result of one session. Dashed lines indicate the discontinuity in time, which is also labeled in the x-axis. (b) Only the channels with observed spikes in both first session and last session in the figures were examined for the directional tuning. Each marker in the figures represents the result of one session within 120 days after the first session. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01).
1
1
0.8
0.8
0.8
0.6 0.4
LFP SUA MUA
0.2
0.6 0.4
0.6 0.4
LFP ** SUA ** MUA **
0.2
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantaton(days)
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantaton(days)
1
1
0.8
0.8
0.8
0.6 0.4
0.6 0.4
LFP
SUA
MUA
0
0.6 0.4 0.2
0.2
0.2 0
Accuracy
1
Accuracy
Accuracy
LFP SUA ** MUA **
0.2
0 120 140 160 180 200 220 240 Time since implantaton(days)
(b)
Accuracy
1
Accuracy
Accuracy
(a)
LFP
SUA
MUA
0
LFP
SUA
MUA
Figure 6. Direction decoding performance based on LFPs, SUA and MUA using retrained decoder over time. (a) Direction decoding accuracy of LFPs, SUA and MUA over time. Each marker represents the result of one session. The downward-pointing triangle means that the decoding accuracy of LFPs was significantly better than that of SUA and MUA in that session (one-tailed paired Student test, p < 0.05). Dashed lines indicate the discontinuity in time. (b) The distribution of direction decoding accuracy for LFPs, SUA and MUA for three monkeys, respectively. On each box, the central red mark is the median, the edges of the box are the 25th and 75th percentiles and red ‘+’ labels the outliers. Dashed line indicates the chance level. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases (∗ p < 0.05, ∗∗ p < 0.01).
performance obtained from static decoder was significantly worse than that from retrained decoder in both monkeys (onetailed paired Student test, p < 0.05). Although significantly decreasing trends in static decoding were found in both types of neural signals across sessions, LFPs provided significantly
better performance than MUA in static decoding in most late sessions, which were labeled with downward-pointing triangles in figure 10 (one-tailed paired Student test, p < 0.05). Figure 11 further shows that LFPs outperformed MUA in static decoder in long-term decoding stability in B01. The variances 8
D Wang et al
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4 0.2
0.2
LFP ** SUA ** MUA U **
1
1
0.8
0.8
0.8
0.6
0.6
0.6
LFP SUA ** MUA **
CC
0.4
0.4 0.2
CC
1
140 160 180 200 220 240 Time since implantation(days)
LFP SUA ** MUA **
0.4 0.2
LFP ** SUA ** MUA **
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4 0.2 0 120
LFP * SUA * MUA **
0.4 0.2
CC
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
140 160 180 200 220 240 Time since implantation(days)
LFP SUA ** MUA **
0.4 0.2
LFP ** SUA ** MUA **
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4 0.2 0 120
140 160 180 200 220 240 Time since implantation(days)
LFP * SUA ** MUA ** 140 160 180 200 220 240 Time since implantation(days)
0.4 0.2
CC
CC
X-velocity
0.4
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
0.2
CC
0.2
*
LFP SUA ** MUA **
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
0 120
Y-velocity
0.4
*
CC
CC
Y-position
0 120
LFP SUA MUA
CC
1
CC
1
CC
CC
X-position
J. Neural Eng. 11 (2014) 036009
LFP SUA ** MUA **
0.4 0.2
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
LFP ** SUA ** MUA **
0 280 300 320 560 580 600 620 640 660 680 700 Time since implantation(days)
Figure 7. Continuous decoding performance using retrained decoder over time. The figures show the decoding performance for position and
velocity based on LFPs, SUA and MUA respectively over time for three monkeys. The results were displayed by correlation coefficient (CC). Each marker indicates one dataset. The downward-pointing triangle means that the decoding performance of LFPs was significantly better than that of SUA and MUA for that kinematics in that session (one-tailed paired Student test, p < 0.05). Dashed lines indicate the discontinuity in time. Linear regression was used to test if the trends significantly linearly decreased. Asterisks represent that the curve significantly linearly decreases over time (∗ p < 0.05, ∗∗ p < 0.01).
could cause the decreasing decoding performance when the quality of neural recording seriously decayed. As the number of directional tuning signals decreased over time, the decoding performance showed different decreasing trends in discrete and continuous decoding. We found that many more directional tuning signals were needed to keep a high level in continuous decoding, compared with the discrete decoding. In discrete decoding, we found that the decoding performance of LFPs remained at a high level and did not drop sharply until the number of tuned LFP channels decreased to less than about 25 in B03 (figure 12). Similar results were observed when the number of tuned SUA and MUA decreased to less than 20 in both B01 and B03. It was also noted from figure 5(a) that the number of tuned LFP channels was still higher than 25 when the number of tuned SUA or MUA decreased to less than 20 during recording degradation.
of decoding performance were close among LFPs and MUA in static decoder in B04, it might be relevant that the quality of neural recording did not decay seriously in short time and the advantage of long-term stability of LFPs was not shown. All these results suggested that LFPs were more stable than MUA in long-term decoding. 3.5. Correlation analysis
We further examined the correlation between the number of directional tuning signals and decoding performance. The results are shown in figure 12 and supplementary figure S3, available from stacks.iop.org/JNE/11/036009/mmedia. From the results, we found that the number of directional tuning signals was nonlinearly correlated with decoding performance and the decline in the number of directional tuning signals 9
D Wang et al
J. Neural Eng. 11 (2014) 036009
LFP decoding
actual trajectory decoded trajectory
5
0 CC=0.71 -5
0
20
40
60
80
100 120 time(s) SUA decoding
140
160
180
200
actual trajectory decoded trajectory
4 2 0 -2
CC=0.79 -4
0
20
40
60
80
100 120 time(s) MUA decoding
140
160
180
200
actual trajectory decoded trajectory
4 2 0 -2
CC=0.78 -4
0
20
40
60
80
100 time(s)
120
140
160
180
200
Figure 8. Examples of decoding performance based on LFPs, SUA and MUA for Y-position. This dataset was recorded 115 days
post-implantation from monkey B01. The predicted trajectory was close to the actual trajectory and most of the peaks could be decoded correctly using these three types of neural signals.
* *
*
*
* *
*
*
1
*
0.8
0.8
0.6
0.6 CC
CC
*
0.4 0.2 0 X-position Y-position X-velocity
*
*
*
*
*
*
0.8
*
* * *
*
*
*
* *
*
0.6
0.4 0.2
Y-velocity
*
1
*
* *
CC
1
0.4 0.2
0 X-position Y-position X-velocity
Y-velocity
0 X-position Y-position X-velocity
Y-velocity
Figure 9. The distribution of continuous decoding performance using retrained decoder for LFPs, SUA and MUA. The results were displayed by correlation coefficient (CC) for position and velocity for each monkey. On each box, the central red mark is the median, the edges of the box are the 25th and 75th percentiles and red ‘+’ labels the outliers. The asterisk above LFPs and SUA/MUA boxes means that the decoding performance of LFPs was significantly better than that of SUA/MUA. The asterisk above SUA and MUA boxes means that the decoding performance of SUA was significantly better than that of MUA. One-tailed paired Student test was performed here (p < 0.05). 10
D Wang et al
J. Neural Eng. 11 (2014) 036009
1 CC
CC
X-position
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
1 CC
CC
Y-position
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
1 CC
CC
X-velocity
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
1 CC
CC
Y-velocity
1
0.5
0 120
140 160 180 200 220 Time since implantation(days)
0.5
0 100 120 140 160 180 200 220 240 260 280 300 Time since implantation(days)
240
Figure 10. Continuous decoding performance using static decoder over time. The static decoder prediction results of LFPs and MUA were
obtained by using a session to train the decoder and decoding the following several sessions selected respectively. The decoding performance using retrained decoder for LFPs and MUA are shown here for comparison. The results are displayed by CC. Each marker indicates one dataset. The downward-pointing triangle means that the decoding performance of LFPs based on static decoder was significantly better than that of MUA for the kinematics in that session (one-tailed paired Student test, p < 0.05).
properties and offline decoding performance. Two factors were considered here: the degradation of neural recording and the non-stationarity of neural activities. The results showed that the number of LFP channels, SUA and MUA recorded decreased over time, but LFPs decreased more slowly than the other two signals. The mean power in each frequency band in LFPs decreased over time along with decreasing mean firing rates of spikes. The number of directional tuning LFP channels also decreased much more slowly than that of SUA and MUA. The variable preferred directions for the same signal features showed the effect of neural non-stationarity on directional tuning. When the quality of neural signals seriously decayed in chronic recording, LFPs achieved better decoding performance than SUA and MUA did in retrained
Therefore, sharply dropping decoding performance in LFPs appeared later than for SUA/MUA, which was also found in continuous decoding analysis (supplementary figure S3, available from stacks.iop.org/JNE/11/036009/mmedia). These results suggested that the less decoding performance decline of LFPs in chronic recording was relevant to LFPs retaining directional tuning information more durably than SUA and MUA. 4. Discussion In this study, we examined the long-term stability of LFPs and compared it with that of spike signals (SUA and MUA) in the characteristics of neural signals, directional tuning 11
D Wang et al
J. Neural Eng. 11 (2014) 036009
1
1 *
0.8
0.8
0.6
0.6 CC
CC
*
0.4
*
0.4 0.2
0.2 0 X-position
Y-position
X-velocity
0 X-position
Y-velocity
Y-position
X-velocity
Y-velocity
Figure 11. The distribution of continuous decoding performance using static decoder for LFPs and MUA. The results were displayed by
1
1
0.8
0.8
0.8
0.6 0.4 0.2 0 100
B04 B01 B03 80 60 40 20 Directional al tuning number
0.6 0.4 0.2
0
Accuracy
1
Accuracy
Accuracy
correlation coefficient (CC) for position and velocity for each monkey. On each box, the central red mark is the median, the edges of the box are the 25th and 75th percentiles and red ‘+’ labels the outliers. The asterisk above boxes means that the decoding performance of LFPs was significantly better than that of MUA using static decoder (one-tailed paired Student test, p < 0.05).
0 80
B04 B01 B03
0.6 0.4 0.2
60 40 20 Directional Di ti l ttuning i number b
0
0 80
B04 B01 B03 60 40 20 Directional tuning t ning number n mb
0
Figure 12. The correlation between the number of directional tuning signals and their direction decoding accuracies. For LFPs, we plot the
scatter figure for the number of directional tuning LFP channels versus decoding accuracy (left). For SUA/MUA, we plot the figure for the number of directional tuning SUA/MUA versus decoding accuracy (middle/right). Dot marker represents individual session result. Note that the decoding accuracy remained stable at a high level when the directional tuning number was more than 25 for LFPs and 20 for SUA and MUA. It also shows when the directional tuning number was less than 25 for LFPs and 20 for SUA and MUA, poorer decoding accuracy was obtained with fewer directional tuning signals. The decoding performances here were significantly correlated with the number of directional tuning signals (spearman correlation, p < 0.05).
Furthermore, we examined the effect of neural degradation on LFPs recording quality over time. To our knowledge, this is the first study to systematically examine the degradation of LFPs in chronic recording. Declining trends were found in both the number of LFP channels recorded and mean power in each frequency band. The number of directional tuning LFP channels also decreased due to the degradation of neural recording. The close correlation between the number of directional tuning signals and decoding performance further suggested the degradation of neural recording was one reason causing the decreasing decoding performance in chronic recording. The results of correlation analysis also indicated that the number of directional tuning signals could be used as a good measure of BMI performance compared with the amplitude used in a previous study [19]. Our results in this study showed that both LFPs and spike signals decayed over time in the signal quality under the effect of neural recording degradation, which might be related to similar reasons, such as tissue reaction at the recording site, engineering or material failures, micromotion and so on [50–56]. However, multiple mechanisms underlying degradation of neural recording were still unclear. The long-term tissue reaction or material examination needs to be further explored
decoder and outperformed MUA in static decoder. Especially, we provided strong evidence that LFPs could still be used to decode kinematics even when neither SUA nor MUA could be recorded from arrays. All results suggested that LFPs could provide better performance in durability and stability. 4.1. The impact of degradation of neural recording on long-term decoding performance
Previous studies have shown that the recording quality of spike signals (SUA and MUA) decayed over time and it was hard to record them over several years except for several groups [19, 32, 46, 47]. Williams et al showed large declines in SUA using wire microelectrode arrays over months [48]. Vetter et al reported that mean signal-to-noise ratio (SNR) of SUA decreased slowly over time using Michigan microelectrode arrays [49]. Chestek and Gilja et al found that the peak-to-peak voltage of action potentials decreased over months and was significantly correlated with changes in offline performance using all three metrics in one array which was recorded over three years [19, 31]. Here we also got declining quality of spike signals over time, which was consistent with the aforementioned studies. 12
D Wang et al
J. Neural Eng. 11 (2014) 036009
in chronic recording to develop new technology for practical BMI application.
declining trends in decoding performance over time, but LFPs decreased much more slowly and still got significantly better decoding performance than spikes in most recording sessions where the quality of spike signals was poor. The difference in decoding stability between them might be due to the signal characteristics they represented. Spike signals are extracellular action potentials of neurons recorded close to electrode tip, which have high spatial selectivity. LFPs are low frequency signals (
