Journal of Neuroscience Methods 264 (2016) 78–85
Contents lists available at ScienceDirect
Journal of Neuroscience Methods journal homepage: www.elsevier.com/locate/jneumeth
An improved method for measuring mismatch negativity using ensemble empirical mode decomposition Chun-Hsien Hsu a,∗ , Chia-Ying Lee a , Wei-Kuang Liang b a b
Institute of Linguistics, Academia Sinica, No. 128, Section 2, Academia Road, 115 Taipei, Taiwan, ROC Institute of Cognitive Neuroscience, National Central University, Jhongli, Taiwan, ROC
h i g h l i g h t s • We adopt EEMD method and evaluate the statistical significance of ERM measurements. • ERMs can reflect effects of lexical tone changes even with a less number of trials. • EEMD method is helpful to promote clinical and developmental assessments based on EEG.
a r t i c l e
i n f o
Article history: Received 3 September 2015 Received in revised form 14 February 2016 Accepted 16 February 2016 Available online 23 February 2016 Keywords: Ensemble empirical mode decomposition Event-related mode Event-related potential Hilbert–Huang transformation Intrinsic mode function Mismatch negativity
a b s t r a c t Background: Mismatch negativity (MMN) is a component of event-related potentials (ERPs). Conventional approaches to measuring MMN include recording a large number of trials (e.g., 1000 trials per participant) and extracting signals within a low frequency band, e.g., between 2 Hz and 8 Hz. New Method: Ensemble empirical mode decomposition (EEMD) is a method to decompose time series data into intrinsic mode functions (IMFs). Each IMF has a dominant frequency. Similar to ERP measurement, averaging IMFs across trials allows measurement of event-related modes (ERMs). This paper demonstrates a protocol that adopts EEMD and Hilbert spectral analyses and uses ERMs to extract MMN-related activity based on electroencephalography data recorded from 18 participants in an MMN paradigm. The effect of deviants was demonstrated by manipulating changes in lexical tones. Results: The mean amplitudes of ERMs revealed a significant effect of lexical tone on MMN. Based on effect size statistics, a significant effect of lexical tone on MMN could be observed using ERM measurements over fewer trials (about 300 trials per participant) in a small sample size (five to six participants). Comparison with Existing Method(s): The EEMD method provided ERMs with remarkably high signal-tonoise ratios and yielded a strong effect size. Furthermore, the experimental requirements for recording MMN (i.e., the number of trials and the sample size) could be reduced while using the suggested analytic method. Conclusions: ERMs may be useful for applying the MMN paradigm in clinical populations and children. © 2016 Elsevier B.V. All rights reserved.
1. Introduction In event-related potential (ERP) studies of human auditory perception, the mismatch negativity (MMN) response has received substantial attention (Näätänen, 2003; Näätänen et al., 1978). MMN can be elicited by deviant stimuli, which occur among repetitive standard stimuli in oddball paradigms. The contrast between standard and deviant sounds is manipulated by modifying a variety of acoustic features, such as pitch (Moreau et al., 2013), duration
∗ Corresponding author. Tel.: +886 2 2652 5000x6132; fax: +886 2 2785 6622. E-mail address: [email protected] (C.-H. Hsu). http://dx.doi.org/10.1016/j.jneumeth.2016.02.015 0165-0270/© 2016 Elsevier B.V. All rights reserved.
(Chobert et al., 2012), and intensity (Schröger, 1996). MMN typically manifests as a negative ERP that peaks about 150–200 ms after deviant stimulus onset. Across these studies, a robust effect of the size of deviance has been shown in MMN. That is, as the discriminability between the standard and the deviant stimuli increases, MMN amplitudes increase, and the MMN response sometimes occurs earlier. As MMN can be elicited in the absence of attention and with no task requirements, it is particularly suitable for studying clinical populations and children. Researchers have used MMN to explore neurological mechanisms in normal participants and have demonstrated some clinical applicability. For example, MMN has been used as a valid predictor of recovery from coma (Daltrozzo
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
et al., 2007). MMN has also been used as an index for investigating schizophrenia (Michie, 2001), degenerative brain disorders (Jung et al., 2006), and dyslexia (Kujala et al., 2001). The conventional method of estimating MMN involves first estimating ERPs of standards and deviants by averaging several epochs of electroencephalography (EEG) data that correspond to onset of stimuli. Then, differences between waveforms are calculated by subtracting the ERPs of standards from those of deviants. Therefore, participants need to hear standards and deviants hundreds of times. To improve the statistical significance of ERP studies, a simple approach is increasing the number of trials. However, this approach prolongs experimental times for recording EEG data and might not be suitable for studying clinical populations and children. Some analytic methods have been used to improve measurements of MMN. These methods were based on the idea that that MMN activity might have a specific topological distribution and a dominant frequency band. To extract MMN-related activity based on its topological features, one can use independent component analysis (ICA) (Cong et al., 2010). To extract MMN activity by extracting EEG oscillation within a frequency band, there are methods such as optimal digital filtering (Kalyakin et al., 2007), wavelet decomposition (Cong et al., 2011), and empirical mode decomposition (EMD) (Cong et al., 2009). This paper will not exhaustively compare the pros and cons of these different methods. Instead, we aim to provide a practical demonstration to show that an advanced version of EMD, i.e., ensemble empirical mode decomposition (EEMD) (Wu and Huang, 2009), can be used to extract latent ERP activity from single-trial EEG data, and to evaluate the statistical significance of EEG data based on the EEMD method. EEMD and its origin method, EMD, are components of the Hilbert-Huang transformation (HHT), which has been proposed by Huang and colleagues for analyzing nonlinear and non-stationary data (Huang et al., 1998). The procedure of HHT consists of two parts: EMD and Hilbert spectral analysis. EMD is an adaptive method to decompose waveforms into several intrinsic mode functions (IMFs). Although EMD has the ability to extract ERPs, it has a drawback—the mode mixing problem (Huang et al., 2003). That is, a feature of time-frequency activities is not fixed in one IMF, which makes it difficult to determine ERPs and the corresponding IMFs. To overcome the problem of mode mixing, Wu and Huang (2009) have recommended EEMD, a noise-assisted data analysis method. The output of EEMD is a set of IMFs generated from ensemble means of trials by repeating EMD on the same signal with different sets of Gaussian noise. The technical details of EEMD have been described by Al-Subari et al. (2015a, 2015b). Since ERPs are known as low-frequency waveforms (Nitschke et al., 1998), Hilbert spectral analysis, the second step of HHT, can be used to evaluate the frequency of each IMF. Hilbert spectral analysis provides the instantaneous frequency of each IMF. According to Huang et al. (2011), the instantaneous frequency could represent the nonlinear and non-stationary signals without resorting to the mathematical artifact of harmonics. Like measuring ERPs, averaging IMFs across trials provides event-related modes (ERMs). Based on the instantaneous frequency of ERMs, ERP components can be extracted by summing ERMs (Cong et al., 2009; Williams et al., 2011; Wu et al., 2012) or using an ERM within a frequency range (Al-Subari et al., 2015b). For example, Cong et al. (2009) have demonstrated an application of EMD for analyzing MMN. In their study, time-frequency spectra of ERMs with frequencies between 2 and 8.5 Hz were used to estimate MMN waveforms, because Kalyakin et al. (2007) suggested that activities within this frequency range could reflect MMN. For ERPs of sentence comprehension, Williams et al. (2011) used EMD to extract ERMs between 1 and 10 Hz. For olfactory ERPs, Wu et al. (2012) used ERMs between 0.5 and 11.5 Hz. For visual perception,
79
Al-Subari et al. (2015b) demonstrated that ERMs could reflect P100 and N200 elicited in a contour integration task. To date, however, few studies have demonstrated EEMD analyses of EEG data. It is important to explore the extent to which the EEMD could improve the sensitivity of measuring ERP components. In particular, with regard to measuring MMN responses, answering this question would be helpful to promote clinical and developmental assessments based on MMN. In the present study, the dataset of adult EEG in Cheng et al. (2013) was utilized. The effect of lexical tone changes on MMN amplitude in early and late time windows has been addressed in detail in previous studies (Cheng et al., 2013). Specifically, Cheng et al. use Mandarin lexical tones in a multiple-deviant oddball paradigm. The results showed that MMN activity for acoustically dissimilar contrasts (described below) was significant between 100 ms and 200 ms. Therefore, the present study focused on mean amplitudes between 100 ms and 200 ms and evaluated the statistical significance of MMN activity in ERMs. We adopted the method of single-trial based EEMD analysis suggested in Al-Subari et al. (2015b) to extract ERMs as a measurement of MMN. Furthermore, subsets of trials were randomly sampled to explore whether MMN could be measured using ERMs with few trials.
2. Materials and methods 2.1. Experimental design and EEG dataset Since this experiment has been discussed in detail elsewhere (Cheng et al., 2013), only a brief introduction is provided in this article. The participants in the MMN experiment were 18 native speakers of mandarin Chinese (two women, age range = 18–29) with normal hearing. Participants passively listened to streams of auditory stimuli. This study was approved by the Human Subject Research Ethics Committee/Institutional Review Board of Academia Sinica, Taiwan. Written consent forms were obtained from all participants. The stimuli consisted of three Mandarin syllables with different lexical tones: yi1 “clothing” (T1), yi2 “aunt” (T2), and yi3 “chair” (T3), which share the same vowel,/i/, but carry different tonal contours. The T3 was assigned as the standard, with T1 and T2 as deviants. The T1/T3 pair represents the larger deviant contrast, and the T2/T3 pair represents the smaller deviant contrast. Each stimulus lasted 250 ms, and the inter-stimulus interval was 500 ms. The experimental session started with 20 trials of the standard, followed by 1000 trials with 80% standards and 20% deviants (10% for each deviant). During the experiment, participants were instructed to play a puzzle computer game called “super-box” silently. EEG signals were amplified by SYNAMPS2® (Neuroscan, Inc.) in DC mode, low-pass 100 Hz, and digitized at a sampling rate of 500 Hz. EEG data were recorded from 64 Ag/AgCl electrodes (QuickCap, Neuromedical Supplies, Sterling, USA), arranged according to the international 10-20 system, including a reference electrode located between CZ and CPZ and a ground electrode located between FPZ and FZ. Six additional electrodes were attached over the left and right mastoids, supra- and infra-orbital ridges of the left eye (VEOG), and outer canthi of both eyes (HEOG). Before estimating ERPs and ERMs, the EEG data were rereferenced to the average of the left and right mastoids. Continuous EEG was segmented into epochs from 100 ms prior to the onset of the stimulus to 700 ms after the onset. The first 20 trials and epochs with artifacts exceeding ±100 V were rejected. To ensure that the number of trials between the standard and deviant were comparable; only the standard trials that were preceded by at least three successive standards were included in analyses. For each participant, at least 45 accepted deviants were required to
80
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
Fig. 1. Left column: Original ERPs and ERMs of a representative participant at channel FZ; right column: Hilbert spectra of ERMs.
Fig. 2. The dominant frequency of each ERM in each channel of two representative participants.
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
be included in further analyses. Based on the selection criteria described above, the mean number of accepted deviants was 76.6, and that of accepted standards was 173.5. The following processing steps were performed using MATLAB scripts based on the Fieldtrip toolbox (Oostenveld et al., 2011) and the fast EEMD algorithms (Wang et al., 2014). 2.2. Conventional ERP pre-processing After applying a band-pass filter (1–30 Hz) and removing the activity in the pre-stimulus interval (i.e., the baseline correction), the grand averaged ERPs for standard, small deviant, and large deviant were computed for each participant and each electrode. 2.3. Measuring MMN-related ERMs Without applying baseline corrections or band-pass filtering, the same set of EEG data described in Section 2.1 was used in the following EEMD analysis. To avoid potential distortion at the boundary of the EEG segments while applying the EEMD procedure, 100 samples before and after EEG segments were included in EEMD decomposition. Therefore, the time range of EEG segments for EEMD analysis was from 200 ms before the stimuli onset to 800 ms after the onset. The analytic flow of single-trial based EEMD analysis has been described in previous studies (AlSubari et al., 2015a, 2015b). Some essential steps are described below: Step 1: For each signal of a channel in a trial, the EEMD analysis was performed with 10 times of sifting and 40 ensembles. For each time of each ensemble, the amplitude of the Gaussian noise was 10 percent of the EEG segment’s standard deviation. Each EEG segment was decomposed into eight IMFs. The last mode was the residual trend, and only IMFs 1 to 7 were considered when estimating ERMs (Al-Subari et al., 2015b). Subsequent analyses were based on IMFs between −100 ms and 700 ms. Step 2: Based on the method described in Huang et al. (2009), time-frequency spectra of IMFs were calculated for each trial and each channel. Step 3: ERMs were estimated by averaging IMFs over trials and were estimated for each IMF, each stimulus condition, and each channel (the left column of Fig. 1). Step 4: For each power spectrum estimated in Step 2, activities were summed across time. Fig. 2 shows the dominant frequency at the maximum of marginal spectrum of each ERM in each channel of two representative participants. The horizontal axis corresponds to the log10 transformed frequency. The results demonstrated that the dominant frequencies of ERMs were consistent across channels. Therefore, we followed the approach suggested by Al-Subari et al. (2015b) and averaged spectra over channels. This step revealed the event-related Hilbert spectrum corresponding to each ERM (the right column of Fig. 1). The output of this step indicated the dominant frequency of each ERM. The abovementioned steps used one participant’s EEG data to perform EEMD analysis (Step 1), Hilbert spectral analysis (Step 2), event-related averaging of IMFs (Step 3, which in turn provides ERMs) and event-related averaging of Hilbert spectra (Step 4). The left column of Fig. 1 shows the original ERP waveform and the ERMs in channel FZ of one participant, and the right column of Fig. 1 shows Hilbert spectra for the same participant. The results showed that the dominant frequencies (defined by the peak) in Hilbert spectra were around 66.3 Hz, 37.7 Hz, 18.6 Hz, 8 Hz, 3.5 Hz, 2 Hz, and 1 Hz from ERM 1 to ERM 7. After analyzing every participant’s data, the means and standard deviations of the dominant frequency are reported in Table 1. To extract MMN-related activities, this study focused on ERMs with frequencies mainly distributed between 2 and 8 Hz. For each participant, the frequency of ERM 4 and 5 were
81
Table 1 Means and standard deviations (in parentheses) of the dominant frequency of each ERM. Instantaneous frequency (Hz) ERM1 ERM2 ERM3 ERM4 ERM5 ERM6 ERM7
79.51 (11.75) 45.76 (5.33) 19.51 (1.48) 8.24 (0.52) 3.63 (0.22) 2.01 (0.00) 1.34 (0.24)
was between 2 and 8 Hz, so ERM 4 and 5 were categorized as MMNrelated ERMs. Therefore, to build up MMN-related waveforms, each stimulus type’s ERM 4 and ERM 5 were summed together for each participant. 2.4. Statistical analyses using ERPs/ERMs data For statistical analyses, there were two types of dependent variables, including the mean amplitude between 100 and 200 ms of ERPs and that of ERMs. Dependent variables were subjected to be repeated measures ANOVA analyses with conditions (standard, large deviant, and small deviant) and electrode sites (F3, FZ, F4, FC3, FCZ, FC4, C3, CZ, and C4) as within-subject factors. 2.5. Estimating MMN responses using subsets of EEG trials Single-trial EEGs were randomly sampled by taking 90%, 80% 70%, 60%, 50%, 40%, 30%, 20%, and 10% of the trials for each participant. Then, sampled trials were used to estimate ERPs and ERMs. The ANOVA analysis was also applied to mean amplitudes measured in sampled ERPs and ERMs. For each data type (ERPs and ERMs) and each percentage level of trials (from 90% to 10%), the sampling procedures and subsequent ANOVA analyses were repeated 500 times. The number of trials in each percentage level is described in Table 2. For each ANOVA analysis described in Section 2.4 and 2.5, the effect size of the condition was estimated by calculating the generalized eta-squared statistic (Bakeman, 2005; Olejnik and Algina, 2003). Each ANOVA analysis was done by using the ezANOVA program of the ez-package (Lawrence, 2013). The generalized eta-squared statistic of condition was estimated using the method described in Bakeman (2005), which was supported by the ez-package. This package was supplied in the R system for statistical computing (version 3.1.0, R Development Core Team, 2014). 2.6. Estimating inter-trial variability of mean amplitudes To evaluate the performance of EEMD analysis, we use all trials to estimate inter-trial variability of the mean amplitudes of ERPs and ERMs in the channel FZ. The mean amplitudes between 100 ms and 200 ms were averaged for each trial. Then, standard deviations were calculated for each participant and each data type. 3. Results Fig. 3a and b shows the grand-averaged ERPs and ERMs, respectively, in channel FZ. Both ERPs and ERMs reflected the typical pattern of MMN responses. That is, large deviant stimuli elicited more negative activity than standard stimuli between 100 and 200 ms after stimuli onset. On the other hand, small deviant stimuli elicited a delayed MMN response at around 300 ms. When considering all artifact-free trials, the amplitudes of ERPs revealed a significant effect of condition (F(2, 34) = 57.1, p = 0.000000000013,
82
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
Table 2 Ranges of trial numbers in each percentage level, and counts of ANOVA analyses that accepted the null hypothesis of condition effects. Number of trials
Counts of accepting the null hypothesis of condition effects
Percentage of trials
Standard (max–min)
Deviant 1 (max–min)
Deviant 2 (max–min)
Mean amplitudes of ERPs
Mean amplitudes of ERMs
90 80 70 60 50 40 30 20 10
207–98 184–87 161–76 138–65 115–55 92–44 69–33 46–22 23–11
90–39 80–34 70–30 60–26 50–22 40–17 30–13 20–9 10–4
90–45 80–40 70–35 60–30 50–25 40–20 30–15 20–10 10–5
0 0 0 0 0 0 0 2 41
0 0 0 0 0 0 0 0 4
Fig. 3. (a) Averaged ERPs of each stimulus and (b) averaged ERMs of each stimulus.
Fig. 4. Means of the mean amplitudes based on ERPs (a) and ERMs (b) by averaging activities from 100 ms to 200 ms after stimulus onset. Error bars indicate 95% confidence interval.
2 = 0.41). Fig. 4a shows the means and 95% confidence intervals of the mean amplitude of each stimulus condition based on ERPs. Consistent with the findings of Cheng et al. (2013), large deviant stimuli elicited significantly more negative activity than standard stimuli, while the difference between small deviant stimuli and standard stimuli was not significant between 100 ms and 200 ms. With regard to ERM amplitudes (Fig. 4b), there was a significant
effect of condition as well (F(2, 34) = 57.1, p = 0.0000000000003, 2 = 0.58). It is worth noting that ERMs yielded an effect size of condition of 0.58, which was higher than 0.41, the corresponding effect size drawn from ERPs. In summary, ANOVA analysis indicated that using mean amplitudes of ERMs with dominant frequencies between 2 Hz and 10 Hz could elicit the typical result of lexical tone MMN.
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
83
Table 3 Trial-wise variability of mean amplitudes between 100 ms and 200 ms. Participants
S001 S002 S003 S004 S005 S006 S007 S008 S009 S010 S011 S012 S013 S014 S015 S016 S017 S018
Fig. 5. Means of the generalized eta-square of the condition effects on the amplitudes of ERMs and ERPs.
Table 2 shows the counts of ANOVA analyses that failed to reject the null hypothesis of condition (˛ = 0.05). The results indicated that ERPs yielded significant effects of condition in ANOVA analyses when using 90–30% of data. ERMs yielded significant effects of conditions in ANOVA analyses when using 90–20% of data. To investigate the effect size, the generalized eta-square of the condition effect was computed for each ANOVA analysis. In Fig. 5, the dots indicate the mean of the effect size, and the black line indicates the level at 0.41, which was the effect size of condition while using amplitudes of ERPs with 100% of data. The results showed that, in every percentage level, ERMs yielded higher mean effect sizes than ERPs. As expected, means of effect size decreased with a decreasing number of trials. The descending pattern of effect size indicated that amplitudes of ERMs with 30% of data yielded a mean effect size of 0.428, which remained higher than 0.41. When using ERMs with 20% and 10% of data, the means of the effect size were 0.36 and 0.241, respectively. By sampling parts of trials, the analysis of effect size implied that ERMs with one third of the trials could yield a significant effect of condition, and this effect seemed to be at the same level of significance based on ERPs using 100% of data. To demonstrate this finding, three sets of difference waves were estimated as follows. Fig. 6 shows the grand-average of difference waves of large deviant stimuli (blue line) and small deviant stimuli (red line) superimposed on areas corresponding to the 95% confidence intervals. Fig. 6a shows difference waves of ERPs based on 100% of trials, and Fig. 6b shows difference waves of ERPs using the first third of trials. Comparing Fig. 6a and b, the typical influence of trial number on ERPs was observed. That is, decreasing numbers of trials would introduce noise in the ERP waveforms. Therefore, Fig. 6b shows less smooth waves and wider areas of confidence intervals than Fig. 6a. Finally, Fig. 6c shows difference waves based on ERMs using data from the first third of trials. Through visual inspection, ERMs yielded smaller confidence intervals than ERPs. Table 3 shows the trial-wise variability of mean amplitudes of each participant. For all participants, ERMs yielded smaller standard deviations than ERPs (p < 0.001, Wilcoxon signed rank test). This finding suggested that mean amplitudes of ERMs had low inter-trial variability.
Standard deviation ERP
ERM
9.36 10.69 8.65 9.84 9.95 7.92 6.13 9.01 10.32 8.69 10.6 11.93 9.79 8.89 10 10.24 9.63 8.52
4.57 5.46 3.41 4.19 4.19 4.13 3.08 4.25 4.46 5.22 5.47 6.7 4.73 4.06 5.23 4.7 5.42 4.53
4. Discussion This study demonstrated that measuring amplitudes of ERMs yielded higher effect size statistics of MMN than measuring ERPs. By evaluating effects of lexical tone MMN, the mean amplitude of ERMs revealed a better sensitivity to lexical tone changes than that of ERPs. This result implies that EEMD methods could remarkably improve the signal-to-noise ratios (SNR) and could substantially improve measurement of EEG data. By observing the effect size, it seems that effects of lexical tone contrasts on MMN could be observed with one third of the trials used in this study (about 300 trials) which is fewer than the requirement in conventional MMN paradigms (more than 1000 trials in total). While ERMs could reflect MMN with 30% of trials, the corresponding mean of the effect size (0.428) was still large. This implies that future studies could replicate the effect of condition with a sample size of less than 18 participants. Kirk (1995) introduced a procedure for estimating the sample size based on effect size statistics (see pp. 186 of Kirk, 1995). Given that the generalized eta-square measures the strength of association between levels of condition and mean amplitudes of ERPs/ERMs, one can use the design of the experiment to set parameters as follows. By using Kirk’s approach, the parameter v1 was 2 (the number of levels of conditions minus 1), and the parameter v2 was 2(n − 1) because condition was a within-subject variable. Accordingly, Kirk’s procedure suggested that, given these parameters, a factor with an effect size statistic between 0.4 and 0.5 would require at least five or six participants to reveal a significant effect (while setting ˛ as 0.05 and 1 − ˇ as 0.8). Therefore, when conducting future studies to measure Mandarin lexical tone MMN over the frontal scalp, using ERMs could allow us to decrease recording time and to use a smaller sample size. In addition, the correspondence between ERMs and EEG oscillations in the present study were consistent with the results in Al-Subari et al. (2015b). For example, in Fig. 1, ERM 1 shows a dominant frequency range above 40 Hz, which corresponds to the high gamma band. ERM 2 showed oscillations mainly distributed from 20 Hz to 60 Hz, which reflected both beta and low gamma bands. ERM 3 showed oscillation around 18 Hz, corresponding to the low beta band. ERM 4 showed a dominant frequency at around 8 Hz, which indicated an alpha oscillation. ERM 5 showed oscillation around 3 Hz, corresponding to the theta band. Finally, ERM 6 and 7 showed dominant frequency ranges below 3 Hz, which indicated the delta oscillation. The correspondences between EEG
84
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
Fig. 6. The grand-average of difference waves of large deviant stimuli (blue line) and small deviant stimuli (red line) superimposed on areas corresponding to the 95% confidence intervals. (a) ERPs using 100% of trials, (b) ERPs using the first third of trials, and (c) ERMs using the first third of trials. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.).
oscillations and the Hilbert spectra of ERMs might imply that HHT could reflect meaningful aspects of EEG oscillations. Currently, only Cong et al. (2009) have explored the time-frequency spectrum of EEG data with HHT. Future EEG studies could use ERMs and the corresponding time-frequency spectra to identify brain mechanisms of perceptual and cognitive functions. In addition to increasing the effect size, measuring ERMs has other advantages. First, the computation of EEMD is performed on each trial on each EEG channel. Since the number of channels is not a concern, the EEMD method permits high flexibility for users. On the other hand, although ICA and spatial principle component analysis are powerful methods for extracting ERPs, these methods require EEG data with multiple channels (Onton and Makeig, 2006). Secondly, according to Cong et al. (2009) and Williams et al. (2011), ERMs provide better estimation of amplitude and latency than ERPs that are extracted by wavelet analyses and band-pass filtering. This is because wavelet analyses and Fourier based methods use a prior function to estimate brain activities within the desired frequency range (e.g., the sinusoid function and the Morlet function), and a fixed prior function cannot represent signals generated from nonlinear and non-stationary systems. While exploring ERMs with EEG data, one should be aware that it is important to keep EEG data in the original form. That is, any off-line filtering of EEG data should be avoided while applying the EEMD method. For future EEG studies that are intended to measure ERMs, researchers are encouraged to set a high boundary of the low pass filter (e.g., 200 Hz), which would provide a better resolution in the high gamma band. Finally, the high pass filter might be turned off during on-line recording. In conclusion, this paper indicates a practical method of measuring MMN with the EEMD method. The results suggested that MMN could be measured using ERMs with fewer trials. Although HHT has been recently adopted in analyzing EEG data, this approach has not attracted much attention. The method presented here could be easily applied to analyze other ERP components. To utilize the EEMD
method, Al-Subari et al. (2015a) have implemented the EMDLAB toolbox as a plug-in of the EEGLAB toolbox. In addition, the MATLAB scripts developed within this study were integrated into the Fieldtrip toolbox and will be available at http://www.fieldtriptoolbox. org/.
Acknowledgments This work was supported by research grants from Academia Sinica, Taiwan (AS-99-TP-AC1 and AS-102-TP-C06). The authors would like to thank two anonymous reviewers for their detailed comments on this paper. We are grateful to Norden E. Huang, JiaRong Yeh, Chu-Lan Kao, and Chi-Hung Juan for discussions on using the HHT algorithms.
References Al-Subari K, Al-Baddai S, Tome AM, Goldhacker M, Faltermeier R, Lang EW. EMDLAB: a toolbox for analysis of single-trial EEG dynamics using empirical mode decomposition. J Neurosci Methods 2015a;253:193–205. Al-Subari K, Al-Baddai S, Tome AM, Volberg G, Hammwohner R, Lang EW. Ensemble empirical mode decomposition analysis of EEG data collected during a contour integration task. PLoS One 2015b;10:e0119489. Bakeman R. Recommended effect size statistics for repeated measures designs. Behav Res Methods 2005;37:379–84. Cheng Y-Y, Wu H-C, Tzeng Y-L, Yang M-T, Zhao L-L, Lee C-Y. The development of mismatch responses to mandarin lexical tones in early infancy. Dev Neuropsychol 2013;38:281–300. Chobert J, Francois C, Habib M, Besson M. Deficit in the preattentive processing of syllabic duration and VOT in children with dyslexia. Neuropsychologia 2012;50:2044–55. Cong F, Kalyakin I, Huttunen-Scott T, Li H, Lyytinen H, Ristaniemi T. Single-trial based independent component analysis on mismatch negativity in children. Int J Neural Syst 2010;20:279–92. Cong F, Kalyakin I, Li H, Huttunen-Scott T, Huang Y, Lyytinen H, et al. Answering six questions in extracting children’s mismatch negativity through combining wavelet decomposition and independent component analysis. Cogn Neurodyn 2011;5:343–59.
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85 Cong F, Sipola T, Huttunen-Scott T, Xu X, Ristaniemi T, Lyytinen H. Hilbert-Huang versus Morlet wavelet transformation on mismatch negativity of children in uninterrupted sound paradigm. Nonlinear Biomed Phys 2009;3:1. Daltrozzo J, Wioland N, Mutschler V, Kotchoubey B. Predicting coma and other low responsive patients outcome using event-related brain potentials: a metaanalysis. Clin Neurophysiol 2007;118:606–14. Huang NE, Chen X, Lo MT, Wu Z. On Hilbert spectral representation: a true timefrequency representation for nonlinear and nonstationary data. Adv Adapt Data Anal 2011;3:63–93. Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, et al. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc R Soc Lond A Math Phys Sci 1998;454:903–95. Huang NE, Wu MLC, Long SR, Shen SS, Qu W, Gloersen P, et al. A confidence limit for the empirical mode decomposition and Hilbert spectral analysis. Proc R Soc Lond A Math Phys Sci 2003;459:2317–45. Huang NE, Wu Z, Long SR, Arnold KC, Chen X, Blank K. On instantaneous frequency. Adv Adapt Data Anal 2009;1:177–229. Jung J, Morlet D, Mercier B, Confavreux C, Fischer C. Mismatch negativity (MMN) in multiple sclerosis: an event-related potentials study in 46 patients. Clin Neurophysiol 2006;117:85–93. Kalyakin I, Gonzalez N, Joutsensalo J, Huttunen T, Kaartinen J, Lyytinen H. Optimal digital filtering versus difference waves on the mismatch negativity in an uninterrupted sound paradigm. Develop Neuropsychol 2007;31: 429–52. Kirk RE. Experimental design: procedures for the behavioral sciences. 3rd ed. Pacific Grove, California: Brooks/Cole; 1995. Kujala T, Karma K, Ceponiene R, Belitz S, Turkkila P, Tervaniemi M, et al. Plastic neural changes and reading improvement caused by audiovisual training in readingimpaired children. Proc Natl Acad Sci USA 2001;98:10509–14. Lawrence MA. ez: easy analysis and visualization of factorial experiments; 2013, http://cran.r-project.org/package=ez.
85
Michie PT. What has MMN revealed about the auditory system in schizophrenia? Int J Psychophysiol 2001;42:177–94. Moreau P, Jolicoeur P, Peretz I. Pitch discrimination without awareness in congenital amusia: evidence from event-related potentials. Brain Cognit 2013;81:337–44. Näätänen R. Mismatch negativity: clinical research and possible applications. Int J Psychophysiol 2003;48:179–88. Näätänen R, Gaillard AW, Mantysalo S. Early selective-attention effect on evoked potential reinterpreted. Acta Psychol 1978;42:313–29. Nitschke JB, Miller GA, Cook EW. Digital filtering in EEG/ERP analysis: some technical and empirical comparisons. Behav Res Methods Instrum Comp 1998;30:54–67. Olejnik S, Algina J. Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol Methods 2003;8:434–47. Onton J, Makeig S. Information-based modeling of event-related brain dynamics. Prog Brain Res 2006;159:99–120. Oostenveld R, Fries P, Maris E, Schoffelen JM. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011;2011:156869. R Core Team. R: a language and environment for statistical computing; 2014, http:// www.r-project.org. Schröger E. The influence of stimulus intensity and inter-stimulus interval on the detection of pitch and loudness changes. Electroencephalogr Clin Neurophysiol 1996;100:517–26. Wang YH, Yeh CH, Young HWV, Hu K, Lo MT. On the computational complexity of the empirical mode decomposition algorithm. Phys A 2014;400:159–67. Williams N, Nasuto SJ, Saddy JD. Evaluation of empirical mode decomposition for event-related potential analysis. EURASIP J Adv Signal Process 2011;1:965237. Wu CH, Lee PL, Shu CH, Yang CY, Lo MT, Chang CY, et al. Empirical mode decomposition-based approach for intertrial analysis of olfactory event-related potential features. Chemosens Percept 2012;5:280–91. Wu Z, Huang NE. Ensemble empirical mode decomposition: a noise-assisted data analysis method. Adv Adapt Data Anal 2009;1:1–41.
Contents lists available at ScienceDirect
Journal of Neuroscience Methods journal homepage: www.elsevier.com/locate/jneumeth
An improved method for measuring mismatch negativity using ensemble empirical mode decomposition Chun-Hsien Hsu a,∗ , Chia-Ying Lee a , Wei-Kuang Liang b a b
Institute of Linguistics, Academia Sinica, No. 128, Section 2, Academia Road, 115 Taipei, Taiwan, ROC Institute of Cognitive Neuroscience, National Central University, Jhongli, Taiwan, ROC
h i g h l i g h t s • We adopt EEMD method and evaluate the statistical significance of ERM measurements. • ERMs can reflect effects of lexical tone changes even with a less number of trials. • EEMD method is helpful to promote clinical and developmental assessments based on EEG.
a r t i c l e
i n f o
Article history: Received 3 September 2015 Received in revised form 14 February 2016 Accepted 16 February 2016 Available online 23 February 2016 Keywords: Ensemble empirical mode decomposition Event-related mode Event-related potential Hilbert–Huang transformation Intrinsic mode function Mismatch negativity
a b s t r a c t Background: Mismatch negativity (MMN) is a component of event-related potentials (ERPs). Conventional approaches to measuring MMN include recording a large number of trials (e.g., 1000 trials per participant) and extracting signals within a low frequency band, e.g., between 2 Hz and 8 Hz. New Method: Ensemble empirical mode decomposition (EEMD) is a method to decompose time series data into intrinsic mode functions (IMFs). Each IMF has a dominant frequency. Similar to ERP measurement, averaging IMFs across trials allows measurement of event-related modes (ERMs). This paper demonstrates a protocol that adopts EEMD and Hilbert spectral analyses and uses ERMs to extract MMN-related activity based on electroencephalography data recorded from 18 participants in an MMN paradigm. The effect of deviants was demonstrated by manipulating changes in lexical tones. Results: The mean amplitudes of ERMs revealed a significant effect of lexical tone on MMN. Based on effect size statistics, a significant effect of lexical tone on MMN could be observed using ERM measurements over fewer trials (about 300 trials per participant) in a small sample size (five to six participants). Comparison with Existing Method(s): The EEMD method provided ERMs with remarkably high signal-tonoise ratios and yielded a strong effect size. Furthermore, the experimental requirements for recording MMN (i.e., the number of trials and the sample size) could be reduced while using the suggested analytic method. Conclusions: ERMs may be useful for applying the MMN paradigm in clinical populations and children. © 2016 Elsevier B.V. All rights reserved.
1. Introduction In event-related potential (ERP) studies of human auditory perception, the mismatch negativity (MMN) response has received substantial attention (Näätänen, 2003; Näätänen et al., 1978). MMN can be elicited by deviant stimuli, which occur among repetitive standard stimuli in oddball paradigms. The contrast between standard and deviant sounds is manipulated by modifying a variety of acoustic features, such as pitch (Moreau et al., 2013), duration
∗ Corresponding author. Tel.: +886 2 2652 5000x6132; fax: +886 2 2785 6622. E-mail address: [email protected] (C.-H. Hsu). http://dx.doi.org/10.1016/j.jneumeth.2016.02.015 0165-0270/© 2016 Elsevier B.V. All rights reserved.
(Chobert et al., 2012), and intensity (Schröger, 1996). MMN typically manifests as a negative ERP that peaks about 150–200 ms after deviant stimulus onset. Across these studies, a robust effect of the size of deviance has been shown in MMN. That is, as the discriminability between the standard and the deviant stimuli increases, MMN amplitudes increase, and the MMN response sometimes occurs earlier. As MMN can be elicited in the absence of attention and with no task requirements, it is particularly suitable for studying clinical populations and children. Researchers have used MMN to explore neurological mechanisms in normal participants and have demonstrated some clinical applicability. For example, MMN has been used as a valid predictor of recovery from coma (Daltrozzo
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
et al., 2007). MMN has also been used as an index for investigating schizophrenia (Michie, 2001), degenerative brain disorders (Jung et al., 2006), and dyslexia (Kujala et al., 2001). The conventional method of estimating MMN involves first estimating ERPs of standards and deviants by averaging several epochs of electroencephalography (EEG) data that correspond to onset of stimuli. Then, differences between waveforms are calculated by subtracting the ERPs of standards from those of deviants. Therefore, participants need to hear standards and deviants hundreds of times. To improve the statistical significance of ERP studies, a simple approach is increasing the number of trials. However, this approach prolongs experimental times for recording EEG data and might not be suitable for studying clinical populations and children. Some analytic methods have been used to improve measurements of MMN. These methods were based on the idea that that MMN activity might have a specific topological distribution and a dominant frequency band. To extract MMN-related activity based on its topological features, one can use independent component analysis (ICA) (Cong et al., 2010). To extract MMN activity by extracting EEG oscillation within a frequency band, there are methods such as optimal digital filtering (Kalyakin et al., 2007), wavelet decomposition (Cong et al., 2011), and empirical mode decomposition (EMD) (Cong et al., 2009). This paper will not exhaustively compare the pros and cons of these different methods. Instead, we aim to provide a practical demonstration to show that an advanced version of EMD, i.e., ensemble empirical mode decomposition (EEMD) (Wu and Huang, 2009), can be used to extract latent ERP activity from single-trial EEG data, and to evaluate the statistical significance of EEG data based on the EEMD method. EEMD and its origin method, EMD, are components of the Hilbert-Huang transformation (HHT), which has been proposed by Huang and colleagues for analyzing nonlinear and non-stationary data (Huang et al., 1998). The procedure of HHT consists of two parts: EMD and Hilbert spectral analysis. EMD is an adaptive method to decompose waveforms into several intrinsic mode functions (IMFs). Although EMD has the ability to extract ERPs, it has a drawback—the mode mixing problem (Huang et al., 2003). That is, a feature of time-frequency activities is not fixed in one IMF, which makes it difficult to determine ERPs and the corresponding IMFs. To overcome the problem of mode mixing, Wu and Huang (2009) have recommended EEMD, a noise-assisted data analysis method. The output of EEMD is a set of IMFs generated from ensemble means of trials by repeating EMD on the same signal with different sets of Gaussian noise. The technical details of EEMD have been described by Al-Subari et al. (2015a, 2015b). Since ERPs are known as low-frequency waveforms (Nitschke et al., 1998), Hilbert spectral analysis, the second step of HHT, can be used to evaluate the frequency of each IMF. Hilbert spectral analysis provides the instantaneous frequency of each IMF. According to Huang et al. (2011), the instantaneous frequency could represent the nonlinear and non-stationary signals without resorting to the mathematical artifact of harmonics. Like measuring ERPs, averaging IMFs across trials provides event-related modes (ERMs). Based on the instantaneous frequency of ERMs, ERP components can be extracted by summing ERMs (Cong et al., 2009; Williams et al., 2011; Wu et al., 2012) or using an ERM within a frequency range (Al-Subari et al., 2015b). For example, Cong et al. (2009) have demonstrated an application of EMD for analyzing MMN. In their study, time-frequency spectra of ERMs with frequencies between 2 and 8.5 Hz were used to estimate MMN waveforms, because Kalyakin et al. (2007) suggested that activities within this frequency range could reflect MMN. For ERPs of sentence comprehension, Williams et al. (2011) used EMD to extract ERMs between 1 and 10 Hz. For olfactory ERPs, Wu et al. (2012) used ERMs between 0.5 and 11.5 Hz. For visual perception,
79
Al-Subari et al. (2015b) demonstrated that ERMs could reflect P100 and N200 elicited in a contour integration task. To date, however, few studies have demonstrated EEMD analyses of EEG data. It is important to explore the extent to which the EEMD could improve the sensitivity of measuring ERP components. In particular, with regard to measuring MMN responses, answering this question would be helpful to promote clinical and developmental assessments based on MMN. In the present study, the dataset of adult EEG in Cheng et al. (2013) was utilized. The effect of lexical tone changes on MMN amplitude in early and late time windows has been addressed in detail in previous studies (Cheng et al., 2013). Specifically, Cheng et al. use Mandarin lexical tones in a multiple-deviant oddball paradigm. The results showed that MMN activity for acoustically dissimilar contrasts (described below) was significant between 100 ms and 200 ms. Therefore, the present study focused on mean amplitudes between 100 ms and 200 ms and evaluated the statistical significance of MMN activity in ERMs. We adopted the method of single-trial based EEMD analysis suggested in Al-Subari et al. (2015b) to extract ERMs as a measurement of MMN. Furthermore, subsets of trials were randomly sampled to explore whether MMN could be measured using ERMs with few trials.
2. Materials and methods 2.1. Experimental design and EEG dataset Since this experiment has been discussed in detail elsewhere (Cheng et al., 2013), only a brief introduction is provided in this article. The participants in the MMN experiment were 18 native speakers of mandarin Chinese (two women, age range = 18–29) with normal hearing. Participants passively listened to streams of auditory stimuli. This study was approved by the Human Subject Research Ethics Committee/Institutional Review Board of Academia Sinica, Taiwan. Written consent forms were obtained from all participants. The stimuli consisted of three Mandarin syllables with different lexical tones: yi1 “clothing” (T1), yi2 “aunt” (T2), and yi3 “chair” (T3), which share the same vowel,/i/, but carry different tonal contours. The T3 was assigned as the standard, with T1 and T2 as deviants. The T1/T3 pair represents the larger deviant contrast, and the T2/T3 pair represents the smaller deviant contrast. Each stimulus lasted 250 ms, and the inter-stimulus interval was 500 ms. The experimental session started with 20 trials of the standard, followed by 1000 trials with 80% standards and 20% deviants (10% for each deviant). During the experiment, participants were instructed to play a puzzle computer game called “super-box” silently. EEG signals were amplified by SYNAMPS2® (Neuroscan, Inc.) in DC mode, low-pass 100 Hz, and digitized at a sampling rate of 500 Hz. EEG data were recorded from 64 Ag/AgCl electrodes (QuickCap, Neuromedical Supplies, Sterling, USA), arranged according to the international 10-20 system, including a reference electrode located between CZ and CPZ and a ground electrode located between FPZ and FZ. Six additional electrodes were attached over the left and right mastoids, supra- and infra-orbital ridges of the left eye (VEOG), and outer canthi of both eyes (HEOG). Before estimating ERPs and ERMs, the EEG data were rereferenced to the average of the left and right mastoids. Continuous EEG was segmented into epochs from 100 ms prior to the onset of the stimulus to 700 ms after the onset. The first 20 trials and epochs with artifacts exceeding ±100 V were rejected. To ensure that the number of trials between the standard and deviant were comparable; only the standard trials that were preceded by at least three successive standards were included in analyses. For each participant, at least 45 accepted deviants were required to
80
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
Fig. 1. Left column: Original ERPs and ERMs of a representative participant at channel FZ; right column: Hilbert spectra of ERMs.
Fig. 2. The dominant frequency of each ERM in each channel of two representative participants.
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
be included in further analyses. Based on the selection criteria described above, the mean number of accepted deviants was 76.6, and that of accepted standards was 173.5. The following processing steps were performed using MATLAB scripts based on the Fieldtrip toolbox (Oostenveld et al., 2011) and the fast EEMD algorithms (Wang et al., 2014). 2.2. Conventional ERP pre-processing After applying a band-pass filter (1–30 Hz) and removing the activity in the pre-stimulus interval (i.e., the baseline correction), the grand averaged ERPs for standard, small deviant, and large deviant were computed for each participant and each electrode. 2.3. Measuring MMN-related ERMs Without applying baseline corrections or band-pass filtering, the same set of EEG data described in Section 2.1 was used in the following EEMD analysis. To avoid potential distortion at the boundary of the EEG segments while applying the EEMD procedure, 100 samples before and after EEG segments were included in EEMD decomposition. Therefore, the time range of EEG segments for EEMD analysis was from 200 ms before the stimuli onset to 800 ms after the onset. The analytic flow of single-trial based EEMD analysis has been described in previous studies (AlSubari et al., 2015a, 2015b). Some essential steps are described below: Step 1: For each signal of a channel in a trial, the EEMD analysis was performed with 10 times of sifting and 40 ensembles. For each time of each ensemble, the amplitude of the Gaussian noise was 10 percent of the EEG segment’s standard deviation. Each EEG segment was decomposed into eight IMFs. The last mode was the residual trend, and only IMFs 1 to 7 were considered when estimating ERMs (Al-Subari et al., 2015b). Subsequent analyses were based on IMFs between −100 ms and 700 ms. Step 2: Based on the method described in Huang et al. (2009), time-frequency spectra of IMFs were calculated for each trial and each channel. Step 3: ERMs were estimated by averaging IMFs over trials and were estimated for each IMF, each stimulus condition, and each channel (the left column of Fig. 1). Step 4: For each power spectrum estimated in Step 2, activities were summed across time. Fig. 2 shows the dominant frequency at the maximum of marginal spectrum of each ERM in each channel of two representative participants. The horizontal axis corresponds to the log10 transformed frequency. The results demonstrated that the dominant frequencies of ERMs were consistent across channels. Therefore, we followed the approach suggested by Al-Subari et al. (2015b) and averaged spectra over channels. This step revealed the event-related Hilbert spectrum corresponding to each ERM (the right column of Fig. 1). The output of this step indicated the dominant frequency of each ERM. The abovementioned steps used one participant’s EEG data to perform EEMD analysis (Step 1), Hilbert spectral analysis (Step 2), event-related averaging of IMFs (Step 3, which in turn provides ERMs) and event-related averaging of Hilbert spectra (Step 4). The left column of Fig. 1 shows the original ERP waveform and the ERMs in channel FZ of one participant, and the right column of Fig. 1 shows Hilbert spectra for the same participant. The results showed that the dominant frequencies (defined by the peak) in Hilbert spectra were around 66.3 Hz, 37.7 Hz, 18.6 Hz, 8 Hz, 3.5 Hz, 2 Hz, and 1 Hz from ERM 1 to ERM 7. After analyzing every participant’s data, the means and standard deviations of the dominant frequency are reported in Table 1. To extract MMN-related activities, this study focused on ERMs with frequencies mainly distributed between 2 and 8 Hz. For each participant, the frequency of ERM 4 and 5 were
81
Table 1 Means and standard deviations (in parentheses) of the dominant frequency of each ERM. Instantaneous frequency (Hz) ERM1 ERM2 ERM3 ERM4 ERM5 ERM6 ERM7
79.51 (11.75) 45.76 (5.33) 19.51 (1.48) 8.24 (0.52) 3.63 (0.22) 2.01 (0.00) 1.34 (0.24)
was between 2 and 8 Hz, so ERM 4 and 5 were categorized as MMNrelated ERMs. Therefore, to build up MMN-related waveforms, each stimulus type’s ERM 4 and ERM 5 were summed together for each participant. 2.4. Statistical analyses using ERPs/ERMs data For statistical analyses, there were two types of dependent variables, including the mean amplitude between 100 and 200 ms of ERPs and that of ERMs. Dependent variables were subjected to be repeated measures ANOVA analyses with conditions (standard, large deviant, and small deviant) and electrode sites (F3, FZ, F4, FC3, FCZ, FC4, C3, CZ, and C4) as within-subject factors. 2.5. Estimating MMN responses using subsets of EEG trials Single-trial EEGs were randomly sampled by taking 90%, 80% 70%, 60%, 50%, 40%, 30%, 20%, and 10% of the trials for each participant. Then, sampled trials were used to estimate ERPs and ERMs. The ANOVA analysis was also applied to mean amplitudes measured in sampled ERPs and ERMs. For each data type (ERPs and ERMs) and each percentage level of trials (from 90% to 10%), the sampling procedures and subsequent ANOVA analyses were repeated 500 times. The number of trials in each percentage level is described in Table 2. For each ANOVA analysis described in Section 2.4 and 2.5, the effect size of the condition was estimated by calculating the generalized eta-squared statistic (Bakeman, 2005; Olejnik and Algina, 2003). Each ANOVA analysis was done by using the ezANOVA program of the ez-package (Lawrence, 2013). The generalized eta-squared statistic of condition was estimated using the method described in Bakeman (2005), which was supported by the ez-package. This package was supplied in the R system for statistical computing (version 3.1.0, R Development Core Team, 2014). 2.6. Estimating inter-trial variability of mean amplitudes To evaluate the performance of EEMD analysis, we use all trials to estimate inter-trial variability of the mean amplitudes of ERPs and ERMs in the channel FZ. The mean amplitudes between 100 ms and 200 ms were averaged for each trial. Then, standard deviations were calculated for each participant and each data type. 3. Results Fig. 3a and b shows the grand-averaged ERPs and ERMs, respectively, in channel FZ. Both ERPs and ERMs reflected the typical pattern of MMN responses. That is, large deviant stimuli elicited more negative activity than standard stimuli between 100 and 200 ms after stimuli onset. On the other hand, small deviant stimuli elicited a delayed MMN response at around 300 ms. When considering all artifact-free trials, the amplitudes of ERPs revealed a significant effect of condition (F(2, 34) = 57.1, p = 0.000000000013,
82
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
Table 2 Ranges of trial numbers in each percentage level, and counts of ANOVA analyses that accepted the null hypothesis of condition effects. Number of trials
Counts of accepting the null hypothesis of condition effects
Percentage of trials
Standard (max–min)
Deviant 1 (max–min)
Deviant 2 (max–min)
Mean amplitudes of ERPs
Mean amplitudes of ERMs
90 80 70 60 50 40 30 20 10
207–98 184–87 161–76 138–65 115–55 92–44 69–33 46–22 23–11
90–39 80–34 70–30 60–26 50–22 40–17 30–13 20–9 10–4
90–45 80–40 70–35 60–30 50–25 40–20 30–15 20–10 10–5
0 0 0 0 0 0 0 2 41
0 0 0 0 0 0 0 0 4
Fig. 3. (a) Averaged ERPs of each stimulus and (b) averaged ERMs of each stimulus.
Fig. 4. Means of the mean amplitudes based on ERPs (a) and ERMs (b) by averaging activities from 100 ms to 200 ms after stimulus onset. Error bars indicate 95% confidence interval.
2 = 0.41). Fig. 4a shows the means and 95% confidence intervals of the mean amplitude of each stimulus condition based on ERPs. Consistent with the findings of Cheng et al. (2013), large deviant stimuli elicited significantly more negative activity than standard stimuli, while the difference between small deviant stimuli and standard stimuli was not significant between 100 ms and 200 ms. With regard to ERM amplitudes (Fig. 4b), there was a significant
effect of condition as well (F(2, 34) = 57.1, p = 0.0000000000003, 2 = 0.58). It is worth noting that ERMs yielded an effect size of condition of 0.58, which was higher than 0.41, the corresponding effect size drawn from ERPs. In summary, ANOVA analysis indicated that using mean amplitudes of ERMs with dominant frequencies between 2 Hz and 10 Hz could elicit the typical result of lexical tone MMN.
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
83
Table 3 Trial-wise variability of mean amplitudes between 100 ms and 200 ms. Participants
S001 S002 S003 S004 S005 S006 S007 S008 S009 S010 S011 S012 S013 S014 S015 S016 S017 S018
Fig. 5. Means of the generalized eta-square of the condition effects on the amplitudes of ERMs and ERPs.
Table 2 shows the counts of ANOVA analyses that failed to reject the null hypothesis of condition (˛ = 0.05). The results indicated that ERPs yielded significant effects of condition in ANOVA analyses when using 90–30% of data. ERMs yielded significant effects of conditions in ANOVA analyses when using 90–20% of data. To investigate the effect size, the generalized eta-square of the condition effect was computed for each ANOVA analysis. In Fig. 5, the dots indicate the mean of the effect size, and the black line indicates the level at 0.41, which was the effect size of condition while using amplitudes of ERPs with 100% of data. The results showed that, in every percentage level, ERMs yielded higher mean effect sizes than ERPs. As expected, means of effect size decreased with a decreasing number of trials. The descending pattern of effect size indicated that amplitudes of ERMs with 30% of data yielded a mean effect size of 0.428, which remained higher than 0.41. When using ERMs with 20% and 10% of data, the means of the effect size were 0.36 and 0.241, respectively. By sampling parts of trials, the analysis of effect size implied that ERMs with one third of the trials could yield a significant effect of condition, and this effect seemed to be at the same level of significance based on ERPs using 100% of data. To demonstrate this finding, three sets of difference waves were estimated as follows. Fig. 6 shows the grand-average of difference waves of large deviant stimuli (blue line) and small deviant stimuli (red line) superimposed on areas corresponding to the 95% confidence intervals. Fig. 6a shows difference waves of ERPs based on 100% of trials, and Fig. 6b shows difference waves of ERPs using the first third of trials. Comparing Fig. 6a and b, the typical influence of trial number on ERPs was observed. That is, decreasing numbers of trials would introduce noise in the ERP waveforms. Therefore, Fig. 6b shows less smooth waves and wider areas of confidence intervals than Fig. 6a. Finally, Fig. 6c shows difference waves based on ERMs using data from the first third of trials. Through visual inspection, ERMs yielded smaller confidence intervals than ERPs. Table 3 shows the trial-wise variability of mean amplitudes of each participant. For all participants, ERMs yielded smaller standard deviations than ERPs (p < 0.001, Wilcoxon signed rank test). This finding suggested that mean amplitudes of ERMs had low inter-trial variability.
Standard deviation ERP
ERM
9.36 10.69 8.65 9.84 9.95 7.92 6.13 9.01 10.32 8.69 10.6 11.93 9.79 8.89 10 10.24 9.63 8.52
4.57 5.46 3.41 4.19 4.19 4.13 3.08 4.25 4.46 5.22 5.47 6.7 4.73 4.06 5.23 4.7 5.42 4.53
4. Discussion This study demonstrated that measuring amplitudes of ERMs yielded higher effect size statistics of MMN than measuring ERPs. By evaluating effects of lexical tone MMN, the mean amplitude of ERMs revealed a better sensitivity to lexical tone changes than that of ERPs. This result implies that EEMD methods could remarkably improve the signal-to-noise ratios (SNR) and could substantially improve measurement of EEG data. By observing the effect size, it seems that effects of lexical tone contrasts on MMN could be observed with one third of the trials used in this study (about 300 trials) which is fewer than the requirement in conventional MMN paradigms (more than 1000 trials in total). While ERMs could reflect MMN with 30% of trials, the corresponding mean of the effect size (0.428) was still large. This implies that future studies could replicate the effect of condition with a sample size of less than 18 participants. Kirk (1995) introduced a procedure for estimating the sample size based on effect size statistics (see pp. 186 of Kirk, 1995). Given that the generalized eta-square measures the strength of association between levels of condition and mean amplitudes of ERPs/ERMs, one can use the design of the experiment to set parameters as follows. By using Kirk’s approach, the parameter v1 was 2 (the number of levels of conditions minus 1), and the parameter v2 was 2(n − 1) because condition was a within-subject variable. Accordingly, Kirk’s procedure suggested that, given these parameters, a factor with an effect size statistic between 0.4 and 0.5 would require at least five or six participants to reveal a significant effect (while setting ˛ as 0.05 and 1 − ˇ as 0.8). Therefore, when conducting future studies to measure Mandarin lexical tone MMN over the frontal scalp, using ERMs could allow us to decrease recording time and to use a smaller sample size. In addition, the correspondence between ERMs and EEG oscillations in the present study were consistent with the results in Al-Subari et al. (2015b). For example, in Fig. 1, ERM 1 shows a dominant frequency range above 40 Hz, which corresponds to the high gamma band. ERM 2 showed oscillations mainly distributed from 20 Hz to 60 Hz, which reflected both beta and low gamma bands. ERM 3 showed oscillation around 18 Hz, corresponding to the low beta band. ERM 4 showed a dominant frequency at around 8 Hz, which indicated an alpha oscillation. ERM 5 showed oscillation around 3 Hz, corresponding to the theta band. Finally, ERM 6 and 7 showed dominant frequency ranges below 3 Hz, which indicated the delta oscillation. The correspondences between EEG
84
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85
Fig. 6. The grand-average of difference waves of large deviant stimuli (blue line) and small deviant stimuli (red line) superimposed on areas corresponding to the 95% confidence intervals. (a) ERPs using 100% of trials, (b) ERPs using the first third of trials, and (c) ERMs using the first third of trials. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.).
oscillations and the Hilbert spectra of ERMs might imply that HHT could reflect meaningful aspects of EEG oscillations. Currently, only Cong et al. (2009) have explored the time-frequency spectrum of EEG data with HHT. Future EEG studies could use ERMs and the corresponding time-frequency spectra to identify brain mechanisms of perceptual and cognitive functions. In addition to increasing the effect size, measuring ERMs has other advantages. First, the computation of EEMD is performed on each trial on each EEG channel. Since the number of channels is not a concern, the EEMD method permits high flexibility for users. On the other hand, although ICA and spatial principle component analysis are powerful methods for extracting ERPs, these methods require EEG data with multiple channels (Onton and Makeig, 2006). Secondly, according to Cong et al. (2009) and Williams et al. (2011), ERMs provide better estimation of amplitude and latency than ERPs that are extracted by wavelet analyses and band-pass filtering. This is because wavelet analyses and Fourier based methods use a prior function to estimate brain activities within the desired frequency range (e.g., the sinusoid function and the Morlet function), and a fixed prior function cannot represent signals generated from nonlinear and non-stationary systems. While exploring ERMs with EEG data, one should be aware that it is important to keep EEG data in the original form. That is, any off-line filtering of EEG data should be avoided while applying the EEMD method. For future EEG studies that are intended to measure ERMs, researchers are encouraged to set a high boundary of the low pass filter (e.g., 200 Hz), which would provide a better resolution in the high gamma band. Finally, the high pass filter might be turned off during on-line recording. In conclusion, this paper indicates a practical method of measuring MMN with the EEMD method. The results suggested that MMN could be measured using ERMs with fewer trials. Although HHT has been recently adopted in analyzing EEG data, this approach has not attracted much attention. The method presented here could be easily applied to analyze other ERP components. To utilize the EEMD
method, Al-Subari et al. (2015a) have implemented the EMDLAB toolbox as a plug-in of the EEGLAB toolbox. In addition, the MATLAB scripts developed within this study were integrated into the Fieldtrip toolbox and will be available at http://www.fieldtriptoolbox. org/.
Acknowledgments This work was supported by research grants from Academia Sinica, Taiwan (AS-99-TP-AC1 and AS-102-TP-C06). The authors would like to thank two anonymous reviewers for their detailed comments on this paper. We are grateful to Norden E. Huang, JiaRong Yeh, Chu-Lan Kao, and Chi-Hung Juan for discussions on using the HHT algorithms.
References Al-Subari K, Al-Baddai S, Tome AM, Goldhacker M, Faltermeier R, Lang EW. EMDLAB: a toolbox for analysis of single-trial EEG dynamics using empirical mode decomposition. J Neurosci Methods 2015a;253:193–205. Al-Subari K, Al-Baddai S, Tome AM, Volberg G, Hammwohner R, Lang EW. Ensemble empirical mode decomposition analysis of EEG data collected during a contour integration task. PLoS One 2015b;10:e0119489. Bakeman R. Recommended effect size statistics for repeated measures designs. Behav Res Methods 2005;37:379–84. Cheng Y-Y, Wu H-C, Tzeng Y-L, Yang M-T, Zhao L-L, Lee C-Y. The development of mismatch responses to mandarin lexical tones in early infancy. Dev Neuropsychol 2013;38:281–300. Chobert J, Francois C, Habib M, Besson M. Deficit in the preattentive processing of syllabic duration and VOT in children with dyslexia. Neuropsychologia 2012;50:2044–55. Cong F, Kalyakin I, Huttunen-Scott T, Li H, Lyytinen H, Ristaniemi T. Single-trial based independent component analysis on mismatch negativity in children. Int J Neural Syst 2010;20:279–92. Cong F, Kalyakin I, Li H, Huttunen-Scott T, Huang Y, Lyytinen H, et al. Answering six questions in extracting children’s mismatch negativity through combining wavelet decomposition and independent component analysis. Cogn Neurodyn 2011;5:343–59.
C.-H. Hsu et al. / Journal of Neuroscience Methods 264 (2016) 78–85 Cong F, Sipola T, Huttunen-Scott T, Xu X, Ristaniemi T, Lyytinen H. Hilbert-Huang versus Morlet wavelet transformation on mismatch negativity of children in uninterrupted sound paradigm. Nonlinear Biomed Phys 2009;3:1. Daltrozzo J, Wioland N, Mutschler V, Kotchoubey B. Predicting coma and other low responsive patients outcome using event-related brain potentials: a metaanalysis. Clin Neurophysiol 2007;118:606–14. Huang NE, Chen X, Lo MT, Wu Z. On Hilbert spectral representation: a true timefrequency representation for nonlinear and nonstationary data. Adv Adapt Data Anal 2011;3:63–93. Huang NE, Shen Z, Long SR, Wu MC, Shih HH, Zheng Q, et al. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc R Soc Lond A Math Phys Sci 1998;454:903–95. Huang NE, Wu MLC, Long SR, Shen SS, Qu W, Gloersen P, et al. A confidence limit for the empirical mode decomposition and Hilbert spectral analysis. Proc R Soc Lond A Math Phys Sci 2003;459:2317–45. Huang NE, Wu Z, Long SR, Arnold KC, Chen X, Blank K. On instantaneous frequency. Adv Adapt Data Anal 2009;1:177–229. Jung J, Morlet D, Mercier B, Confavreux C, Fischer C. Mismatch negativity (MMN) in multiple sclerosis: an event-related potentials study in 46 patients. Clin Neurophysiol 2006;117:85–93. Kalyakin I, Gonzalez N, Joutsensalo J, Huttunen T, Kaartinen J, Lyytinen H. Optimal digital filtering versus difference waves on the mismatch negativity in an uninterrupted sound paradigm. Develop Neuropsychol 2007;31: 429–52. Kirk RE. Experimental design: procedures for the behavioral sciences. 3rd ed. Pacific Grove, California: Brooks/Cole; 1995. Kujala T, Karma K, Ceponiene R, Belitz S, Turkkila P, Tervaniemi M, et al. Plastic neural changes and reading improvement caused by audiovisual training in readingimpaired children. Proc Natl Acad Sci USA 2001;98:10509–14. Lawrence MA. ez: easy analysis and visualization of factorial experiments; 2013, http://cran.r-project.org/package=ez.
85
Michie PT. What has MMN revealed about the auditory system in schizophrenia? Int J Psychophysiol 2001;42:177–94. Moreau P, Jolicoeur P, Peretz I. Pitch discrimination without awareness in congenital amusia: evidence from event-related potentials. Brain Cognit 2013;81:337–44. Näätänen R. Mismatch negativity: clinical research and possible applications. Int J Psychophysiol 2003;48:179–88. Näätänen R, Gaillard AW, Mantysalo S. Early selective-attention effect on evoked potential reinterpreted. Acta Psychol 1978;42:313–29. Nitschke JB, Miller GA, Cook EW. Digital filtering in EEG/ERP analysis: some technical and empirical comparisons. Behav Res Methods Instrum Comp 1998;30:54–67. Olejnik S, Algina J. Generalized eta and omega squared statistics: measures of effect size for some common research designs. Psychol Methods 2003;8:434–47. Onton J, Makeig S. Information-based modeling of event-related brain dynamics. Prog Brain Res 2006;159:99–120. Oostenveld R, Fries P, Maris E, Schoffelen JM. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011;2011:156869. R Core Team. R: a language and environment for statistical computing; 2014, http:// www.r-project.org. Schröger E. The influence of stimulus intensity and inter-stimulus interval on the detection of pitch and loudness changes. Electroencephalogr Clin Neurophysiol 1996;100:517–26. Wang YH, Yeh CH, Young HWV, Hu K, Lo MT. On the computational complexity of the empirical mode decomposition algorithm. Phys A 2014;400:159–67. Williams N, Nasuto SJ, Saddy JD. Evaluation of empirical mode decomposition for event-related potential analysis. EURASIP J Adv Signal Process 2011;1:965237. Wu CH, Lee PL, Shu CH, Yang CY, Lo MT, Chang CY, et al. Empirical mode decomposition-based approach for intertrial analysis of olfactory event-related potential features. Chemosens Percept 2012;5:280–91. Wu Z, Huang NE. Ensemble empirical mode decomposition: a noise-assisted data analysis method. Adv Adapt Data Anal 2009;1:1–41.
