+ Human Brain Mapping :iii-vi(1994) +
EDITORIAL
Human Brain Mapping in Both Time and Space* "What?", "Where?', "When?", "How?" These four terse questions cover much of the terrain of modern cognitive neuroscience. "What?" refers to an abstract characterization of a particular sensory, motor, or cognitive problem to be solved by an organism. In the case of vision, for example, an important aspect of "What?" is the problem of segmenting the distribution of light energy striking the retina into contours and objects that guide our locomotion, enable us to read text, or to appreciate art. "What?" corresponds roughly to the level of Computational Theory in Marr's three-level analysis of an information-processing device (the others being the level of RepresentationlAlgorithm and the level of Hardware Implementation). While such an abstract characterization might seem straightforward or even trivial at first glance, one need only consider such cases as memory to realize that our intuitive, nonscientific answers to the "What?" question may be seriously misleading. The extensive body of evidence developed over the last few decades suggests that "declarative" and "procedural" memory represent fundamentally different processes, thus invalidating our intuitive picture of memory as a unitary process that mediates everything we acquire and retain. "Where?" refers to the identification of the brain structures and systems that mediate the sensory, motor, or cognitive process under consideration. As Peter T. Fox's inaugural editorial in Human Brain Mapping illustrated, this is the question to which much of the "convergence of disciplines" that constitute human brain mapping is initially addressed. The "Where?" question has been investigated in humans using an increasingly wide array of techniques and approaches. Correlations of functional deficits with the anatomical loci of brain damage have been made using either post-mortem anatomy or in vivo imaging by computed tomography (CT) or magnetic resonance imaging (MRI).Functional imaging studies using positron emission tomography (PET) and functional MRI (fMRI) have been combined with systematic stimulus, response, and task manipulations in order to identify brain regions associated with particular sensory, mo0 1994 Wiley-Liss, Inc. *This article is a US Government work and, as such, is in the public domain in the United States of America.
tor, and cognitive processes. Similarly, studies of electrical (electroencephalography and event-related potentials, EEG and ERPs) and magnetic (magnetoencephalography and event-related fields, MEG and ERFs) activity of the brain have combined source modeling techniques with systematic task manipulations in an attempt to answer the "Where?" question. Other techniques such as noninvasive optical imaging are under active technical development and should soon join the list. As will be discussed below, different brain mapping techniques have different advantages and limitations in the spatial domain and therefore different strengths and weaknesses with respect to the "Where?" question. "When?" refers to temporal dynamics: when are the brain structures and systems involved in a particular process activated and what are their temporal relationships? To date, most human brain mapping studies have not addressed "When?" questions, largely due to methodological limitations. Although it is possible to ask profitable questions about temporal dynamics based on composite behavioral data (such as reaction time) alone, addressing the ''When?J question with respect to specific brain systems and structures provides a much richer approach, more rooted in neurobiology. Investigating temporal dynamics may also provide a more accurate answer to "Where?" questions because closely adjacent regions of activation may be distinguishable by their timecourse when not distinguishable by location information alone. "How?" refers to the specific neuronal events and operations performed by the brain systems and structures that are required to mediate the process in question. In the example of visual segmentation mentioned above, approaches to this question might include investigation of receptive field properties in various visual areas and the mechanisms that give rise to them. Although "Where?" and "When?' may be initially addressed on the scale of gross anatomical structures suitable for currently available brain mapping techniques, investigating "How?" generally requires techniques capable of resolving finer details of functional organization. At increasing finer levels of
+ Editorial +
detail, examples include cortical columns, the recep- namic and electromagnetic approaches to human tive field properties of individual neurons, and the brain mapping. It is first useful to distinguish between the forward connectivity and synaptic patterns that give rise to them. Ultimately, each of the questions can be asked at problem of electromagnetism, which is well-defined, every scale, although the value of the answers may and the inverse problem, which is not. The forward problem, calculating the electrical potential and/or vary. One important distinction among alternative brain magnetic fields outside the head given a particular mapping techniques (although by no means the only distribution of currents inside, requires two types of one), concerns the ”Where?” and “When?” questions; models: (a) a source model that embodies the propernamely, their sensitivity, resolution, and other charac- ties of the current sources involved; and (b) a volume teristics in the spatial and temporal domains. PET and model (or head model) that embodies the electromagfMRI are capable of making measurements simulta- netic properties of the brain, skull, and other tissues as neously at many locations throughout the brain vol- electrically conductive media. The accuracy of forume at spatial resolutions on the order of a few ward calculations of course depends upon the realism millimeters to a few centimeters. (The absolute spatial of the source and head models, but there is no resolution of these and other techniques to be dis- inherent indeterminacy in ‘calculating surface data cussed is a complex issue that depends on instrumen- given suitable source and head models. Source models tal resolution, the spatial resolution of the signals to be have ranged in complexity from single point-current measured, spatial characteristics of the noise, and the dipoles through dipole sheets and layers, to anatomispecific definition of spatial resolution being em- cally realistic current distributions. Similarly, head ployed). Because PET and fMRI are based on meta- models have ranged from single homogeneous bolic and hemodynamic responses to brain activation: spheres, through multi-compartment spherical shells, (a) they are indirect measures of neuronal activity to models having much greater geometrical, anatomimediated through the intervening relationship be- cal, and electrical realism. The electromagnetic inverse problem is the reverse of tween neuronal activity and the metabolic/hemodynamic response, which remains incompletely under- the forward problem, namely, calculating the current stood; and (b) their temporal resolution is ultimately distributions inside the brain given the electrical and/or limited by the timecourse of the metabolic/hemody- magnetic fields over the surface of the head. Solutions to the inverse problem require the same types of namic response (tenths of seconds to seconds). EEG and MEG, in contrast, are direct physical source and head models required for the forward consequences of neuronal currents and are capable of problem. In addition, the inverse problem requires resolving temporal patterns of neural activity in the techniques for choosing a ”best-fitting” inverse solumillisecond range. But they are limited by an inability tion for the data given the models under considerto estimate currents simultaneously throughout the ation. Two general approaches have been employed brain volume because the problem of estimating cur- which, for convenience, I will term dipole modeling rent sources in the brain from surface EEG and MEG and current imaging, respectively. Both approaches measurement is mathematically ill-posed; that is, it has can be used to model electromagnetic data at a single no unique solution in the most general, unconstrained instant or over an extended time epoch. Dipole modeling combines a small number of current case. Given these compensating strengths and limitations of metaboWhemodynamic and electromagnetic dipole sources (as few as one or as many as 6-10) with measures, it is not surprising that a number of investi- a particular head model and uses nonlinear leastgators have suggested that PET or fMRI could be squares minimization techniques to find the bestprofitably combined with electromagnetic measures fitting parameters for the model under consideration. such as EEG or MEG. But there is a deeper and more Each dipole source requires six parameters, three subtle reason why metabolic/hemodynamic and elec- corresponding to dipole locations and three vector tromagnetic measures could be mutually beneficial in components representing dipole moments (except for answering the “What?” and “When?” questions, one the special case of magnetic measurements using a that is rooted in the mathematics of the electromag- spherical head model in which only two dipole monetic inverse problem. In what follows, I will briefly ments are required). Thus, a 1-dipole model requires summarize the key components of the inverse prob- fitting 6 parameters to the 50-200 measurements lem and two alternative approaches to it, and then typically required for EEG/MEG source localization suggest how these approaches provide a strong math- studies, while a 6-dipole model requires fitting 36 ematical basis for integrating metabolic/hemody- parameters. Nonlinear minimization techniques are
Editorial
typically used for the dipole modeling approach because surface electrical and magnetic fields are nonlinear functions of the location parameters. In effect, such nonlinear minimization techniques calculate the forward problem over and over again with slight differences in parameter values in order to identify the combination of parameters that minimizes the error between the empirical data and the values calculated by the model. The resulting best-fitting solution for the dipole modeling approach is always relative to the assumptions of the model and careful investigators vary key properties of the model (e.g., the number of current sources) to determine that the solutions are as robust as possible. Current imaging also requires both a source model and a head model, and uses exactly the same mathematical formulation of the forward problem for a given head model as dipole modeling. But instead of assuming a small number of dipoles or other current sources, current imaging assumes a surface or volume of possible current elements (termed the reconstruction surface or volume) whose values are to be estimated. In other words, a small number of dipoles with unknown locations are replaced by a larger number of dipoles with known locations. This seemingly small change in assumptions has two important consequences: 1. Although the forward equations are nonlinear in the three location parameters for each source as noted above, they are linear in the three moment parameters for each source. The surface electrical and magnetic fields for any combination of sources represent the superposition of the fields from all active sources. Therefore, one needs to calculate the complete nonlinear forward problem for each assumed current element in the volume only once, and then the entire problem becomes a much simpler linear minimization problem in which only the moments for each assumed current element are estimated. (The approach of separating linear from nonlinear components of the inverse problem is so much more efficient that some investigators implement the conventional nonlinear minimization for dipole modeling by pre-computing the forward solutions for a sufficiently fine grid of source locations throughout the volume and then using exhaustive search to find the best solution for a particular dataset.) 2. The second major consequence of the strategy of pre-specifying the locations of possible currents is that the number of locations needed to represent adequately the surface or volume of tissue of interest is typically so large that it results in an underdetermined linear problem. That is, instead of the 1-10 dipole
sources typically used in the dipole fitting approach, the current imaging approach may utilize hundreds or thousands of source locations. This in turn yields systems of linear equations in which the number of unknowns greatly exceeds the number of measurements. To combat the problem of underdetermination while retaining the computational advantages of the current imaging approach, a number of investigators have explored ways to minimize the number of source locations required in the current imaging approach. One important approach under investigation by a number of groups is the use of anatomically constrained reconstruction volumes derived from individual subjects’ MRIs. In this approach, only regions of the brain that are possible significant contributors to surface EEG and/or MEG data are included in the reconstruction space, resulting in a much smaller degree of underdetermination. Because of its proximity to electromagnetic sensors and its anatomical structure suitable for generating electromagnetic fields recordable at a distance, neocortex has figured prominently in most explorations of anatomically constrained current estimation techniques. Such constraints are typically implemented by creating a segmented geometric model of the cortical mantle derived from anatomical MRI, populating the cortical mantle with current sources at a selected spatial scale, and calculating the forward problem once for each of the three orthogonal current orientations at each location. The results of such forward calculations are then stored as a “basis matrix” to be used in subsequent linear minimizations. Although the rationale for including or excluding particular anatomical structures in such constrained solutions is beyond the scope of this editorial, it is sufficient to note that both location and orientation constraints derived from realistic cortical geometry have been explored with promising results by multiple laboratories. It should now be obvious to many readers that the same mathematical framework used to implement anatomical constraints from MRI provides a means of integrating metabolic/hemodynamic and electromagnetic measures. In addition to constraining the reconstruction space of current imaging models using location and orientation information from anatomical MRI, it is possible to constrain those locations using patterns of activation derived from PET and/or fMRI data. Once the locations of activation have been correctly determined by whatever means (i.e., either from the electromagnetic data alone or by the addition of anatomical and/or functional constraints from other imaging modalities), then the timecourses of activa-
* v *
+ Editorial
tion can be computed using simple linear techniques, yielding information about the temporal dynamics of activated regions at millisecond resolution. A similar strategy can be used with the dipole modeling approach. In that case, regions of activation derived from PET and/or fMRI data could be used to estimate both the number and locations of dipole sources for use with nonlinear minimization techniques. In summary, while it is essential to continue to compare and integrate brain mapping techniques (e.g., including PET, MRI, MEG and EEG), there is, of course, much left to be done to improve our understanding of each technique alone and its relationship to the others. In particular, more work is needed: (a) to elucidate the detailed relationship between neuronal currents that generate EEG and MEG and the metabolic/hemodynamic responses evident in PET and fh4RI; and (b) to compare systematically the locations and extents of activation evident with different techniques. I believe the future is particularly bright for using the mathematical aspects of the electromagnetic
inverse problem discussed above to exploit the comparative strengths and offset the limitations of metabolic/hemodynamic and electromagnetic techniques. We should be gratified that a sound conceptual and mathematical base is in place for the integration of these approaches. Both "When?" and "Where?" questions in human brain mapping may soon become approachable to a much greater degree than we may have imagined. ACKNOWLEDGMENTS
Supported by the U.S. Department of Energy, National Institutes of Health, and Los Alamos National Laboratory. I am grateful to C.J. Aine, J.S. George, and H.A. Schlitt for helpful comments.
+ vi +
C.C. Wood Los Alamos National Laboratory, Los Alamos, New Mexico
EDITORIAL
Human Brain Mapping in Both Time and Space* "What?", "Where?', "When?", "How?" These four terse questions cover much of the terrain of modern cognitive neuroscience. "What?" refers to an abstract characterization of a particular sensory, motor, or cognitive problem to be solved by an organism. In the case of vision, for example, an important aspect of "What?" is the problem of segmenting the distribution of light energy striking the retina into contours and objects that guide our locomotion, enable us to read text, or to appreciate art. "What?" corresponds roughly to the level of Computational Theory in Marr's three-level analysis of an information-processing device (the others being the level of RepresentationlAlgorithm and the level of Hardware Implementation). While such an abstract characterization might seem straightforward or even trivial at first glance, one need only consider such cases as memory to realize that our intuitive, nonscientific answers to the "What?" question may be seriously misleading. The extensive body of evidence developed over the last few decades suggests that "declarative" and "procedural" memory represent fundamentally different processes, thus invalidating our intuitive picture of memory as a unitary process that mediates everything we acquire and retain. "Where?" refers to the identification of the brain structures and systems that mediate the sensory, motor, or cognitive process under consideration. As Peter T. Fox's inaugural editorial in Human Brain Mapping illustrated, this is the question to which much of the "convergence of disciplines" that constitute human brain mapping is initially addressed. The "Where?" question has been investigated in humans using an increasingly wide array of techniques and approaches. Correlations of functional deficits with the anatomical loci of brain damage have been made using either post-mortem anatomy or in vivo imaging by computed tomography (CT) or magnetic resonance imaging (MRI).Functional imaging studies using positron emission tomography (PET) and functional MRI (fMRI) have been combined with systematic stimulus, response, and task manipulations in order to identify brain regions associated with particular sensory, mo0 1994 Wiley-Liss, Inc. *This article is a US Government work and, as such, is in the public domain in the United States of America.
tor, and cognitive processes. Similarly, studies of electrical (electroencephalography and event-related potentials, EEG and ERPs) and magnetic (magnetoencephalography and event-related fields, MEG and ERFs) activity of the brain have combined source modeling techniques with systematic task manipulations in an attempt to answer the "Where?" question. Other techniques such as noninvasive optical imaging are under active technical development and should soon join the list. As will be discussed below, different brain mapping techniques have different advantages and limitations in the spatial domain and therefore different strengths and weaknesses with respect to the "Where?" question. "When?" refers to temporal dynamics: when are the brain structures and systems involved in a particular process activated and what are their temporal relationships? To date, most human brain mapping studies have not addressed "When?" questions, largely due to methodological limitations. Although it is possible to ask profitable questions about temporal dynamics based on composite behavioral data (such as reaction time) alone, addressing the ''When?J question with respect to specific brain systems and structures provides a much richer approach, more rooted in neurobiology. Investigating temporal dynamics may also provide a more accurate answer to "Where?" questions because closely adjacent regions of activation may be distinguishable by their timecourse when not distinguishable by location information alone. "How?" refers to the specific neuronal events and operations performed by the brain systems and structures that are required to mediate the process in question. In the example of visual segmentation mentioned above, approaches to this question might include investigation of receptive field properties in various visual areas and the mechanisms that give rise to them. Although "Where?" and "When?' may be initially addressed on the scale of gross anatomical structures suitable for currently available brain mapping techniques, investigating "How?" generally requires techniques capable of resolving finer details of functional organization. At increasing finer levels of
+ Editorial +
detail, examples include cortical columns, the recep- namic and electromagnetic approaches to human tive field properties of individual neurons, and the brain mapping. It is first useful to distinguish between the forward connectivity and synaptic patterns that give rise to them. Ultimately, each of the questions can be asked at problem of electromagnetism, which is well-defined, every scale, although the value of the answers may and the inverse problem, which is not. The forward problem, calculating the electrical potential and/or vary. One important distinction among alternative brain magnetic fields outside the head given a particular mapping techniques (although by no means the only distribution of currents inside, requires two types of one), concerns the ”Where?” and “When?” questions; models: (a) a source model that embodies the propernamely, their sensitivity, resolution, and other charac- ties of the current sources involved; and (b) a volume teristics in the spatial and temporal domains. PET and model (or head model) that embodies the electromagfMRI are capable of making measurements simulta- netic properties of the brain, skull, and other tissues as neously at many locations throughout the brain vol- electrically conductive media. The accuracy of forume at spatial resolutions on the order of a few ward calculations of course depends upon the realism millimeters to a few centimeters. (The absolute spatial of the source and head models, but there is no resolution of these and other techniques to be dis- inherent indeterminacy in ‘calculating surface data cussed is a complex issue that depends on instrumen- given suitable source and head models. Source models tal resolution, the spatial resolution of the signals to be have ranged in complexity from single point-current measured, spatial characteristics of the noise, and the dipoles through dipole sheets and layers, to anatomispecific definition of spatial resolution being em- cally realistic current distributions. Similarly, head ployed). Because PET and fMRI are based on meta- models have ranged from single homogeneous bolic and hemodynamic responses to brain activation: spheres, through multi-compartment spherical shells, (a) they are indirect measures of neuronal activity to models having much greater geometrical, anatomimediated through the intervening relationship be- cal, and electrical realism. The electromagnetic inverse problem is the reverse of tween neuronal activity and the metabolic/hemodynamic response, which remains incompletely under- the forward problem, namely, calculating the current stood; and (b) their temporal resolution is ultimately distributions inside the brain given the electrical and/or limited by the timecourse of the metabolic/hemody- magnetic fields over the surface of the head. Solutions to the inverse problem require the same types of namic response (tenths of seconds to seconds). EEG and MEG, in contrast, are direct physical source and head models required for the forward consequences of neuronal currents and are capable of problem. In addition, the inverse problem requires resolving temporal patterns of neural activity in the techniques for choosing a ”best-fitting” inverse solumillisecond range. But they are limited by an inability tion for the data given the models under considerto estimate currents simultaneously throughout the ation. Two general approaches have been employed brain volume because the problem of estimating cur- which, for convenience, I will term dipole modeling rent sources in the brain from surface EEG and MEG and current imaging, respectively. Both approaches measurement is mathematically ill-posed; that is, it has can be used to model electromagnetic data at a single no unique solution in the most general, unconstrained instant or over an extended time epoch. Dipole modeling combines a small number of current case. Given these compensating strengths and limitations of metaboWhemodynamic and electromagnetic dipole sources (as few as one or as many as 6-10) with measures, it is not surprising that a number of investi- a particular head model and uses nonlinear leastgators have suggested that PET or fMRI could be squares minimization techniques to find the bestprofitably combined with electromagnetic measures fitting parameters for the model under consideration. such as EEG or MEG. But there is a deeper and more Each dipole source requires six parameters, three subtle reason why metabolic/hemodynamic and elec- corresponding to dipole locations and three vector tromagnetic measures could be mutually beneficial in components representing dipole moments (except for answering the “What?” and “When?” questions, one the special case of magnetic measurements using a that is rooted in the mathematics of the electromag- spherical head model in which only two dipole monetic inverse problem. In what follows, I will briefly ments are required). Thus, a 1-dipole model requires summarize the key components of the inverse prob- fitting 6 parameters to the 50-200 measurements lem and two alternative approaches to it, and then typically required for EEG/MEG source localization suggest how these approaches provide a strong math- studies, while a 6-dipole model requires fitting 36 ematical basis for integrating metabolic/hemody- parameters. Nonlinear minimization techniques are
Editorial
typically used for the dipole modeling approach because surface electrical and magnetic fields are nonlinear functions of the location parameters. In effect, such nonlinear minimization techniques calculate the forward problem over and over again with slight differences in parameter values in order to identify the combination of parameters that minimizes the error between the empirical data and the values calculated by the model. The resulting best-fitting solution for the dipole modeling approach is always relative to the assumptions of the model and careful investigators vary key properties of the model (e.g., the number of current sources) to determine that the solutions are as robust as possible. Current imaging also requires both a source model and a head model, and uses exactly the same mathematical formulation of the forward problem for a given head model as dipole modeling. But instead of assuming a small number of dipoles or other current sources, current imaging assumes a surface or volume of possible current elements (termed the reconstruction surface or volume) whose values are to be estimated. In other words, a small number of dipoles with unknown locations are replaced by a larger number of dipoles with known locations. This seemingly small change in assumptions has two important consequences: 1. Although the forward equations are nonlinear in the three location parameters for each source as noted above, they are linear in the three moment parameters for each source. The surface electrical and magnetic fields for any combination of sources represent the superposition of the fields from all active sources. Therefore, one needs to calculate the complete nonlinear forward problem for each assumed current element in the volume only once, and then the entire problem becomes a much simpler linear minimization problem in which only the moments for each assumed current element are estimated. (The approach of separating linear from nonlinear components of the inverse problem is so much more efficient that some investigators implement the conventional nonlinear minimization for dipole modeling by pre-computing the forward solutions for a sufficiently fine grid of source locations throughout the volume and then using exhaustive search to find the best solution for a particular dataset.) 2. The second major consequence of the strategy of pre-specifying the locations of possible currents is that the number of locations needed to represent adequately the surface or volume of tissue of interest is typically so large that it results in an underdetermined linear problem. That is, instead of the 1-10 dipole
sources typically used in the dipole fitting approach, the current imaging approach may utilize hundreds or thousands of source locations. This in turn yields systems of linear equations in which the number of unknowns greatly exceeds the number of measurements. To combat the problem of underdetermination while retaining the computational advantages of the current imaging approach, a number of investigators have explored ways to minimize the number of source locations required in the current imaging approach. One important approach under investigation by a number of groups is the use of anatomically constrained reconstruction volumes derived from individual subjects’ MRIs. In this approach, only regions of the brain that are possible significant contributors to surface EEG and/or MEG data are included in the reconstruction space, resulting in a much smaller degree of underdetermination. Because of its proximity to electromagnetic sensors and its anatomical structure suitable for generating electromagnetic fields recordable at a distance, neocortex has figured prominently in most explorations of anatomically constrained current estimation techniques. Such constraints are typically implemented by creating a segmented geometric model of the cortical mantle derived from anatomical MRI, populating the cortical mantle with current sources at a selected spatial scale, and calculating the forward problem once for each of the three orthogonal current orientations at each location. The results of such forward calculations are then stored as a “basis matrix” to be used in subsequent linear minimizations. Although the rationale for including or excluding particular anatomical structures in such constrained solutions is beyond the scope of this editorial, it is sufficient to note that both location and orientation constraints derived from realistic cortical geometry have been explored with promising results by multiple laboratories. It should now be obvious to many readers that the same mathematical framework used to implement anatomical constraints from MRI provides a means of integrating metabolic/hemodynamic and electromagnetic measures. In addition to constraining the reconstruction space of current imaging models using location and orientation information from anatomical MRI, it is possible to constrain those locations using patterns of activation derived from PET and/or fMRI data. Once the locations of activation have been correctly determined by whatever means (i.e., either from the electromagnetic data alone or by the addition of anatomical and/or functional constraints from other imaging modalities), then the timecourses of activa-
* v *
+ Editorial
tion can be computed using simple linear techniques, yielding information about the temporal dynamics of activated regions at millisecond resolution. A similar strategy can be used with the dipole modeling approach. In that case, regions of activation derived from PET and/or fMRI data could be used to estimate both the number and locations of dipole sources for use with nonlinear minimization techniques. In summary, while it is essential to continue to compare and integrate brain mapping techniques (e.g., including PET, MRI, MEG and EEG), there is, of course, much left to be done to improve our understanding of each technique alone and its relationship to the others. In particular, more work is needed: (a) to elucidate the detailed relationship between neuronal currents that generate EEG and MEG and the metabolic/hemodynamic responses evident in PET and fh4RI; and (b) to compare systematically the locations and extents of activation evident with different techniques. I believe the future is particularly bright for using the mathematical aspects of the electromagnetic
inverse problem discussed above to exploit the comparative strengths and offset the limitations of metabolic/hemodynamic and electromagnetic techniques. We should be gratified that a sound conceptual and mathematical base is in place for the integration of these approaches. Both "When?" and "Where?" questions in human brain mapping may soon become approachable to a much greater degree than we may have imagined. ACKNOWLEDGMENTS
Supported by the U.S. Department of Energy, National Institutes of Health, and Los Alamos National Laboratory. I am grateful to C.J. Aine, J.S. George, and H.A. Schlitt for helpful comments.
+ vi +
C.C. Wood Los Alamos National Laboratory, Los Alamos, New Mexico
