Journal of Parenteral and Enteral Nutrition http://pen.sagepub.com/
Identifiability and Parameter Estimation John A. Jacquez JPEN J Parenter Enteral Nutr 1991 15: 55S DOI: 10.1177/014860719101500355S The online version of this article can be found at: http://pen.sagepub.com/content/15/3/55S
Published by: http://www.sagepublications.com
On behalf of:
The American Society for Parenteral & Enteral Nutrition
Additional services and information for Journal of Parenteral and Enteral Nutrition can be found at: Email Alerts: http://pen.sagepub.com/cgi/alerts Subscriptions: http://pen.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://pen.sagepub.com/content/15/3/55S.refs.html
>> Version of Record - May 1, 1991 What is This?
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
55S
Identifiability and Parameter Estimation JOHN A. JACQUEZ, M.D. From the
Department of Physiology,
The
Uniuersity of Michigan
ABSTRACT. In experiments on biological systems one often cannot measure all state variables (compartments). Given a particular experiment of that type, a basic kinetic parameter may have no effect on the observations; such a parameter is an insensible parameter for that experiment. A parameter may influence the observations and not be uniquely determinable; such a parameter is nonidentifiable for that experiment. Only
Medical School, Ann Arbor,
Michigan
identifiable parameters can be estimated uniquely, by that experiment. I review the basic theory to check identifiability for a nominal value of a parameter (local identifiability), and present some examples of problems that may arise in estimation. ( Journal of Parenteral and Enteral Nutrition 15: 55S-59S,
1991)
The analysis of the distribution of materials in the body and of intermediary metabolism requires estimation of parameters of models of the processes involved. But, we
often cannot
measure
every
compartment. Then, it is
possible that the observations cannot uniquely determine some
of the parameters; such parameters
able. We start with
a
very
system consisting of zyme reaction in
a
a
are
unidentifi-
simple example. Consider the one-substrate, one-product
en-
closed vessel of constant volume.
Let Xi,x2 be the concentrations of S and P, and yl,y2 be those of E and ES, respectively. The model for the kinetics is given by the differential Equations 2 and 3,
If our observations consist only of Equation 6 for a number of substrate concentrations, we can estimate V m and K&dquo;~; those two compound parameters are identifiable from that experiment. Assuming we know how much enzyme was added we can estimate k3. However, there is no way we can estimate ki and k2- Of the basic kinetic parameters k3 is identifiable but kl and k2 are unidentifiable. Furthermore, one of the basic kinetic parameters, k4, does not influence the observations; k4 is said to be nonsensible for this particular experiment. From this example a number of significant points
stand out. 1. From the observations in the
experiments a set of compound parameters is defined which are functions of the basic kinetic parameters. The compound parameters are defined by the equations for the observations.
2. The basic kinetic
parameters may or
uniquely defined by the observations.
Eo is total
enzyme
concentration, So is total substrate
product concentration. Suppose we do the experiment of adding substrate to a system containing no S or
and
P at t = 0 and measure the initial rate of formation of product. If the intermediate product ES is formed almost instantaneously and then changes little thereafter, we obtain the standard Michaelis-Menten approximation to the initial velocity.
may not be
If not,
they
are
unidentifiable. 3. Observational errors add uncertainty to the estimation of parameters but have nothing to do with iden-
tifiability. SYSTEM, MODEL, EXPERIMENT
As
can
be
seen
from the
example just given
we
do
our
systems in the real world but analyze the results with use of a model of the system. The model specification includes the significant variables descrip-
experiments
on
tive of the model, i.e., the state variables, and the basic kinetic parameters. For us, model specification means writing the differential equations for the processes occurring in the system. W’e keep separate from the idea of a model of a system, the idea of an experiment done on a model of the system because we can define many
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
56S
different experiments that can potentially be done on a system and therefore for analysis we model the different experiments on the model of the system. To make these ideas explicit we give them in a general form that includes nonlinear and linear systems. Let x be the vector of state variables of a model. We write the differential
Here 9* is vector of
a
equations in general form.
vector of
parameters of the model,
input time functions and
Xo
are
u is a the initial
conditions. I think of the model as the state variables, model parameters and their relations as given by Equation 9 for initial conditions and inputs which are to be specified. The experiment one chooses to do specifies the initial conditions, the inputs and the observations, y.
Definitions Parameter identifiability. If a parameter, OJ, is defined by the observations so there is only the one solution, 8~, that parameter is globally identifiable, for that model, experiment and nominal values, 8°, for the basic parameters. It is possible for the observations to determine a number of other values for 8;, a finite set of values. Then 0j is said to be locally identifiable. Note that global identifiability is a special case of local identifiability for a
parameter. Structural identifiability. A property of a parameter is ’structural’ if it does not depend on the particular values chosen for the parameters. Thus structural global identifiability and structural local identifiability are generic properties that do not depend on the values chosen for
00. Model identifiability. If for 0 00, all of the parameters of a model are globally identifiable, the model is globally identifiable. If all of the parameters are locally identifiable and at least one is not globally identifiable the model is locally identifiable. There are the corresponding structural global identifiabilities and structural local identifiabilities for the model. =
Equation (10a), G is a function which specifies the observations as functions of the state variables and additional parameters 0’ which may be introduced by the observation process; we call the 0’ experimental parameters. Experimental parameters can also be introduced in initial conditions and the inputs. Thus, the full basic parameter vector, 0, has for components the components of 0* and 0’. If one solves Equation 9 to obtain the state variables as functions of time and the basic parameters, the observations can be rewritten as functions of time and parameters as in Equation 10b. In
In Equation (10b) G is called the response function and is written as a function of t and 0 and also as a function of t and a new vector of compound parameters, 0, the observational parameters. The oi are compound parameters such as VM and Km in Equations 7 and 8 whose values are uniquely determined by the observations, i.e., are identifiable, and which can be written as functions of the basic parameters. IDENTIFIABILITY
The identifiability problem has been recognized in a number of fields and a variety of related terms and ideas have arisen. Now it is necessary to give more precise definitions for the main ideas used in mathematical biology. For further reading see the text in References 1 and 2 and reviews in Reference 3. The material that follows is given in more detail in Ref. 4. In identifiability the concern is with the question of uniqueness of solutions for the parameters for a given model and experiment. Observational errors play no role here so the reader should think of the observations as the values of the response functions (Equation 10b) calculated for nominal values of the basic parameters for a given experiment. Suppose then that we generate observations for a set of values, 8°, of the basic parameters.
Classification of the Parameters We classify the basic parameters into three
sets.
Nonsensible or insensible parameters. If the sensitivity of the observations to variation in the value of a parameter is zero, the parameter is said to be nonsensible. Sensible parameters. If the observations change when a parameter value changes, that parameter is a sensible parameter; a parameter may be sensible and not be identifiable. Locally Identifiable. For the locally identifiable parameters 01, 0,, the observational parameters, oi fi (0), have a finite number of solutions for 01, Ow Nonidentifiable. For the nonidentifiable parameters, the number of solutions for 0 is infinite. ...,
=
...,
CHECKING LOCAL IDENTIFIABILITY AT A POINT
This work was started5,6 by noting that when the observational parameters are available as functions of the basic parameters, as in equations (6) and (7) of our example, the tests of identifiability can be done by examining the solutions of the functional relations, $; fi(81, o,). The oi are generally a nonlinear set of functions of the 0j whether the system model and observations are linear or nonlinear functions of the state variables. For problems of low dimensionality it is easy to generate the oi explicitly as functions of the 0j and check structural identifiability on the relations, (pi Ii (81, 8p). To test for local identifiability of the 0j the Oi are linearized around the initial estimates of the 8;. Local identifiability of a parameter, 0j, in the linearized equations implies local identifiability of the parameter in the nonlinear relations, oi fz(B1, 0p ). The reverse is not true; it is possible for a parameter to be locally identifiable in the nonlinear equations and to be nonidentifiable in the linearized equations. =
...,
=
...,
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
=
...,
57S For problems of even moderate magnitude the algebraic work involved in finding the 1J¡ becomes limiting. But if one has initial estimates of the basic parameters one can determine local identifiability numerically at the initial estimates without having to generate the observational parameters as explicit functions of the basic parameters. That approach involves linearizing the observations, Equation 10b, in the Oj and uses least squares to estimate the deviations of the basic parameters around the initial estimates.’ Here too, local identifiability of a parameter in the linearized observations implies local identifiability in the observations if they are nonlinear in the ~; again the reverse is not true. Least
Squares The following results
to
For the identifiable parameters, 0j, the only possible solutions of Equation 15 are a 0j 0. For the nonidentifiable parameters there should not be such unique solutions. To check that, we use row reduction of gsgs to row echelon form, with pivoting on maximum elements. When completed, the equations should have the following form. 1. Rows with only one nonzero entry and that on the diagonal. The column indexes of these diagonal elements are the indexes for the locally identifiable pa=
rameters. are
derived for
a
scalar observa-
tion ; the derivation extends directly to multiple observations. Let there be p basic parameters and assume we have initial estimates, 0i, Oop. For the Oj set at the initial estimates, calculate observations, yi, y~, at n ...,
...,
points in time for n >
gs. Then the normal equations corresponding the sensible parameters are given by Equation 15.
:matrix
p. Linearize the response function
in the
parameters Oj around the initial now play the role of known values.
estimates which
sum of squared deviations over the simulated data set as in Equation 12. Because there is no error in the observations, yi G°. Hence,
Form the
=
Find the least squares estimate of A0j. Let zi yl - G&dquo;. Let A 0 (A 0,, A 0p) be the least squares estimates of 1:::.8 and g be the sensitivity matrix, Equation 13. =
=
...,
2. Rows with non-zero entries on the diagonal and elsewhere. The column indexes of the non-zero elements of these rows are the indexes of nonidentifiable parameters.
3. Rows with all zero entries. These redundancy in the equations.
are
evidence of
Correlations Between
Identifiable Parameters to calculate the pairwise correlations
The final step is for the identifiable parameters. Two parameters may be identifiable, but it may be difficult to estimate the two separately in the presence of measurement error if they are highly correlated. To check the correlations, we reduce the matrix gigs by eliminating rows and columns corresponding to the nonidentifiable parameters to obtain gig,, which is nonsingular. The correlation matrix is obtained by dividing the i, j element of (gig,)-’ by the square root of the product of they and J&dquo;’ diagonal elements. Note that the values of the correlation coefficients depend on the parameter values and on the choice of points in time where the sensitivities are calculated. IDENTIFIABILITY AND ESTIMABILITY
In theory one tests identifiability first and then proceeds to estimate the identifiable parameters. However, for practical purpose there are areas of overlap. These are discussed in this section.
Identifiable The normal
Equation
equations
for the
estimates A 0j
are
given by
14.
Note that, since zi 0, if the determinant of g’ g ~ 0 the model is structurally locally idqntifiable, because then the only solution possible is 30 0. However, if the determinant of g’g 0, other solutions are possible. =
=
=
Local
Identifiability if Det(g’gJ
=
0
Insensible parameters. Examine the columns of g. For the insensible parameters the corresponding columns have only zero entries. Sensible parameters. Suppose we delete the columns of g corresponding to the insensible parameters to form the
but Poorly Estimable
A model or a set of parameters may be identifiable but the determinant of g’g is close to zero, resulting in poor estimability. That can occur in a number of ways. (1) It can occur as particular parameters approach a subspace (a space of lower dimension) on which those parameters are nonidentifiable. (2) It can also hold in a region of the parameter space (not just a space of lower dimensions) where there are large differences in values of some of the parameters so det(g’g) is near zero throughout the region. In such a region the errors in the observations will have such a large effect on the estimates that the estimates have high variances. One must recognize the problem by checking the det(g’g) and seek other experiments and/or other information to constrain the estimation. (3) Another way for this to occur is if one
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
58S chooses a set of sample points that is far from optimal. That can be a very difficult problem with real data sets.
Nonidentifiable but Estimable for Practical Purposes Even though a parameter may be nonidentifiable, the constraints in most problems restrict the physically permissible solution values to a subspace in parameter space which defines
an
interval in each of the nonidentifiable
parameters. 7,8 If the interval in a parameter is small that may be close
enough
for many
practical purposes.
‘Estimation’ of Nonidentifiable Parameters A large number of parameter fitting programs are used routinely and none of the standard programs check identifiability. What happens if one tries to estimate a nonidentifiable parameter? We will call the subspace of solutions for nonidentifiable parameters the subspace of feasible solutions. What a particular fitting routine does depends on the search technique used. Derivative methods such as the unmodified Gauss-Newton methods9,&dquo; blow up if there are unidentifiable parameters because g’g is singular. However, most modern derivative methods such as the Marquardt method9,10 modify the GaussNewton g’ g matrix by adding a small positive definite matrix to make it nonsingular and such modified methods walk into the feasible subspace and stop when their termination test is satisfied. The same is true of derivative-free methods such as direct search methods. Where such routines end up in the feasible subspace depends on the starting point and on the details of the search method. In what follows we refer specifically to the compartmental model-fitting software, SAAM11 which uses a modified Gauss-Newton method. A finding of a small overall sum of squares, indicating one is getting a good fit to the observations, with a large standard deviation of the estimates of some parameters should make one suspect some of the parameters are unidentifiable or identifiable but very poorly estimable! The following simple example illustrates what happens. Consider the system whose connection diagram is shown in Figure 1. Starting with nothing in the system, inject into compartment 1 at t 0 and then observe compartment 1. For this experiment only fol + f2l is identifiable. If fol + f21= k, that is the subspace of feasible solutions in parameter space, for positive values of the parameters, Figure 2. When presented with this problem, SAAM11 walks into the feasible subspace and stops! Using k 3, we generated data at ten sample points =
FIG. 2.
without added error; Figure 2 shows where the search ended for three different starting points. This example illustrates some of the clues one can get from a SAAM output. Using initial estimates of the parameters generate a set of observations on the model without added error; then fit the observations using a different starting point in the parameters. 1. If during a fit, a parameter doesn’t change in value and its fractional standard deviation (FSD) is high, repeat the run starting at a number of different points. If the same thing happens, the parameter is likely to be insensible. 2. Starting at a number of different initial points, if the estimation routine ends up in different final points a number of possibilities need to be explored. (a) The points are part of a feasible subspace for unidentifiable parameters. This is indicated by a good fit to the data but the parameters have high fractional standard deviations at each of the final points. (b) The final points are different points of local identifiability. All will have the same sum of squares, zero, and the FSDs for the parameters will be very small. (c) The search got hung up in local minima in the sum of squares surface. The sum of squares will generally differ at the different final points. If one runs into such problems a check of local identifiability at a number of points will often clarify the situation.
IDENT
=
FIG. l.
We have been developing a series of programs that check local identifiability of the parameters at a point in parameter space. The program follows the method displayed in Section entitled &dquo;Checking Local Identifiability at a Point&dquo; and so checks identifiability for the observations linearized about the nominal point in parameter space. Because highly accurate calculations are needed all programs use double-precision arithmetic. The user has to provide the differential equations for the model as external functions as well as the values of the parameters and control parameters for the program. The following Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
59S list
gives the characteristics of the different
REFERENCES
programs
and their availabilities. source code is in Fortran. The partial derivatives with respect to the parameters, i.e., the sensitivities, are calculated with a high order finite difference method. It has an interactive front-end which prompts the user through the input phase. Versions are available for the VAX-MTS and for the ATT Unix PC. 2. IDENT2C. An updated and streamlined version of IDENT2 written in C. We run this on a SUN SPARCstation. The coding is more efficient and it runs five to six times faster than IDENT2 on a VAX. 3. IDENT3. Similar to IDENT2 but with the numerical calculation of sensitivities replaced by direct integration of their differential equations. There is a small improvement in the calculated sensitivities but the work of input is markedly increased. This is also available for the VAX and the ATT Unix PC.
1. IDENT2. The
Godfrey K: Compartmental Models and Their Applications. Academic Press, London, 1983 2. Jacquez JA: Compartmental Analysis in Biology and Medicine, 2nd ed. The Univ. of Michigan Press, Ann Arbor, MI, 1985 3. Walter E (ed): Identifiability of Parametric Models. Pergamon Press, Oxford, 1987 4. Jacquez JA, Perry TJ: Parameter estimation: Local identifiability of parameters. Am J Physiol 258:E727-E736, 1990. 5. Jacquez JA, Grief P: Numerical parameter identifiability and estimability: Integrating identifiability, estimability and optimal sampling design. Math Biosci 77:210-227, 1985 6. Jacquez, JA: Identifiability: The first step in parameter estimation. Fed Proc 46:2477-2480, 1987 7. Berman M, Schoenfeld RL: Invariants in experimental data on linear kinetics and the formulation of models. J Appl Physics 1.
27:1361-1370, 1956 8. DiStefano III JJ: Complete parameter bounds and quasiidentifiability conditions for a class of unidentifiable linear systems. Math Biosci 65:51-68, 1983 9. Bard Y: Nonlinear Parameter Estimation. Academic Press, New
York, 1974 DM, Watts DG: Nonlinear Regression Analysis and its Applications. John Wiley & Sons, New York, 1988 Foster DM, Boston RC: The use of computers in compartmental analysis: The SAAM and CONSAM programs IN Compartmental Distribution of Radiotracers, Robertson JR (ed). CRC Press, Boca Raton, FL, 1983
10. Bates ACKNOWLEDGMENT
11.
This work was supported by United States Public Health Service Resource grant RR02176-01A1.
Noncompartmental Analysis in Metabolism KENNETH H. NORWICH, M.D., PH.D. From the Institute
of Biomedical Engineering and
the
Department of Physiology of the University of Toronto, Ontario, Canada
ABSTRACT. A new type of model called a department is formulated to complement the compartment. While a compartment may be dominated by convective transport, a department is dominated by diffusive transport. Although exit from a compartment is limited by the rate of biophysical or biochem-
ical removal ("kC"), exit from a department is limited by the rate of transport to the site of biophysical or biochemical removal. ( 59SJournal of Parenteral and Enteral Nutrition 15:
use of compartmental models has become the in recent decades, and perhaps with good reason. Compartments, which are regions of uniform concentration of metabolites, are easy to visualize. In some cases
mental
The
norm
anatomical locations can be ascribed to them. Interchange of substances and their tracers between these regions of uniform concentration seems to accord with our usual view of physiological transport; and excellent computer programs are available for conforming data to these compartmental models. However, there are circumstances in which compart-
64S, 1991)
models, whose solutions always involve sums of exponential functions, fail to describe experimental data in an adequate fashion. In such cases it is necessary to adopt noncompartmental models, even if it should mean the abrogation of our simple view of transport of materials in uiuo. BONE-SEEKING ELEMENTS
Noncompartmental models must be considered whenthe plasma clearance curve of a substance does not conform naturally to functions involving sums of expo-
ever
nentials functions. One of the earliest examples of such a phenomenon was reported by Threefoot et al,l who Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
Identifiability and Parameter Estimation John A. Jacquez JPEN J Parenter Enteral Nutr 1991 15: 55S DOI: 10.1177/014860719101500355S The online version of this article can be found at: http://pen.sagepub.com/content/15/3/55S
Published by: http://www.sagepublications.com
On behalf of:
The American Society for Parenteral & Enteral Nutrition
Additional services and information for Journal of Parenteral and Enteral Nutrition can be found at: Email Alerts: http://pen.sagepub.com/cgi/alerts Subscriptions: http://pen.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://pen.sagepub.com/content/15/3/55S.refs.html
>> Version of Record - May 1, 1991 What is This?
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
55S
Identifiability and Parameter Estimation JOHN A. JACQUEZ, M.D. From the
Department of Physiology,
The
Uniuersity of Michigan
ABSTRACT. In experiments on biological systems one often cannot measure all state variables (compartments). Given a particular experiment of that type, a basic kinetic parameter may have no effect on the observations; such a parameter is an insensible parameter for that experiment. A parameter may influence the observations and not be uniquely determinable; such a parameter is nonidentifiable for that experiment. Only
Medical School, Ann Arbor,
Michigan
identifiable parameters can be estimated uniquely, by that experiment. I review the basic theory to check identifiability for a nominal value of a parameter (local identifiability), and present some examples of problems that may arise in estimation. ( Journal of Parenteral and Enteral Nutrition 15: 55S-59S,
1991)
The analysis of the distribution of materials in the body and of intermediary metabolism requires estimation of parameters of models of the processes involved. But, we
often cannot
measure
every
compartment. Then, it is
possible that the observations cannot uniquely determine some
of the parameters; such parameters
able. We start with
a
very
system consisting of zyme reaction in
a
a
are
unidentifi-
simple example. Consider the one-substrate, one-product
en-
closed vessel of constant volume.
Let Xi,x2 be the concentrations of S and P, and yl,y2 be those of E and ES, respectively. The model for the kinetics is given by the differential Equations 2 and 3,
If our observations consist only of Equation 6 for a number of substrate concentrations, we can estimate V m and K&dquo;~; those two compound parameters are identifiable from that experiment. Assuming we know how much enzyme was added we can estimate k3. However, there is no way we can estimate ki and k2- Of the basic kinetic parameters k3 is identifiable but kl and k2 are unidentifiable. Furthermore, one of the basic kinetic parameters, k4, does not influence the observations; k4 is said to be nonsensible for this particular experiment. From this example a number of significant points
stand out. 1. From the observations in the
experiments a set of compound parameters is defined which are functions of the basic kinetic parameters. The compound parameters are defined by the equations for the observations.
2. The basic kinetic
parameters may or
uniquely defined by the observations.
Eo is total
enzyme
concentration, So is total substrate
product concentration. Suppose we do the experiment of adding substrate to a system containing no S or
and
P at t = 0 and measure the initial rate of formation of product. If the intermediate product ES is formed almost instantaneously and then changes little thereafter, we obtain the standard Michaelis-Menten approximation to the initial velocity.
may not be
If not,
they
are
unidentifiable. 3. Observational errors add uncertainty to the estimation of parameters but have nothing to do with iden-
tifiability. SYSTEM, MODEL, EXPERIMENT
As
can
be
seen
from the
example just given
we
do
our
systems in the real world but analyze the results with use of a model of the system. The model specification includes the significant variables descrip-
experiments
on
tive of the model, i.e., the state variables, and the basic kinetic parameters. For us, model specification means writing the differential equations for the processes occurring in the system. W’e keep separate from the idea of a model of a system, the idea of an experiment done on a model of the system because we can define many
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
56S
different experiments that can potentially be done on a system and therefore for analysis we model the different experiments on the model of the system. To make these ideas explicit we give them in a general form that includes nonlinear and linear systems. Let x be the vector of state variables of a model. We write the differential
Here 9* is vector of
a
equations in general form.
vector of
parameters of the model,
input time functions and
Xo
are
u is a the initial
conditions. I think of the model as the state variables, model parameters and their relations as given by Equation 9 for initial conditions and inputs which are to be specified. The experiment one chooses to do specifies the initial conditions, the inputs and the observations, y.
Definitions Parameter identifiability. If a parameter, OJ, is defined by the observations so there is only the one solution, 8~, that parameter is globally identifiable, for that model, experiment and nominal values, 8°, for the basic parameters. It is possible for the observations to determine a number of other values for 8;, a finite set of values. Then 0j is said to be locally identifiable. Note that global identifiability is a special case of local identifiability for a
parameter. Structural identifiability. A property of a parameter is ’structural’ if it does not depend on the particular values chosen for the parameters. Thus structural global identifiability and structural local identifiability are generic properties that do not depend on the values chosen for
00. Model identifiability. If for 0 00, all of the parameters of a model are globally identifiable, the model is globally identifiable. If all of the parameters are locally identifiable and at least one is not globally identifiable the model is locally identifiable. There are the corresponding structural global identifiabilities and structural local identifiabilities for the model. =
Equation (10a), G is a function which specifies the observations as functions of the state variables and additional parameters 0’ which may be introduced by the observation process; we call the 0’ experimental parameters. Experimental parameters can also be introduced in initial conditions and the inputs. Thus, the full basic parameter vector, 0, has for components the components of 0* and 0’. If one solves Equation 9 to obtain the state variables as functions of time and the basic parameters, the observations can be rewritten as functions of time and parameters as in Equation 10b. In
In Equation (10b) G is called the response function and is written as a function of t and 0 and also as a function of t and a new vector of compound parameters, 0, the observational parameters. The oi are compound parameters such as VM and Km in Equations 7 and 8 whose values are uniquely determined by the observations, i.e., are identifiable, and which can be written as functions of the basic parameters. IDENTIFIABILITY
The identifiability problem has been recognized in a number of fields and a variety of related terms and ideas have arisen. Now it is necessary to give more precise definitions for the main ideas used in mathematical biology. For further reading see the text in References 1 and 2 and reviews in Reference 3. The material that follows is given in more detail in Ref. 4. In identifiability the concern is with the question of uniqueness of solutions for the parameters for a given model and experiment. Observational errors play no role here so the reader should think of the observations as the values of the response functions (Equation 10b) calculated for nominal values of the basic parameters for a given experiment. Suppose then that we generate observations for a set of values, 8°, of the basic parameters.
Classification of the Parameters We classify the basic parameters into three
sets.
Nonsensible or insensible parameters. If the sensitivity of the observations to variation in the value of a parameter is zero, the parameter is said to be nonsensible. Sensible parameters. If the observations change when a parameter value changes, that parameter is a sensible parameter; a parameter may be sensible and not be identifiable. Locally Identifiable. For the locally identifiable parameters 01, 0,, the observational parameters, oi fi (0), have a finite number of solutions for 01, Ow Nonidentifiable. For the nonidentifiable parameters, the number of solutions for 0 is infinite. ...,
=
...,
CHECKING LOCAL IDENTIFIABILITY AT A POINT
This work was started5,6 by noting that when the observational parameters are available as functions of the basic parameters, as in equations (6) and (7) of our example, the tests of identifiability can be done by examining the solutions of the functional relations, $; fi(81, o,). The oi are generally a nonlinear set of functions of the 0j whether the system model and observations are linear or nonlinear functions of the state variables. For problems of low dimensionality it is easy to generate the oi explicitly as functions of the 0j and check structural identifiability on the relations, (pi Ii (81, 8p). To test for local identifiability of the 0j the Oi are linearized around the initial estimates of the 8;. Local identifiability of a parameter, 0j, in the linearized equations implies local identifiability of the parameter in the nonlinear relations, oi fz(B1, 0p ). The reverse is not true; it is possible for a parameter to be locally identifiable in the nonlinear equations and to be nonidentifiable in the linearized equations. =
...,
=
...,
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
=
...,
57S For problems of even moderate magnitude the algebraic work involved in finding the 1J¡ becomes limiting. But if one has initial estimates of the basic parameters one can determine local identifiability numerically at the initial estimates without having to generate the observational parameters as explicit functions of the basic parameters. That approach involves linearizing the observations, Equation 10b, in the Oj and uses least squares to estimate the deviations of the basic parameters around the initial estimates.’ Here too, local identifiability of a parameter in the linearized observations implies local identifiability in the observations if they are nonlinear in the ~; again the reverse is not true. Least
Squares The following results
to
For the identifiable parameters, 0j, the only possible solutions of Equation 15 are a 0j 0. For the nonidentifiable parameters there should not be such unique solutions. To check that, we use row reduction of gsgs to row echelon form, with pivoting on maximum elements. When completed, the equations should have the following form. 1. Rows with only one nonzero entry and that on the diagonal. The column indexes of these diagonal elements are the indexes for the locally identifiable pa=
rameters. are
derived for
a
scalar observa-
tion ; the derivation extends directly to multiple observations. Let there be p basic parameters and assume we have initial estimates, 0i, Oop. For the Oj set at the initial estimates, calculate observations, yi, y~, at n ...,
...,
points in time for n >
gs. Then the normal equations corresponding the sensible parameters are given by Equation 15.
:matrix
p. Linearize the response function
in the
parameters Oj around the initial now play the role of known values.
estimates which
sum of squared deviations over the simulated data set as in Equation 12. Because there is no error in the observations, yi G°. Hence,
Form the
=
Find the least squares estimate of A0j. Let zi yl - G&dquo;. Let A 0 (A 0,, A 0p) be the least squares estimates of 1:::.8 and g be the sensitivity matrix, Equation 13. =
=
...,
2. Rows with non-zero entries on the diagonal and elsewhere. The column indexes of the non-zero elements of these rows are the indexes of nonidentifiable parameters.
3. Rows with all zero entries. These redundancy in the equations.
are
evidence of
Correlations Between
Identifiable Parameters to calculate the pairwise correlations
The final step is for the identifiable parameters. Two parameters may be identifiable, but it may be difficult to estimate the two separately in the presence of measurement error if they are highly correlated. To check the correlations, we reduce the matrix gigs by eliminating rows and columns corresponding to the nonidentifiable parameters to obtain gig,, which is nonsingular. The correlation matrix is obtained by dividing the i, j element of (gig,)-’ by the square root of the product of they and J&dquo;’ diagonal elements. Note that the values of the correlation coefficients depend on the parameter values and on the choice of points in time where the sensitivities are calculated. IDENTIFIABILITY AND ESTIMABILITY
In theory one tests identifiability first and then proceeds to estimate the identifiable parameters. However, for practical purpose there are areas of overlap. These are discussed in this section.
Identifiable The normal
Equation
equations
for the
estimates A 0j
are
given by
14.
Note that, since zi 0, if the determinant of g’ g ~ 0 the model is structurally locally idqntifiable, because then the only solution possible is 30 0. However, if the determinant of g’g 0, other solutions are possible. =
=
=
Local
Identifiability if Det(g’gJ
=
0
Insensible parameters. Examine the columns of g. For the insensible parameters the corresponding columns have only zero entries. Sensible parameters. Suppose we delete the columns of g corresponding to the insensible parameters to form the
but Poorly Estimable
A model or a set of parameters may be identifiable but the determinant of g’g is close to zero, resulting in poor estimability. That can occur in a number of ways. (1) It can occur as particular parameters approach a subspace (a space of lower dimension) on which those parameters are nonidentifiable. (2) It can also hold in a region of the parameter space (not just a space of lower dimensions) where there are large differences in values of some of the parameters so det(g’g) is near zero throughout the region. In such a region the errors in the observations will have such a large effect on the estimates that the estimates have high variances. One must recognize the problem by checking the det(g’g) and seek other experiments and/or other information to constrain the estimation. (3) Another way for this to occur is if one
Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
58S chooses a set of sample points that is far from optimal. That can be a very difficult problem with real data sets.
Nonidentifiable but Estimable for Practical Purposes Even though a parameter may be nonidentifiable, the constraints in most problems restrict the physically permissible solution values to a subspace in parameter space which defines
an
interval in each of the nonidentifiable
parameters. 7,8 If the interval in a parameter is small that may be close
enough
for many
practical purposes.
‘Estimation’ of Nonidentifiable Parameters A large number of parameter fitting programs are used routinely and none of the standard programs check identifiability. What happens if one tries to estimate a nonidentifiable parameter? We will call the subspace of solutions for nonidentifiable parameters the subspace of feasible solutions. What a particular fitting routine does depends on the search technique used. Derivative methods such as the unmodified Gauss-Newton methods9,&dquo; blow up if there are unidentifiable parameters because g’g is singular. However, most modern derivative methods such as the Marquardt method9,10 modify the GaussNewton g’ g matrix by adding a small positive definite matrix to make it nonsingular and such modified methods walk into the feasible subspace and stop when their termination test is satisfied. The same is true of derivative-free methods such as direct search methods. Where such routines end up in the feasible subspace depends on the starting point and on the details of the search method. In what follows we refer specifically to the compartmental model-fitting software, SAAM11 which uses a modified Gauss-Newton method. A finding of a small overall sum of squares, indicating one is getting a good fit to the observations, with a large standard deviation of the estimates of some parameters should make one suspect some of the parameters are unidentifiable or identifiable but very poorly estimable! The following simple example illustrates what happens. Consider the system whose connection diagram is shown in Figure 1. Starting with nothing in the system, inject into compartment 1 at t 0 and then observe compartment 1. For this experiment only fol + f2l is identifiable. If fol + f21= k, that is the subspace of feasible solutions in parameter space, for positive values of the parameters, Figure 2. When presented with this problem, SAAM11 walks into the feasible subspace and stops! Using k 3, we generated data at ten sample points =
FIG. 2.
without added error; Figure 2 shows where the search ended for three different starting points. This example illustrates some of the clues one can get from a SAAM output. Using initial estimates of the parameters generate a set of observations on the model without added error; then fit the observations using a different starting point in the parameters. 1. If during a fit, a parameter doesn’t change in value and its fractional standard deviation (FSD) is high, repeat the run starting at a number of different points. If the same thing happens, the parameter is likely to be insensible. 2. Starting at a number of different initial points, if the estimation routine ends up in different final points a number of possibilities need to be explored. (a) The points are part of a feasible subspace for unidentifiable parameters. This is indicated by a good fit to the data but the parameters have high fractional standard deviations at each of the final points. (b) The final points are different points of local identifiability. All will have the same sum of squares, zero, and the FSDs for the parameters will be very small. (c) The search got hung up in local minima in the sum of squares surface. The sum of squares will generally differ at the different final points. If one runs into such problems a check of local identifiability at a number of points will often clarify the situation.
IDENT
=
FIG. l.
We have been developing a series of programs that check local identifiability of the parameters at a point in parameter space. The program follows the method displayed in Section entitled &dquo;Checking Local Identifiability at a Point&dquo; and so checks identifiability for the observations linearized about the nominal point in parameter space. Because highly accurate calculations are needed all programs use double-precision arithmetic. The user has to provide the differential equations for the model as external functions as well as the values of the parameters and control parameters for the program. The following Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
59S list
gives the characteristics of the different
REFERENCES
programs
and their availabilities. source code is in Fortran. The partial derivatives with respect to the parameters, i.e., the sensitivities, are calculated with a high order finite difference method. It has an interactive front-end which prompts the user through the input phase. Versions are available for the VAX-MTS and for the ATT Unix PC. 2. IDENT2C. An updated and streamlined version of IDENT2 written in C. We run this on a SUN SPARCstation. The coding is more efficient and it runs five to six times faster than IDENT2 on a VAX. 3. IDENT3. Similar to IDENT2 but with the numerical calculation of sensitivities replaced by direct integration of their differential equations. There is a small improvement in the calculated sensitivities but the work of input is markedly increased. This is also available for the VAX and the ATT Unix PC.
1. IDENT2. The
Godfrey K: Compartmental Models and Their Applications. Academic Press, London, 1983 2. Jacquez JA: Compartmental Analysis in Biology and Medicine, 2nd ed. The Univ. of Michigan Press, Ann Arbor, MI, 1985 3. Walter E (ed): Identifiability of Parametric Models. Pergamon Press, Oxford, 1987 4. Jacquez JA, Perry TJ: Parameter estimation: Local identifiability of parameters. Am J Physiol 258:E727-E736, 1990. 5. Jacquez JA, Grief P: Numerical parameter identifiability and estimability: Integrating identifiability, estimability and optimal sampling design. Math Biosci 77:210-227, 1985 6. Jacquez, JA: Identifiability: The first step in parameter estimation. Fed Proc 46:2477-2480, 1987 7. Berman M, Schoenfeld RL: Invariants in experimental data on linear kinetics and the formulation of models. J Appl Physics 1.
27:1361-1370, 1956 8. DiStefano III JJ: Complete parameter bounds and quasiidentifiability conditions for a class of unidentifiable linear systems. Math Biosci 65:51-68, 1983 9. Bard Y: Nonlinear Parameter Estimation. Academic Press, New
York, 1974 DM, Watts DG: Nonlinear Regression Analysis and its Applications. John Wiley & Sons, New York, 1988 Foster DM, Boston RC: The use of computers in compartmental analysis: The SAAM and CONSAM programs IN Compartmental Distribution of Radiotracers, Robertson JR (ed). CRC Press, Boca Raton, FL, 1983
10. Bates ACKNOWLEDGMENT
11.
This work was supported by United States Public Health Service Resource grant RR02176-01A1.
Noncompartmental Analysis in Metabolism KENNETH H. NORWICH, M.D., PH.D. From the Institute
of Biomedical Engineering and
the
Department of Physiology of the University of Toronto, Ontario, Canada
ABSTRACT. A new type of model called a department is formulated to complement the compartment. While a compartment may be dominated by convective transport, a department is dominated by diffusive transport. Although exit from a compartment is limited by the rate of biophysical or biochem-
ical removal ("kC"), exit from a department is limited by the rate of transport to the site of biophysical or biochemical removal. ( 59SJournal of Parenteral and Enteral Nutrition 15:
use of compartmental models has become the in recent decades, and perhaps with good reason. Compartments, which are regions of uniform concentration of metabolites, are easy to visualize. In some cases
mental
The
norm
anatomical locations can be ascribed to them. Interchange of substances and their tracers between these regions of uniform concentration seems to accord with our usual view of physiological transport; and excellent computer programs are available for conforming data to these compartmental models. However, there are circumstances in which compart-
64S, 1991)
models, whose solutions always involve sums of exponential functions, fail to describe experimental data in an adequate fashion. In such cases it is necessary to adopt noncompartmental models, even if it should mean the abrogation of our simple view of transport of materials in uiuo. BONE-SEEKING ELEMENTS
Noncompartmental models must be considered whenthe plasma clearance curve of a substance does not conform naturally to functions involving sums of expo-
ever
nentials functions. One of the earliest examples of such a phenomenon was reported by Threefoot et al,l who Downloaded from pen.sagepub.com at St Petersburg State University on January 2, 2014
