perspec tives
nature publishing group
What Happened to the Modeling and Simulation Revolution? PL Bonate1
Modeling and simulation (M&S) is not as influential as it could be in drug development because of poor model communicators and insulated success stories. Until pharmacometricians are rewarded for being better communicators, and until they start presenting their research at clinical meetings and publishing their results outside the field of clinical pharmacology, the model-based drug development revolution will remain stalled and not progress to the next level. In 2003, I gave a presentation at the Annual Meeting of the American Association of Pharmaceutical Scientists entitled “Overcoming Impediments to Modeling and Simulation (M&S) in Drug Development.” I focused largely on the technical challenges facing pharmacometricians. At the conclusion of my talk, I stated that M&S would revolutionize drug development. More than 10 years later, I have to ask what happened to the revolution. Some might argue that the revolution has taken place; just look at Pfizer,1,2 which has been promoting model-based drug development (MBDD) and its success stories for years. But I would argue that MBDD may not be all sunshine and lollipops. Some recent presentations suggest that MBDD should perhaps have been named “model-facilitated” or “modelinformed drug development.” Jack Cook, of Pfizer, recently presented a report card on MBDD’s status at Pfizer.3 He gave a B in “more efficient drug development,” a D in “compound selection”, and an “Incomplete” in “portfolio management and comparative effectiveness.” Christoffer Tornoe and the American Society of Clinical Pharmacology and Therapeutics’ Pharmacometrics and Pharmacokinetics Scientific Section reported that most pharmacometric analyses are
still requested by a company’s modeling department.4 If MBDD is having such an impact, wouldn’t other departments, such as Regulatory Affairs and Clinical Development, be demanding it? Finally (you can discount this one if you wish, because it’s anecdotal), in my conversations with other pharmacometricians I can’t tell you how many times they have bemoaned the difficulty of pushing an MBDD agenda at their companies. But why is that? Why is pushing a MBDD agenda so hard? Maybe it’s because the revolution has stalled. Scientific revolutions are supposed to change the way people view the world. Ten years ago, there was a buzz in the air at meetings and conferences about M&S; today, not so much. What happened? There are many reasons that the revolution has stalled, and it has nothing to do with modeling. I think the biggest reason is that most pharmacometricians have poor communication skills. I know many brilliant pharmacometricians who are incredibly boring presenters. When presenting to project teams, who really know very little about M&S, they show tables of parameter estimates and goodness-of-fit plots— things the team cares nothing about and cannot understand. They talk about thetas and etas and FOCE and drops in the objective function. Few outside our community know what those terms mean. Poor communication comes as no surprise, given that communication skills are largely ignored by graduate schools and obtuse writing in scientific journals is encouraged in the name of scientific rigor. For the past decade I have encouraged new pharmacometricians entering industry to sign up for classes in public speaking or join a community group such as Toastmasters. Few do so, because they
see no incentive to being good public speakers. In fact, most of the pharmacometricians I know think they are good public speakers, and I don’t have the heart to tell them otherwise. (This includes me, so I am not sure what that says about me.) Theirs is a mistaken perspective, because, as the head of M&S at a global pharmaceutical company, I would rather hire a solid modeler with great communication skills than a great modeler with poor communication skills. In fact, in an effort to hire better communicators, at Astellas we have stopped the practice of the scientific seminar during the pharmacometrician hiring process. Instead, candidates are given NONMEM output for a relatively simple model and are asked to pretend that they have to present these modeling results to a project team. Before their interview they must also prepare a one- to two-page written summary, and on the day of the interview they have 15 minutes for their presentation. In this manner we have standardized the evaluation process so that candidates’ communication skills can be equally compared. Besides using incomprehensible jargon, pharmacometricians shoot themselves in the foot with the words they commonly use. Let’s start with “pharmacometrics.” When you get a spare moment at work next time, ask someone who is not in clinical pharmacology to define the word. Most don’t know, and those who think they know are often wrong. Thus, when we go outside clinical pharmacology and talk about pharmacometrics, most people have no idea what we are talking about. Another example of unclear communication is provided by pharmacometricians’ quoting George Box: “all models are wrong, some are useful.”5 Pharmacometricians
1 Pharmacokinetics, Modeling & Simulation, Astellas Pharma, Northbrook, Illinois, USA. Correspondence: PL Bonate ([email protected])
doi:10.1038/clpt.2014.123 416
VOLUME 96 NUMBER 4 | OCTOBER 2014 | www.nature.com/cpt
perspec tives know the quotation means that biological data are complicated. A mechanistic mathematical model could never replicate all the systems that are in play, but a model—one that may be a quite simple representation of reality—may adequately explain observed data and predict future responses. When that quotation is repeated to a nontechnical audience, what do you think they hear? They hear that your model is “wrong.” If your model is wrong, why should they listen to you? If it’s a wrong model, why are you talking about it? It’s confusing to them. As my last example—and there are plenty more I could have chosen—pharmacometricians use the term “the final model.” Pharmacometricians know that the final model was the last in a long series of models examined and was deemed superior to the other models. Outside our field, the word “final” has different meanings to different people. Some may understand what a pharmacometrician intends, but others will take it to mean “last,” “supreme,” “ultimate,” “not to be altered,” “accurate,” or “correct and without error.” Consider the outcome when new data are available, the model is updated, and the pharmacometrician presents the new “final model.” The nonmodeler thinks to himself, “I thought last time was the final model; now they are telling me they have another final model. Was a mistake made last time? Why is there a new final model? Will there be another final model?” A better phrase than “the final model” is needed. I use “the best model” because “best” implies that many models were examined and this model was superior to the others. “Best” also allows for the possibility that in the future a better model might be found, such as when the model is updated with new data. The message here is that as scientists we need to be
cognizant that the words we use matter, even the simple ones. The last reason for my belief that the revolution has stalled is that pharmacometricians are too insulated in that they seldom go outside the clinical pharmacology arena with their models. As I was writing this, I did a keyword search on “model-based drug development” in PubMed. A total of 41 citations were retrieved. Every single article was from a journal in clinical pharmacology, pharmaceutics, or pharmacokinetics. It comes as no surprise, then, that when you talk about MBDD to drug developers outside clinical pharmacology, you get blank stares. I understand that when an elegant exposure–response relationship is developed, we want to get credit from our peers who appreciate its beauty, but we should not limit ourselves by preaching to the choir. If pharmacometricians want to truly have influence outside clinical pharmacology, they need to publish in the clinical journals and present at clinical or professional meetings such as those of the American Society of Clinical Oncology and the Drug Information Association, rather than focus on the usual disciplinerelated journals and meetings. I could go on and on. Most people don’t understand models: they don’t know how they work, they don’t know how to evaluate them, they don’t know what makes a good model, etc. There is a general lack of training among biologists and physicians in quantitative methods. Most people—even scientists—have a poor understanding of probability, especially low probability. Models take too long to be developed, and by the time a model is developed the decision has usually already been made. In most companies, pharmacometricians sit at the periphery of the project team and are not central members of the development process.
The list of obstacles to M&S’s widespread acceptance is long, but I think that poor communication skills are at the top. Pharmacometricians (a group to which I obviously belong) need to do better. We need to learn that it isn’t just about the model. Communicating about the model is as important, if not more important, because if you can’t communicate the results of the model to a nontechnical project team, the model is not going to be used. If you learn to be a model communicator (pun intended), you will be a highly sought-after individual within your company and will have tremendous influence. Pharmacometricians also need to be better messengers and to champion M&S outside our field. We need to present success stories at forums other than the usual clinical pharmacology and pharmacokinetics meetings and show how our models improved decision making. Only as pharmacometrics permeates other fields will its influence be truly felt, and MBDD will stop being a dream and become reality. Vive la révolution! CONFLICT OF INTEREST The author declared no conflict of interest. © 2014 ASCPT
1. Lalonde, R. et al. Model-based drug development. Clin. Pharmacol. Ther. 82, 21–32 (2007). 2. Milligan, P.A. et al. Model-based drug development: a rational approach to efficiently accelerate drug development. Clin. Pharmacol. Ther. 93, 502–514 (2013). 3. Cook, J. Is model-based drug development delivering on its promise?: It depends on what the meaning of the word “is” is. American Conference on Pharmacometrics, Fort Lauderdale, FL, 12–15 May 2013. 4. Tornoe, C.W.; the ASCPT PMK Committee. PMX in submission: where are we now? Where should we go? American Society of Clinical Pharmacology and Therapeutics Annual Meeting, Atlanta, GA, 18–22 March 2014. 5. Box, G.E.P. & Draper, N. Empirical Model-Building and Response Surfaces 424 (Wiley, New York, 1987).
Clinical pharmacology & Therapeutics | VOLUME 96 NUMBER 4 | OCTOBER 2014
417
nature publishing group
What Happened to the Modeling and Simulation Revolution? PL Bonate1
Modeling and simulation (M&S) is not as influential as it could be in drug development because of poor model communicators and insulated success stories. Until pharmacometricians are rewarded for being better communicators, and until they start presenting their research at clinical meetings and publishing their results outside the field of clinical pharmacology, the model-based drug development revolution will remain stalled and not progress to the next level. In 2003, I gave a presentation at the Annual Meeting of the American Association of Pharmaceutical Scientists entitled “Overcoming Impediments to Modeling and Simulation (M&S) in Drug Development.” I focused largely on the technical challenges facing pharmacometricians. At the conclusion of my talk, I stated that M&S would revolutionize drug development. More than 10 years later, I have to ask what happened to the revolution. Some might argue that the revolution has taken place; just look at Pfizer,1,2 which has been promoting model-based drug development (MBDD) and its success stories for years. But I would argue that MBDD may not be all sunshine and lollipops. Some recent presentations suggest that MBDD should perhaps have been named “model-facilitated” or “modelinformed drug development.” Jack Cook, of Pfizer, recently presented a report card on MBDD’s status at Pfizer.3 He gave a B in “more efficient drug development,” a D in “compound selection”, and an “Incomplete” in “portfolio management and comparative effectiveness.” Christoffer Tornoe and the American Society of Clinical Pharmacology and Therapeutics’ Pharmacometrics and Pharmacokinetics Scientific Section reported that most pharmacometric analyses are
still requested by a company’s modeling department.4 If MBDD is having such an impact, wouldn’t other departments, such as Regulatory Affairs and Clinical Development, be demanding it? Finally (you can discount this one if you wish, because it’s anecdotal), in my conversations with other pharmacometricians I can’t tell you how many times they have bemoaned the difficulty of pushing an MBDD agenda at their companies. But why is that? Why is pushing a MBDD agenda so hard? Maybe it’s because the revolution has stalled. Scientific revolutions are supposed to change the way people view the world. Ten years ago, there was a buzz in the air at meetings and conferences about M&S; today, not so much. What happened? There are many reasons that the revolution has stalled, and it has nothing to do with modeling. I think the biggest reason is that most pharmacometricians have poor communication skills. I know many brilliant pharmacometricians who are incredibly boring presenters. When presenting to project teams, who really know very little about M&S, they show tables of parameter estimates and goodness-of-fit plots— things the team cares nothing about and cannot understand. They talk about thetas and etas and FOCE and drops in the objective function. Few outside our community know what those terms mean. Poor communication comes as no surprise, given that communication skills are largely ignored by graduate schools and obtuse writing in scientific journals is encouraged in the name of scientific rigor. For the past decade I have encouraged new pharmacometricians entering industry to sign up for classes in public speaking or join a community group such as Toastmasters. Few do so, because they
see no incentive to being good public speakers. In fact, most of the pharmacometricians I know think they are good public speakers, and I don’t have the heart to tell them otherwise. (This includes me, so I am not sure what that says about me.) Theirs is a mistaken perspective, because, as the head of M&S at a global pharmaceutical company, I would rather hire a solid modeler with great communication skills than a great modeler with poor communication skills. In fact, in an effort to hire better communicators, at Astellas we have stopped the practice of the scientific seminar during the pharmacometrician hiring process. Instead, candidates are given NONMEM output for a relatively simple model and are asked to pretend that they have to present these modeling results to a project team. Before their interview they must also prepare a one- to two-page written summary, and on the day of the interview they have 15 minutes for their presentation. In this manner we have standardized the evaluation process so that candidates’ communication skills can be equally compared. Besides using incomprehensible jargon, pharmacometricians shoot themselves in the foot with the words they commonly use. Let’s start with “pharmacometrics.” When you get a spare moment at work next time, ask someone who is not in clinical pharmacology to define the word. Most don’t know, and those who think they know are often wrong. Thus, when we go outside clinical pharmacology and talk about pharmacometrics, most people have no idea what we are talking about. Another example of unclear communication is provided by pharmacometricians’ quoting George Box: “all models are wrong, some are useful.”5 Pharmacometricians
1 Pharmacokinetics, Modeling & Simulation, Astellas Pharma, Northbrook, Illinois, USA. Correspondence: PL Bonate ([email protected])
doi:10.1038/clpt.2014.123 416
VOLUME 96 NUMBER 4 | OCTOBER 2014 | www.nature.com/cpt
perspec tives know the quotation means that biological data are complicated. A mechanistic mathematical model could never replicate all the systems that are in play, but a model—one that may be a quite simple representation of reality—may adequately explain observed data and predict future responses. When that quotation is repeated to a nontechnical audience, what do you think they hear? They hear that your model is “wrong.” If your model is wrong, why should they listen to you? If it’s a wrong model, why are you talking about it? It’s confusing to them. As my last example—and there are plenty more I could have chosen—pharmacometricians use the term “the final model.” Pharmacometricians know that the final model was the last in a long series of models examined and was deemed superior to the other models. Outside our field, the word “final” has different meanings to different people. Some may understand what a pharmacometrician intends, but others will take it to mean “last,” “supreme,” “ultimate,” “not to be altered,” “accurate,” or “correct and without error.” Consider the outcome when new data are available, the model is updated, and the pharmacometrician presents the new “final model.” The nonmodeler thinks to himself, “I thought last time was the final model; now they are telling me they have another final model. Was a mistake made last time? Why is there a new final model? Will there be another final model?” A better phrase than “the final model” is needed. I use “the best model” because “best” implies that many models were examined and this model was superior to the others. “Best” also allows for the possibility that in the future a better model might be found, such as when the model is updated with new data. The message here is that as scientists we need to be
cognizant that the words we use matter, even the simple ones. The last reason for my belief that the revolution has stalled is that pharmacometricians are too insulated in that they seldom go outside the clinical pharmacology arena with their models. As I was writing this, I did a keyword search on “model-based drug development” in PubMed. A total of 41 citations were retrieved. Every single article was from a journal in clinical pharmacology, pharmaceutics, or pharmacokinetics. It comes as no surprise, then, that when you talk about MBDD to drug developers outside clinical pharmacology, you get blank stares. I understand that when an elegant exposure–response relationship is developed, we want to get credit from our peers who appreciate its beauty, but we should not limit ourselves by preaching to the choir. If pharmacometricians want to truly have influence outside clinical pharmacology, they need to publish in the clinical journals and present at clinical or professional meetings such as those of the American Society of Clinical Oncology and the Drug Information Association, rather than focus on the usual disciplinerelated journals and meetings. I could go on and on. Most people don’t understand models: they don’t know how they work, they don’t know how to evaluate them, they don’t know what makes a good model, etc. There is a general lack of training among biologists and physicians in quantitative methods. Most people—even scientists—have a poor understanding of probability, especially low probability. Models take too long to be developed, and by the time a model is developed the decision has usually already been made. In most companies, pharmacometricians sit at the periphery of the project team and are not central members of the development process.
The list of obstacles to M&S’s widespread acceptance is long, but I think that poor communication skills are at the top. Pharmacometricians (a group to which I obviously belong) need to do better. We need to learn that it isn’t just about the model. Communicating about the model is as important, if not more important, because if you can’t communicate the results of the model to a nontechnical project team, the model is not going to be used. If you learn to be a model communicator (pun intended), you will be a highly sought-after individual within your company and will have tremendous influence. Pharmacometricians also need to be better messengers and to champion M&S outside our field. We need to present success stories at forums other than the usual clinical pharmacology and pharmacokinetics meetings and show how our models improved decision making. Only as pharmacometrics permeates other fields will its influence be truly felt, and MBDD will stop being a dream and become reality. Vive la révolution! CONFLICT OF INTEREST The author declared no conflict of interest. © 2014 ASCPT
1. Lalonde, R. et al. Model-based drug development. Clin. Pharmacol. Ther. 82, 21–32 (2007). 2. Milligan, P.A. et al. Model-based drug development: a rational approach to efficiently accelerate drug development. Clin. Pharmacol. Ther. 93, 502–514 (2013). 3. Cook, J. Is model-based drug development delivering on its promise?: It depends on what the meaning of the word “is” is. American Conference on Pharmacometrics, Fort Lauderdale, FL, 12–15 May 2013. 4. Tornoe, C.W.; the ASCPT PMK Committee. PMX in submission: where are we now? Where should we go? American Society of Clinical Pharmacology and Therapeutics Annual Meeting, Atlanta, GA, 18–22 March 2014. 5. Box, G.E.P. & Draper, N. Empirical Model-Building and Response Surfaces 424 (Wiley, New York, 1987).
Clinical pharmacology & Therapeutics | VOLUME 96 NUMBER 4 | OCTOBER 2014
417
