Supplement For reprint orders, please contact: [email protected]
Pulmonary metastasectomy: a call for better data collection, presentation and analysis
Francesca Fiorentino*,1 & Tom Treasure2
Abstract A systematic review of the literature for outcomes in pulmonary metastasectomy has revealed the variability in reporting and the paucity of data that would allow a clear understanding of the effectiveness of this operation. The authors, a surgeon and a mathematician, start from the experience of assessing the evidence on which the practice of pulmonary metastasectomy is based and give some simple examples on a more adequate approach to the collection and analysis of surgical data and the importance of its correct interpretation. Retrospective data analysis is constrained by the availability of data. While this can give insight on certain aspects, it is important to discern what data are necessary to give a complete understanding of the effectiveness of a practice. Typically well designed prospective studies and randomised controlled trials with a pre-specified data collection plan give more complete, consistent and reliable data than follow-up or retrospective studies. Pulmonary metastasectomy lends itself well as an example of practice based on uncertain evidence and biased reporting. All the available published studies are follow-up studies, there is no randomised controlled trial, so no control data to estimate its treatment effect on patient’s survival. The pool of colorectal or sarcoma patients from which patients are selected to have a pulmonary metastasectomy is never reported on, thus it is hard to estimate the degree of selection and the influence of the surgeon’s decision. As result of systematically reviewing the literature for outcomes in surgical metastasectomy series we became aware of the variability in what was reported and the complete absence of some information that we would have liked to be available in evaluating the effectiveness of these operations [1–3] . We were by no means the first to make this observation. In 1908 Ernest Hey Groves, a surgeon in Bristol, England wrote ‘A plea for a uniform registration of operation results’ which opened as follows: “However much or little value is attached to the statistics of surgical operations by the profession at large or by individual surgeons, one fact is abundantly clear from all the medical literature, and that is that these statistics are employed by all writers in order to summarize their results or to compare them with those of others. Indeed there is no other way possible by which the results of a large number of individual operations can be briefly represented. But when the method of compilation of the figures relating to operations in examined, is it not usually found to be very inadequate?” [4] . A century later, after systematically reviewing the evidence for pulmonary metastasectomy, paper by paper, table by table, graph after graph, for both colorectal cancer and sarcoma [1,3] , we agree with Hey Groves: we need to analyze results for surgical operations but the method of compilation of the published figures is still often inadequate. What Hey Groves wrote about was what we would now call ‘audit.’ It is still good practice to compile data to see if we are achieving the results we expect
Keywords
• data • statistics • surgery
Department of Cardiothoracic Surgery, NHLI, Imperial College London, London, UK Clinical Operational Research Unit, University College London, London, UK *Author for correspondence: [email protected] 1 2
10.2217/FON.14.263 © 2015 Future Medicine Ltd
Future Oncol. (2015) 11(2s), 19–23
part of
ISSN 1479-6694
19
Supplement Fiorentino & Treasure and that our patients deserve. It is at that level our paper is pitched. For prospective studies of disease clinicians should work with epidemiologists and other statistical experts. What we present here are personal views and at times ‘home spun’ tips on how to avoid the common pitfalls in the presentation of surgeons’ own data, even if only for internal audit purposes. Defining terms: data & summary statistics The first term to define (and we will define several others in the course of our essay) is Hey Groves use of the word ‘statistics’. It was not until the 1920s and 1930s that Pearson and Fisher developed means of testing data for ‘significant’ difference as opposed to chance variation. The word ‘statistics’ is now used in the sense of drawing inferences. Used by Hey Groves in 1908 ‘statistics’ meant simply the compilation of data. The comment made in 1910 that politicians ‘use statistics as a drunken man uses lamp posts – for support rather than illumination’ [5] was about giving numbers as evidence. At the time ‘the figures relating to operations’ were what we would call the raw data (used here as a plural noun as in Latin) and ‘the method of compilation of figures’ as the summary statistics such as the median and interquartile range, or the mean and standard deviation. The present authors work together as mathematician and surgeon with a shared interest in the analysis of data to provide more secure evidence for clinical practice. We borrowed Hey Groves’ word ‘plea’ for our contribution to the European Society of Thoracic Surgeon’s Lung Metastasectomy Project [2] . We returned to the subject for the Catania symposium. We will give examples related to lung metastasectomy but the same points will apply to other areas of surgical practice. Collection of data: you can only analyze the data you have The first step is to collect data and straight away we run into a problem. Most published results of lung metastasectomy are in the form of an individual surgeon’s or institution’s follow-up studies. A list of patients who have undergone lung metastasectomy is the usual starting point; the case notes are retrieved and the work of extracting data begins. But the clinical record was not made for research purposes. The age, sex, the number of metastases and the date of the operation will be found but many of the
20
Future Oncol. (2015) 11(2s)
things we might like to know about are not. For example, we might, in hindsight, be interested in the effect on breathing, chest wall pain and quality of life of lung resections in patients who have had a thoracotomy but these measurements are missing [1] even in reports of multiple repeated operations [6] . At the time the original case notes were written the patient would usually have been asymptomatic, fit, and with good lung function. There was no clinical need for these facts, self-evident to the admitting doctor, to be recorded. Without them we can draw no conclusions concerning any gain or loss in the symptomatic well-being of the patients or quality of life. What we can glean is that for sarcoma, for example, a halt is called to repeated pulmonary resections because the patient runs out of breathing capacity [7] . With the wide availability of computers from the 1980s onwards there was the opportunity to collect data. Many units took to recording as much detail as possible into a computer database in the naive belief that in years to come, all they would have to do was to “press the button and the paper will write itself.” In the event many things were entered that were never used in any subsequent analysis and elements of data that would later be considered important were missing or incomplete. Good quality research requires a research question to be considered from the outset. The data necessary to answer the question are recorded meticulously in a purpose made case report form (CRF) and the analysis plan is specified. The completeness, consistency and reliability of the data are reasons why prospective studies are more valued as evidence than follow-up studies [8] . One of the inherent merits of randomized trials is the systematic collection of a uniform dataset, an advantage inherent in all properly planned prospective studies. Registries & research databases serve different purposes To be of value a registry must be comprehensive and include some essential data about every individual within its compass. So a regional cancer registry seeks to record every cancer diagnosis made within its catchment area. Typically the entry will record a limited set of data typically confined to the patient’s dates of birth, diagnosis and death and the cancer site, histological classification and stage. By contrast a research database is confined to one disease, institution or a
future science group
Pulmonary metastasectomy: a call for better data collection, presentation & analysis special interest groups; it is not based on a whole population but is a sample. A research database has many more data fields and usually requires repeated entries over time. If a registry asks too many questions or asks for updates too often, responses will lapse and the registry fails in its prime purpose of ‘registration’ of all cases [9] . Hey Groves’ plea was for ‘uniform registration … by which the results of a large number of individual operations can be briefly represented’. Registries are broad but shallow; research databases are narrow but can be drilled deeper. Compilation of data Once you have your data do not feed them blindly into a software package; first look at them. With your computer spreadsheet you can sort and rank the data and look at their distribution. Run your eye up and down the columns. Are the data grouped toward the middle (distributed equally around the norm, or ‘normally’ distributed) or are they bunched up at one end (a skewed distribution)? Do they cluster in two or more groups revealing that more than one ‘population’ has been captured as we see in the age distribution of pneumothorax? (Figure 1)
Supplement
[10] . The clinician and the analyst should look at them together. In this instance the analyst may spot that there is a bimodal distribution and when it is pointed out the surgeon will know the reason: the patients have all been referred with pneumothorax but they are a mixture of younger patients with primary spontaneous pneumothorax and older patients in whom the pneumothorax is secondary to parenchymal lung disease. By looking at the distribution of the raw data surgeon and analyst can check that they make sense to both of them and consider how they reflect clinical practice. A safe starting point for surgeons is to summarize the data with the median and quartiles or deciles. This will ensure that if the data are skewed the summary statistic will be appropriate. Only if the data have a ‘normal’ distribution may the mean and standard deviation be safely used. Make plots of the data so you can see their patterns rather than collapsing the data into a summary statistic too readily. Analysis packages, like a sports car in the hands of a learner driver, may be too fast and powerful. It may be better to start with a pedestrian approach to data presentation. Like Hey Groves,
25 The management of (spontaneous) pneumothorax
Male Female
Number of entries
20
15
10
5
0 80
Age at surgery (years)
Figure 1. Age distribution of the occurrence of pneumothorax. Reproduced with permission from [10].
future science group
www.futuremedicine.com
21
Supplement Fiorentino & Treasure we would be satisfied with adequate methods of compilation. Inadequacies in the evidence concerning lung metastasectomy ●●There are no control data
The single greatest weakness in the analysis of the results of metastasectomy is the absence of control data. Under some circumstances an intervention is so tightly linked in its timing and mechanistic effect to a beneficial outcome that no more proof is needed [11,12] . Not so when we are dealing with data concerning one or more metastasectomy operations; the time course is years and there are multiple factors in the cancer and its treatment that influence survival. Well informed case selection and effective chemotherapy result in cohorts with longer survival which may have been erroneously attributed to surgery. The true effect of metastasectomy cannot be discerned without careful comparison with similar patients who do not undergo metastasectomy. Randomized controlled trials are the only way to be sure [13] . The unknown & dwindling denominator Patients are selected for metastasectomy, but from among how many? We know that roughly 2–3% of patients with colorectal cancer have a lung metastasectomy [14] . In surgeons’ followup studies we do not have similar data on the other 97–98% who did not have metastasectomy. No reliable inference can be drawn on the effectiveness of metastasectomy. Of those who have a first metastasectomy some have a second or further metastasectomy. Having a second metastasectomy requires survival to that point with progression slow enough to consider operating again. This introduces a biased group of patients with a strong inherent tendency to survive longer. Any conclusion about how the survival in operated patients compares with similar unoperated patients is guesswork, not science. In fact ‘survival’ itself is a confounding factor in analysis. Lack of intention to treat analysis Studies of metastasectomy start with patients who have had metastasectomy. An unknown number of those being worked up for metastasectomy were excluded from the record. Their metastases may have progressed while they were on chemotherapy, other disease was discovered or an R0 resection proved impossible. Their
22
Future Oncol. (2015) 11(2s)
outcomes are part of the full picture of an intention to treat by metastasectomy and they should be included in the reported series. Confusion between prognosis & prediction The number of metastases and how early in the course of the cancer they are seen are both general prognostic factors. They may be associated with survival after metastasectomy, but it is common error of interpretation to believe they are predictive of effectiveness of the metastasectomy [15] . They are general prognostic features. Confusing reporting by complexity of methods It is best for the analyst to have access to raw data. Reports of the larger case series often include multivariate analysis of a mixture of prognostic and predictive factors. The most complete systematic review on survival after lung metastasectomy in colorectal cancer patients provides hazard ratios based on the already opaque output of previous aggregated data [16] . We are told that more than one metastasis, interval between resection of the primary and appearance of metastases under 3 years, an elevated CEA or intrathoracic nodal involvement each approximately doubles the hazard ratio for death. They double the hazard, but from what baseline? There is no estimate available. Absolute numbers are more informative and allow others to use the data to model outcomes in more understandable ways. Conclusion Most of these inadequacies would be resolved with prospective data collection in randomized controlled trials and transparent presentation of all the data on intention to treat [17] . That is what has to be done for cancer drugs. Why not for cancer surgery? However, much of the data in the literature were collected for the internal record and for audit. If in future surgeons intend to go beyond that and regard their data as adding to real scientific knowledge, this will required expertise beyond the scope of this short article. There is a whole other level of epidemiological knowledge and big books setting it out. Our intention is to encourage surgical colleagues presenting their data to be more thoughtful and critical to ensure that when they write up their results they use ‘words that count, numbers that speak’ [18] .
future science group
Pulmonary metastasectomy: a call for better data collection, presentation & analysis Financial & competing interests disclosure The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes
References 1
2
3
Fiorentino F, Hunt I, Teoh K, Treasure T, Utley M. Pulmonary metastasectomy in colorectal cancer: a systematic review and quantitative synthesis. J. R. Soc. Med. 103, 60–66 (2010). Fiorentino F, Treasure T. A plea for consistency in the reporting of surgical series illustrated with an analysis of 51 follow-up reports of pulmonary metastasectomy in colorectal carcinoma. J. Thorac. Oncol. 5, S192–S195 (2010). Treasure T, Fiorentino F, Scarci M, Moller H, Utley M. Pulmonary metastasectomy for sarcoma: a systematic review of reported outcomes in the context of Thames Cancer Registry data. BMJ Open 2, 6 (2012).
4
Hey Groves EW. A plea for a uniform registration of operation results. BMJ 1908(ii), 1008–1009 (1908).
5
Wikiquote. Andrew Lang. http://en.wikiquote.org/wiki/Andrew_Lang
6
Burt BM, Ocejo S, Mery CM et al. Repeated and aggressive pulmonary resections for leiomyosarcoma metastases extends survival. Ann. Thorac. Surg. 92, 1202–1207 (2011).
future science group
7
Supplement
employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending or royalties. No writing assistance was utilized in the production of this manuscript.
Treasure T, Utley M. Surgical removal of asymptomatic pulmonary metastases: time for better evidence. BMJ 346, 21–23 (2013).
14 Embun R, Fiorentino F, Treasure T, Rivas JJ,
Molins L. Pulmonary metastasectomy in colorectal cancer: a prospective study of demography and clinical characteristics of 543 patients in the Spanish colorectal metastasectomy registry (GECMP-CCR). BMJ Open 10(3), 296–302 (2013).
8
Treasure T, Senan S. The need for randomized trials in mesothelioma: Let’s walk the talk. Oncologist 16, 261–263 (2011).
9
Williams WG. Uses and limitations of registry and academic databases. Semin. Thorac. Cardiovasc. Surg. Pediatr. Card. Surg. Annu. 13, 66–70 (2010).
15 Simms L, Barraclough H, Govindan R.
10 Treasure T. Oligometastatic cancer: an entity,
16 Gonzalez M, Poncet A, Combescure C,
a useful concept, or a therapeutic opportunity? J. R. Soc. Med. 105, 242–246 (2012). 11 Glasziou P, Chalmers I, Rawlins M,
McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ 334, 349–351 (2007). 12 Fiorentino F, Treasure T. Pulmonary
metastasectomy: are observational studies sufficient evidence for effectiveness? Ann. Thorac. Surg. 96, 1129–1131 (2013). 13 Godlee F. Colorectal cancer: a cautionary
tale. BMJ Open 348, (2014).
Biostatistics primer: what a clinician ought to know-prognostic and predictive factors. J. Thorac. Oncol. 8, 808–813 (2013). Robert J, Ris HB, Gervaz P. Risk factors for survival after lung metastasectomy in colorectal cancer patients: a systematic review and meta-analysis. Ann. Surg. Oncol. 20, 572–579 (2013). 17 Fiorentino F, Treasure T. Pulmonary
metastasectomy for colorectal cancer: making the case for a randomized controlled trial in the zone of uncertainty. J. Thorac. Cardiovasc. Surg. 146, 748–752 (2013). 18 Treasure W. Diagnosis and Risk Management
in Primary Care: Words That Count, Numbers That Speak. Radcliffe Publishing, London, UK (2011).
www.futuremedicine.com
23
Pulmonary metastasectomy: a call for better data collection, presentation and analysis
Francesca Fiorentino*,1 & Tom Treasure2
Abstract A systematic review of the literature for outcomes in pulmonary metastasectomy has revealed the variability in reporting and the paucity of data that would allow a clear understanding of the effectiveness of this operation. The authors, a surgeon and a mathematician, start from the experience of assessing the evidence on which the practice of pulmonary metastasectomy is based and give some simple examples on a more adequate approach to the collection and analysis of surgical data and the importance of its correct interpretation. Retrospective data analysis is constrained by the availability of data. While this can give insight on certain aspects, it is important to discern what data are necessary to give a complete understanding of the effectiveness of a practice. Typically well designed prospective studies and randomised controlled trials with a pre-specified data collection plan give more complete, consistent and reliable data than follow-up or retrospective studies. Pulmonary metastasectomy lends itself well as an example of practice based on uncertain evidence and biased reporting. All the available published studies are follow-up studies, there is no randomised controlled trial, so no control data to estimate its treatment effect on patient’s survival. The pool of colorectal or sarcoma patients from which patients are selected to have a pulmonary metastasectomy is never reported on, thus it is hard to estimate the degree of selection and the influence of the surgeon’s decision. As result of systematically reviewing the literature for outcomes in surgical metastasectomy series we became aware of the variability in what was reported and the complete absence of some information that we would have liked to be available in evaluating the effectiveness of these operations [1–3] . We were by no means the first to make this observation. In 1908 Ernest Hey Groves, a surgeon in Bristol, England wrote ‘A plea for a uniform registration of operation results’ which opened as follows: “However much or little value is attached to the statistics of surgical operations by the profession at large or by individual surgeons, one fact is abundantly clear from all the medical literature, and that is that these statistics are employed by all writers in order to summarize their results or to compare them with those of others. Indeed there is no other way possible by which the results of a large number of individual operations can be briefly represented. But when the method of compilation of the figures relating to operations in examined, is it not usually found to be very inadequate?” [4] . A century later, after systematically reviewing the evidence for pulmonary metastasectomy, paper by paper, table by table, graph after graph, for both colorectal cancer and sarcoma [1,3] , we agree with Hey Groves: we need to analyze results for surgical operations but the method of compilation of the published figures is still often inadequate. What Hey Groves wrote about was what we would now call ‘audit.’ It is still good practice to compile data to see if we are achieving the results we expect
Keywords
• data • statistics • surgery
Department of Cardiothoracic Surgery, NHLI, Imperial College London, London, UK Clinical Operational Research Unit, University College London, London, UK *Author for correspondence: [email protected] 1 2
10.2217/FON.14.263 © 2015 Future Medicine Ltd
Future Oncol. (2015) 11(2s), 19–23
part of
ISSN 1479-6694
19
Supplement Fiorentino & Treasure and that our patients deserve. It is at that level our paper is pitched. For prospective studies of disease clinicians should work with epidemiologists and other statistical experts. What we present here are personal views and at times ‘home spun’ tips on how to avoid the common pitfalls in the presentation of surgeons’ own data, even if only for internal audit purposes. Defining terms: data & summary statistics The first term to define (and we will define several others in the course of our essay) is Hey Groves use of the word ‘statistics’. It was not until the 1920s and 1930s that Pearson and Fisher developed means of testing data for ‘significant’ difference as opposed to chance variation. The word ‘statistics’ is now used in the sense of drawing inferences. Used by Hey Groves in 1908 ‘statistics’ meant simply the compilation of data. The comment made in 1910 that politicians ‘use statistics as a drunken man uses lamp posts – for support rather than illumination’ [5] was about giving numbers as evidence. At the time ‘the figures relating to operations’ were what we would call the raw data (used here as a plural noun as in Latin) and ‘the method of compilation of figures’ as the summary statistics such as the median and interquartile range, or the mean and standard deviation. The present authors work together as mathematician and surgeon with a shared interest in the analysis of data to provide more secure evidence for clinical practice. We borrowed Hey Groves’ word ‘plea’ for our contribution to the European Society of Thoracic Surgeon’s Lung Metastasectomy Project [2] . We returned to the subject for the Catania symposium. We will give examples related to lung metastasectomy but the same points will apply to other areas of surgical practice. Collection of data: you can only analyze the data you have The first step is to collect data and straight away we run into a problem. Most published results of lung metastasectomy are in the form of an individual surgeon’s or institution’s follow-up studies. A list of patients who have undergone lung metastasectomy is the usual starting point; the case notes are retrieved and the work of extracting data begins. But the clinical record was not made for research purposes. The age, sex, the number of metastases and the date of the operation will be found but many of the
20
Future Oncol. (2015) 11(2s)
things we might like to know about are not. For example, we might, in hindsight, be interested in the effect on breathing, chest wall pain and quality of life of lung resections in patients who have had a thoracotomy but these measurements are missing [1] even in reports of multiple repeated operations [6] . At the time the original case notes were written the patient would usually have been asymptomatic, fit, and with good lung function. There was no clinical need for these facts, self-evident to the admitting doctor, to be recorded. Without them we can draw no conclusions concerning any gain or loss in the symptomatic well-being of the patients or quality of life. What we can glean is that for sarcoma, for example, a halt is called to repeated pulmonary resections because the patient runs out of breathing capacity [7] . With the wide availability of computers from the 1980s onwards there was the opportunity to collect data. Many units took to recording as much detail as possible into a computer database in the naive belief that in years to come, all they would have to do was to “press the button and the paper will write itself.” In the event many things were entered that were never used in any subsequent analysis and elements of data that would later be considered important were missing or incomplete. Good quality research requires a research question to be considered from the outset. The data necessary to answer the question are recorded meticulously in a purpose made case report form (CRF) and the analysis plan is specified. The completeness, consistency and reliability of the data are reasons why prospective studies are more valued as evidence than follow-up studies [8] . One of the inherent merits of randomized trials is the systematic collection of a uniform dataset, an advantage inherent in all properly planned prospective studies. Registries & research databases serve different purposes To be of value a registry must be comprehensive and include some essential data about every individual within its compass. So a regional cancer registry seeks to record every cancer diagnosis made within its catchment area. Typically the entry will record a limited set of data typically confined to the patient’s dates of birth, diagnosis and death and the cancer site, histological classification and stage. By contrast a research database is confined to one disease, institution or a
future science group
Pulmonary metastasectomy: a call for better data collection, presentation & analysis special interest groups; it is not based on a whole population but is a sample. A research database has many more data fields and usually requires repeated entries over time. If a registry asks too many questions or asks for updates too often, responses will lapse and the registry fails in its prime purpose of ‘registration’ of all cases [9] . Hey Groves’ plea was for ‘uniform registration … by which the results of a large number of individual operations can be briefly represented’. Registries are broad but shallow; research databases are narrow but can be drilled deeper. Compilation of data Once you have your data do not feed them blindly into a software package; first look at them. With your computer spreadsheet you can sort and rank the data and look at their distribution. Run your eye up and down the columns. Are the data grouped toward the middle (distributed equally around the norm, or ‘normally’ distributed) or are they bunched up at one end (a skewed distribution)? Do they cluster in two or more groups revealing that more than one ‘population’ has been captured as we see in the age distribution of pneumothorax? (Figure 1)
Supplement
[10] . The clinician and the analyst should look at them together. In this instance the analyst may spot that there is a bimodal distribution and when it is pointed out the surgeon will know the reason: the patients have all been referred with pneumothorax but they are a mixture of younger patients with primary spontaneous pneumothorax and older patients in whom the pneumothorax is secondary to parenchymal lung disease. By looking at the distribution of the raw data surgeon and analyst can check that they make sense to both of them and consider how they reflect clinical practice. A safe starting point for surgeons is to summarize the data with the median and quartiles or deciles. This will ensure that if the data are skewed the summary statistic will be appropriate. Only if the data have a ‘normal’ distribution may the mean and standard deviation be safely used. Make plots of the data so you can see their patterns rather than collapsing the data into a summary statistic too readily. Analysis packages, like a sports car in the hands of a learner driver, may be too fast and powerful. It may be better to start with a pedestrian approach to data presentation. Like Hey Groves,
25 The management of (spontaneous) pneumothorax
Male Female
Number of entries
20
15
10
5
0 80
Age at surgery (years)
Figure 1. Age distribution of the occurrence of pneumothorax. Reproduced with permission from [10].
future science group
www.futuremedicine.com
21
Supplement Fiorentino & Treasure we would be satisfied with adequate methods of compilation. Inadequacies in the evidence concerning lung metastasectomy ●●There are no control data
The single greatest weakness in the analysis of the results of metastasectomy is the absence of control data. Under some circumstances an intervention is so tightly linked in its timing and mechanistic effect to a beneficial outcome that no more proof is needed [11,12] . Not so when we are dealing with data concerning one or more metastasectomy operations; the time course is years and there are multiple factors in the cancer and its treatment that influence survival. Well informed case selection and effective chemotherapy result in cohorts with longer survival which may have been erroneously attributed to surgery. The true effect of metastasectomy cannot be discerned without careful comparison with similar patients who do not undergo metastasectomy. Randomized controlled trials are the only way to be sure [13] . The unknown & dwindling denominator Patients are selected for metastasectomy, but from among how many? We know that roughly 2–3% of patients with colorectal cancer have a lung metastasectomy [14] . In surgeons’ followup studies we do not have similar data on the other 97–98% who did not have metastasectomy. No reliable inference can be drawn on the effectiveness of metastasectomy. Of those who have a first metastasectomy some have a second or further metastasectomy. Having a second metastasectomy requires survival to that point with progression slow enough to consider operating again. This introduces a biased group of patients with a strong inherent tendency to survive longer. Any conclusion about how the survival in operated patients compares with similar unoperated patients is guesswork, not science. In fact ‘survival’ itself is a confounding factor in analysis. Lack of intention to treat analysis Studies of metastasectomy start with patients who have had metastasectomy. An unknown number of those being worked up for metastasectomy were excluded from the record. Their metastases may have progressed while they were on chemotherapy, other disease was discovered or an R0 resection proved impossible. Their
22
Future Oncol. (2015) 11(2s)
outcomes are part of the full picture of an intention to treat by metastasectomy and they should be included in the reported series. Confusion between prognosis & prediction The number of metastases and how early in the course of the cancer they are seen are both general prognostic factors. They may be associated with survival after metastasectomy, but it is common error of interpretation to believe they are predictive of effectiveness of the metastasectomy [15] . They are general prognostic features. Confusing reporting by complexity of methods It is best for the analyst to have access to raw data. Reports of the larger case series often include multivariate analysis of a mixture of prognostic and predictive factors. The most complete systematic review on survival after lung metastasectomy in colorectal cancer patients provides hazard ratios based on the already opaque output of previous aggregated data [16] . We are told that more than one metastasis, interval between resection of the primary and appearance of metastases under 3 years, an elevated CEA or intrathoracic nodal involvement each approximately doubles the hazard ratio for death. They double the hazard, but from what baseline? There is no estimate available. Absolute numbers are more informative and allow others to use the data to model outcomes in more understandable ways. Conclusion Most of these inadequacies would be resolved with prospective data collection in randomized controlled trials and transparent presentation of all the data on intention to treat [17] . That is what has to be done for cancer drugs. Why not for cancer surgery? However, much of the data in the literature were collected for the internal record and for audit. If in future surgeons intend to go beyond that and regard their data as adding to real scientific knowledge, this will required expertise beyond the scope of this short article. There is a whole other level of epidemiological knowledge and big books setting it out. Our intention is to encourage surgical colleagues presenting their data to be more thoughtful and critical to ensure that when they write up their results they use ‘words that count, numbers that speak’ [18] .
future science group
Pulmonary metastasectomy: a call for better data collection, presentation & analysis Financial & competing interests disclosure The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes
References 1
2
3
Fiorentino F, Hunt I, Teoh K, Treasure T, Utley M. Pulmonary metastasectomy in colorectal cancer: a systematic review and quantitative synthesis. J. R. Soc. Med. 103, 60–66 (2010). Fiorentino F, Treasure T. A plea for consistency in the reporting of surgical series illustrated with an analysis of 51 follow-up reports of pulmonary metastasectomy in colorectal carcinoma. J. Thorac. Oncol. 5, S192–S195 (2010). Treasure T, Fiorentino F, Scarci M, Moller H, Utley M. Pulmonary metastasectomy for sarcoma: a systematic review of reported outcomes in the context of Thames Cancer Registry data. BMJ Open 2, 6 (2012).
4
Hey Groves EW. A plea for a uniform registration of operation results. BMJ 1908(ii), 1008–1009 (1908).
5
Wikiquote. Andrew Lang. http://en.wikiquote.org/wiki/Andrew_Lang
6
Burt BM, Ocejo S, Mery CM et al. Repeated and aggressive pulmonary resections for leiomyosarcoma metastases extends survival. Ann. Thorac. Surg. 92, 1202–1207 (2011).
future science group
7
Supplement
employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending or royalties. No writing assistance was utilized in the production of this manuscript.
Treasure T, Utley M. Surgical removal of asymptomatic pulmonary metastases: time for better evidence. BMJ 346, 21–23 (2013).
14 Embun R, Fiorentino F, Treasure T, Rivas JJ,
Molins L. Pulmonary metastasectomy in colorectal cancer: a prospective study of demography and clinical characteristics of 543 patients in the Spanish colorectal metastasectomy registry (GECMP-CCR). BMJ Open 10(3), 296–302 (2013).
8
Treasure T, Senan S. The need for randomized trials in mesothelioma: Let’s walk the talk. Oncologist 16, 261–263 (2011).
9
Williams WG. Uses and limitations of registry and academic databases. Semin. Thorac. Cardiovasc. Surg. Pediatr. Card. Surg. Annu. 13, 66–70 (2010).
15 Simms L, Barraclough H, Govindan R.
10 Treasure T. Oligometastatic cancer: an entity,
16 Gonzalez M, Poncet A, Combescure C,
a useful concept, or a therapeutic opportunity? J. R. Soc. Med. 105, 242–246 (2012). 11 Glasziou P, Chalmers I, Rawlins M,
McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ 334, 349–351 (2007). 12 Fiorentino F, Treasure T. Pulmonary
metastasectomy: are observational studies sufficient evidence for effectiveness? Ann. Thorac. Surg. 96, 1129–1131 (2013). 13 Godlee F. Colorectal cancer: a cautionary
tale. BMJ Open 348, (2014).
Biostatistics primer: what a clinician ought to know-prognostic and predictive factors. J. Thorac. Oncol. 8, 808–813 (2013). Robert J, Ris HB, Gervaz P. Risk factors for survival after lung metastasectomy in colorectal cancer patients: a systematic review and meta-analysis. Ann. Surg. Oncol. 20, 572–579 (2013). 17 Fiorentino F, Treasure T. Pulmonary
metastasectomy for colorectal cancer: making the case for a randomized controlled trial in the zone of uncertainty. J. Thorac. Cardiovasc. Surg. 146, 748–752 (2013). 18 Treasure W. Diagnosis and Risk Management
in Primary Care: Words That Count, Numbers That Speak. Radcliffe Publishing, London, UK (2011).
www.futuremedicine.com
23
