original articles 23. Ahn MJ, Lee J, Park YH et al. Korean ethnicity as compared with white ethnicity is an independent favorable prognostic factor for overall survival in non-small cell lung cancer before and after the oral epidermal growth factor receptor tyrosine kinase inhibitor era. J Thorac Oncol 2010; 5: 1185–96. 24. Kim JH, Kim SY, Jung KH et al. Randomized phase II study of gemcitabine plus cisplatin versus etoposide plus cisplatin for the treatment of locally advanced or metastatic non-small cell lung cancer: Korean Cancer Study Group experience. Lung Cancer 2006; 52: 75–81. 25. Park JO, Kim SW, Ahn JS et al. Phase III trial of two versus four additional cycles in patients who are nonprogressive after two cycles of platinum-based chemotherapy in non small-cell lung cancer. J Clin Oncol 2007; 25: 5233–9.
Annals of Oncology 26. Okamoto I, Yoshioka H, Takeda K et al. Phase I clinical study of the angiogenesis inhibitor TSU-68 combined with carboplatin and paclitaxel in chemotherapy-naive patients with advanced non-small cell lung cancer. J Thorac Oncol 2012; 7: 427–33. 27. Tsai CM, Au JS, Chang GC et al. Safety and efficacy of first-line bevacizumab with chemotherapy in Asian patients with advanced nonsquamous NSCLC: results from the phase IV MO19390 (SAiL) study. J Thorac Oncol 2011; 6: 1092–7. 28. Zhao X, Mei K, Cai X et al. A randomized phase II study of recombinant human endostatin plus gemcitabine/cisplatin compared with gemcitabine/cisplatin alone as first-line therapy in advanced non-small-cell lung cancer. Invest New Drugs 2012; 30: 1144–9.
Annals of Oncology 25: 536–541, 2014 doi:10.1093/annonc/mdt550 Published online 12 January 2014
T. Grellety1, A. Petit-Monéger2, A. Diallo2, S. Mathoulin-Pelissier2,3,4 & A. Italiano1* 1
Department of Medical Oncology, Institut Bergonié, Regional Comprehensive Cancer Centre, Bordeaux; 2Clinical Research and Clinical Epidemiology Unit, Institut Bergonié, Regional Comprehensive Cancer Centre, Bordeaux; 3INSERM CIC-EC7, ISPED, Bordeaux University, Bordeaux; 4INSERM U897 - Epidemiology and Biostatistics, ISPED, Bordeaux University, Bordeaux, France
Received 22 May 2013; revised 12 August 2013; accepted 18 November 2013
Background: Phase II trials represent an essential step in the development of anticancer drugs. This study assesses the quality of their reporting in highly ranked oncology journals, investigates predictive factors of quality, and proposes reporting guidelines. Patients and methods: We reviewed the table of contents of all volumes of eight peer-reviewed oncology journals published in English between January and December 2011 with a 2011 impact factor (IF) >4. Two reviewers assessed the quality of each report by using a 44-point overall quality score (OQS). Primary end point definition, justification of sample size, and definition of the evaluable population, were assessed separately to establish a 3-point key methodological score (KMS). Exploratory analyses identified predictive factors associated with scores. Results: One hundred fifty-six articles were included. The median OQS was 28 (range: 9–35). OQS subsection analysis showed that reporting of statistical methods was low with a median OQS of 3. Median KMS was 2 (range 0–3). Primary end point definition, justification of sample size and definition of the evaluable population were reported in only 107 (68.6%), 121 (77.6%), and 52 (33.3%) cases, respectively. At multivariate analysis, registration on clinicaltrials.gov and IF >10 were associated with improved OQS. No associations for KMS were observed. Conclusion: Phase II trial reporting is still poor even in journals with strict editorial policies. This may lead to biased interpretation of phase II trial results. Besides using a checklist during the preparation of their manuscript, authors should also provide reviewers and readers with the last version of the study’s protocol. Key words: phase II trials, oncology, quality of reporting, clinical trials
introduction In the field of oncology, many promising new drugs illustrate a new era of cancer drug development based on better knowledge of tumor biology. Phase II trials represent one of the key
*Correspondence to: Dr Antoine Italiano, Department of Medical Oncology, Institut Bergonié, 229 cours de l’Argonne, 33076 Bordeaux Cedex, France. Tel: +33-05-56-3332-44; Fax: +33-05-56-33-33-83; E-mail: [email protected]
elements of this progress. Their main objective is to demonstrate preliminary evidence of efficacy and acceptable toxicity. Accurate reporting of phase II trials is thus crucial. Indeed, such publications may represent the basis to test innovative clinical hypotheses and provide data for the rationale of future phase III, randomized, controlled trials (RCTs). In a study based on 1997 phase II published trials, Mariani et al. [1] showed that only 20% of phase II reports described the statistical design and the results clearly. Strikingly, the authors
© The Author 2014. Published by Oxford University Press on behalf of the European Society for Medical Oncology. All rights reserved. For permissions, please email: [email protected].
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
Quality of reporting of phase II trials: a focus on highly ranked oncology journals
original articles
Annals of Oncology
patients and methods study selection We selected all the peer-reviewed general oncology journals published in English with a 2011 5-year impact factor (IF) >4 [6]. They were Annals of Oncology (Ann Oncol), British Journal of Cancer (Br J Cancer), Clinical Cancer Research (Clin Cancer Res), European Journal of Cancer (Eur J Cancer), Journal of Clinical Oncology (J Clin Oncol), Journal of the National Cancer Institute (J Natl Cancer Inst), Lancet Oncology (Lancet Oncol), and The Oncologist (Oncologist). We reviewed table of contents for all volumes of the journals published between January and December 2011, identifying phase II trials assessing the efficacy and/or tolerance of anticancer drugs, radiotherapy, or surgery, as a single treatment modality or in combination. Appropriate abstracts were selected to identify articles for full-text reading and evaluation. Preclinical, phase I/II, phase II/III and biomarkers studies were excluded from this analysis. We also excluded updated reports from previously published trials, review articles, and studies related to nonmalignant disease or supportive care.
rating of global reporting quality Based on consensus among the authors, 44 items inspired from the revised CONSORT statement for randomized trials [7] and from the Guidelines for Neuro-Oncology: standards for investigational studies (GNOSIS)—reporting of phase I and phase II clinical trials [8] were compiled into a comprehensive list (Appendix 1). All items were categorized across nine subsections (Appendix 1). For each publication, we recorded the frequency of each item. Each item was scored 1 if reported and 0 if not reported or not clearly stated, giving an overall sum (the ‘overall quality score’: OQS) of 0–44.
rating of key methodological items The three items for the key methodological score were selected based on a consensus between the authors. Indeed, definitions of the first end point, definition of the evaluable population for each end point, and justification of sample size were considered as highly important for avoiding interpretation bias and thus distortion in the estimation of the effect. All end points should be identified and completely defined. If relevant, the use or not of previously validated scales or consensus guidelines such as RECIST [9] and of an independent central review [10] must be indicated.
Volume 25 | No. 2 | February 2014
Determination of the number of participants is made to detect clinically relevant treatment effects and must be determined carefully because it enables underlying power calculations. In contrast to phase III trials, ‘intention-to-treat’ analysis is not necessarily required in phase II studies where analysis can be restricted to participants who fulfill the protocol. Whatever the statistical plan used, the population assessable for each end point should be given allowing easy identification of the characteristics of the population to whom the trial results and treatment tested applies. A score of 1 is given for each of the three following variables ‘first end point’, ‘sample size’, and ‘evaluable population’ if clearly provided. The KMS was calculated for each publication by combining the scores of these three factors (range: 0–3).
statistical analysis A standardized data collection form was created to extract data from each article. The data collection form was first tested by four coauthors (TG, APM, SMP, and AI), using a common set of five articles. After finalization and approval of the form, all articles were evaluated by two reviewers (TG and APM). Reproducibility of the results was estimated using the Cohen’s kappa (κ) index for 20 randomly selected articles. The κ coefficient ranged from 0.62 to 1 for all selected items evaluated indicating good agreement between the reviewers. Qualitative variables were described using rates and proportions. The OQS, its 9 subsections scores, and the KMS were described using the median, interquartile range, minimum, and maximum values. Univariate and multivariate regression analyses were carried out to identify factors significantly associated with the OQS (global and by section) and the KMS. The variables included in these analyses were: high (>10) versus moderate/low IF; clinical trial registration in clinicaltrials.gov; type of trial (industry or academic); geographic zone (US A, Europe, transcontinental, or other); population under study (adults only versus adults and children); cancer type (solid tumor or hematological neoplasm); type of treatment (systemic treatment versus other treatment); route of administration (oral or i.v.). Variables significant at a 10% level in the univariate models were included in a multivariate regression model. Variables significant at the 5% level in the final multivariate model were considered as significant predictors. Statistical analyses were carried out using the SAS software, version 9.2 (SAS Institute, Inc., Cary, NC).
results eligible studies characteristics One hundred fifty-eight articles were identified from table of contents. After abstract reading, two were excluded: one reporting on a phase I/II trial and the other on a biomarker-oriented study. A total of 156 articles were thus considered eligible and included in the final analysis (Table 1). The majority of the publications were from two journals: J Clin Oncol and Ann Oncol. No publications were found published in J Natl Cancer Inst. Most publications reported results of studies including adults with solid tumors, managed with i.v. systemic treatments and funded by the pharmaceutical industry. Reports were well balanced in terms of geographical distribution between North America and Europe. Only 27.6% (N = 43) of articles were reported as declared on the clinicaltrials.gov registry, and 28.2% (N = 44) of the trials were randomized.
overall quality rating (OQS) The median OQS was 28 on 44 items with a maximum score of 35 (Table 2). Analysis of the OQS subscores showed that
doi:10.1093/annonc/mdt550 |
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
found that positive findings emerged less frequently from trials reporting statistical design than from those not reporting. The dramatic increase in promising new agents and the large number of institutions involved in clinical research has led to an explosion of phase II trials. However, for most phase II trials, success does not accurately predict for phase III success [2]. This concern has influenced editorial policies of some oncology journals regarding the reporting of phase II trials [3]. Unlike phase III RCT for whom methodological recommendations for publication are available ( Consolidated Standards of Reporting Trials, CONSORT) [4] since the mid-1990s, which have improved reporting of phase III RCT [5], no recommendations are currently available for phase II trials. The aim of this study was to assess the quality of reporting of phase II clinical trials by focusing on publications in highly ranked oncology journals. We also investigate factors associated with better reporting quality and suggest guidelines for authors to improve their reports and to facilitate readers’ interpretation and critical comment of phase II trial results.
original articles
Annals of Oncology
Table 1. Checklist of items Item
Intention to treat Source of analysis RESULTS Dates of inclusion Patients accrued Patient ineligible Median follow-up Reason for discontinuation Table of patient characteristics Flow charts Distribution in analyses Confidence intervals Safety DISCUSSION Conclusion
| Grellety et al.
‘Phase II’ in the title Type of phase II indicated (single-arm, randomized, etc.) Type of intervention mentioned Statement of the disease setting (epidemiology, reference treatment) Scientific rationale with earlier results of the studying treatment explained Drug background (name, trademark name) and mechanisms of action indicated
Quote and definition of the first end point Quote and definition of secondary end points Age limit for eligibility indicated Performance status indicated Indication of the estimated survival Indication of the required levels for laboratory tests Description of the type of disease: histologies and/or specific molecular aberrations Indication of the disease evaluation criteria (RECIST or other…) If ≥2nd line setting, mention of criteria for determining progression at inclusion (radiological review, etc.) Prior treatments received before the study entrance and specified as allowed or not for inclusion Washout period from previous treatments received before the study entrance Co-morbidities that preclude inclusion Informed consent and IRB approval Planned doses of treatment indicated Description of treatment duration or number of cycle planned Criteria for removal of study Description of reason for dose modifications Forbidden concomitant medications or supportive care allowed in the trial
Type and frequency of the radiographic monitoring If relevant, indication of presence of central review imaging for first end point Frequency and type of laboratory monitoring Hypothesis and justification for determination of sample size Definition of rules to consider patients as evaluable for first and secondary end points Rules to define patient as nonassessable for analysis Statistical methods used for analyses of primary and secondary end points (information relevant to phase II study design, probability concerning null and alternative hypothesis) Specify if intention-to-treat analysis is used and assessment of number of patients in accordance with that analysis Statement of source of analysis (authors, institution, or industry) Dates defining accrual specified Number of patients included in analysis Number of patients deemed ineligible Median follow-up of the population reported Description of reasons for discontinuation of treatment Characteristics of patients in tabular form Details on the flow of participants through each stage of the trials (number of patients randomly assigned, receiving intended treatment, completing the protocol, and analyzed) Number of patients included in each analysis Confidence intervals indicated for first and secondary end points Description of all important adverse effects resulting from the intervention Conclusion about the first end point consistent with results provided by the study and with initial hypothesis
Volume 25 | No. 2 | February 2014
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
TITLE Phase II Type of phase II trial Intervention INTRODUCTION: study rationale State of the art Scientific rationale Mechanisms of action of the drug METHODS Study objectives First end point Secondary end point Patient selection Age Performance status Life expectancy Laboratory tests Histology Radiological criteria/RECIST Progression Previous treatments Recovery period Co-morbidities Informed consent Treatment study Dosing Duration (or number of cycles) Removal of study Criteria for dosing adaptation Concomitant medication/ supportive care Study procedures Imaging Centralized review of imaging assessments Biological assessment Statistical methods Sample size Evaluable population Nonassessable Statistical methods
Required detailed criteria
original articles
Annals of Oncology
Table 2. Characteristics of phase II reports published in highranking oncology journals (N = 156) Characteristics
Publications, N (%)
reporting of statistical methods (six items) was particularly low with a median of 3, and only 18.6% of the publications reporting more than three items. About 25% of the articles did not report any study objectives (24.4%). The highest ratio score for subsections was observed for the ‘study rationale’ section (three items) with a mean ratio of 80%. Only 31.8% of the articles published in very high IF journals (N = 66) had a OQG >30.
rating of key methodological items The median KMS was 2 with only 16.1% of the articles reporting all three items and almost a third reporting 10) were associated with improved OQS. These two factors remained independently associated with improved OQS on multivariate analysis {registration on clinicaltrials.gov: OR = 4.3 [95% confidence interval (CI) 1.8–10.1; P = 0.0008], IF >10: OR = 3.2 (95% CI 1.4–7.4; P = 0.006)}. No significant associations for KMS were observed (Table 4).
discussion Quality of reporting of phase II trials in oncology remains poor even in highly ranked journals. As well as describing a low OQS, our results also show that crucial points, mandatory for an accurate interpretation of data by readers, are described in only a fraction of papers. On average, only one of the two ‘end points description’ items and three of the six ‘statistical methods’ items were reported. Although a clear definition of primary (and secondary where appropriate) outcome measures are crucial for the interpretation of phase II trials, this item was reported accurately in only 35% of articles. Our results showed that only 16% of the articles reported the three KMS items. Among them, definition of the evaluable populations for the different end points was the most infrequently reported with only 33% of the papers clearly describing this aspect. This issue is clearly relevant in the area of targeted therapies. Indeed, the majority of such drugs are administered orally every day or several days a month. This raises the question of who is assessable for efficacy analyses. In the context of phase III trials, excluding patients after randomization may introduce noncomparability of characteristics across treatment groups leading to bias and is consequently not recommended. However, in the context of phase II studies, it may be justified, for example, to exclude from efficacy analysis patients who did not start treatment received only a few doses of an oral targeted therapy, or were ineligible because they were mistakenly enrolled. This has to be clearly planned in the study protocol and reported in the publication. Although the best design of phase II studies in oncology is still a matter of debate [11–14], our findings indicate that reporting of design remains to be improved. Reporting of clinicaltrials.gov items was found to be predictive of a good OQS. In 2005, the International Committee of Medical Journal Editors first published its statement requiring registration as a precondition for publication [15]. Such registration is supposed to prevent research misconduct, specifically selective reporting, by putting key protocol information in the public domain. However, the quality of the registered data is not guaranteed. Only 27% of the phase II papers published in 2011 and analyzed in our study indicated the registration of the study through the International Clinical Trials Registry Platform (ICTRP, clinicaltrials.gov) in their publications. Besides clinicaltrials.gov registration reporting, our analysis revealed that higher IF was significantly associated with better reporting of statistical methods and results even if phase II published in very high IF journals remain poorly reported. High IF has also already been found associated with better reporting quality of randomized trials [16, 17]. One explanation for this result may be a more
doi:10.1093/annonc/mdt550 |
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
Journal title Annals of Oncology 39 (25) British Journal of Cancer 17 (10.9) Clinical Cancer Research 20 (12.8) European Journal of Cancer 10 (6.4) Journal of Clinical Oncology 58 (37.2) Journal of National Cancer Institute 0 (0) Lancet Oncology 8 (5.1) The Oncologist 4 (2.6) At least one author affiliated to an industrial sponsor Yes 68 (43.6) No 88 (56.4) Funding from pharmaceutical industry Yes 114 (73.1) No 42 (26.9) Trial reported to be registered on clinicaltrials.gov Yes 43 (27.6) No 113 (72.4) Population Adults 149 (95.5) Children 1 (0.6) Both 6 (3.9) Geographical zone North America 54 (34.6) Europe 57 (36.5) Other or transcontinental 45 (28.9) Type of tumor Solid 135 (86.5) Hematology 21 (13.5) Type of treatment Systemic only 130 (83.3) Radiotherapy 1 (0.6) Combined treatment 25 (16.1) Route of administration Intravenous 53 (34) Oral 42 (26.9) Other/combination 49 (31.4) Not specified 12 (7.7) Impact factor >10 66 (42.3) ≤10 90 (57.7)
exploratory analysis: factors associated with better reporting
original articles
Annals of Oncology
Table 3. Scoring of phase II reports quality (N = 156) Score
No. of items
Mean score (ratio %)
Median (range)
Overall Quality Score Subsections score Title Study rationale Patient selection Treatment study Study procedures Study objectives Statistical methods Results Discussion Key methodological score
44
27.5 (62.5)
28 (9–35)
3 3 11 5 3 2 6 10 1 3
2.2 (73.3) 2.4 (80) 6.9 (62.7) 3.1 (44.3) 1.5 (50) 1.1 (55) 2.5 (41.7) 7.3 (73) 0.3 (30) 1.8 (60)
2 (0–3) 2,5 (0–3) 7 (0–11) 3 (0–5) 2 (0–3) 1 (0–2) 3 (0–5) 7 (3–10) – 2 (0–3)
No. of articles with maximum score (%) 0 (0) 58 (37.2) 78 (50) 1 (0.6) 19 (12.2) 16 (10.3) 56 (35.9) 4 (2.6) 7 (4.5) 47 (30.1) 25 (16)
Items
No. of trials reporting this aspect
%
Primary end point Clear referencing and definition of the first end point Sample size Hypothesis and justification of sample size clearly explained Evaluable population Definition of rules to consider patients as evaluable for first and secondary end points
107
68.6
121
77.6
52
33.3
rigorous peer-review process or simply that better-written papers are more likely to be accepted for publication in higher impact journals. Moreover, a study showed that phase II studies were more likely to be published in low IF journals (IF
Annals of Oncology 26. Okamoto I, Yoshioka H, Takeda K et al. Phase I clinical study of the angiogenesis inhibitor TSU-68 combined with carboplatin and paclitaxel in chemotherapy-naive patients with advanced non-small cell lung cancer. J Thorac Oncol 2012; 7: 427–33. 27. Tsai CM, Au JS, Chang GC et al. Safety and efficacy of first-line bevacizumab with chemotherapy in Asian patients with advanced nonsquamous NSCLC: results from the phase IV MO19390 (SAiL) study. J Thorac Oncol 2011; 6: 1092–7. 28. Zhao X, Mei K, Cai X et al. A randomized phase II study of recombinant human endostatin plus gemcitabine/cisplatin compared with gemcitabine/cisplatin alone as first-line therapy in advanced non-small-cell lung cancer. Invest New Drugs 2012; 30: 1144–9.
Annals of Oncology 25: 536–541, 2014 doi:10.1093/annonc/mdt550 Published online 12 January 2014
T. Grellety1, A. Petit-Monéger2, A. Diallo2, S. Mathoulin-Pelissier2,3,4 & A. Italiano1* 1
Department of Medical Oncology, Institut Bergonié, Regional Comprehensive Cancer Centre, Bordeaux; 2Clinical Research and Clinical Epidemiology Unit, Institut Bergonié, Regional Comprehensive Cancer Centre, Bordeaux; 3INSERM CIC-EC7, ISPED, Bordeaux University, Bordeaux; 4INSERM U897 - Epidemiology and Biostatistics, ISPED, Bordeaux University, Bordeaux, France
Received 22 May 2013; revised 12 August 2013; accepted 18 November 2013
Background: Phase II trials represent an essential step in the development of anticancer drugs. This study assesses the quality of their reporting in highly ranked oncology journals, investigates predictive factors of quality, and proposes reporting guidelines. Patients and methods: We reviewed the table of contents of all volumes of eight peer-reviewed oncology journals published in English between January and December 2011 with a 2011 impact factor (IF) >4. Two reviewers assessed the quality of each report by using a 44-point overall quality score (OQS). Primary end point definition, justification of sample size, and definition of the evaluable population, were assessed separately to establish a 3-point key methodological score (KMS). Exploratory analyses identified predictive factors associated with scores. Results: One hundred fifty-six articles were included. The median OQS was 28 (range: 9–35). OQS subsection analysis showed that reporting of statistical methods was low with a median OQS of 3. Median KMS was 2 (range 0–3). Primary end point definition, justification of sample size and definition of the evaluable population were reported in only 107 (68.6%), 121 (77.6%), and 52 (33.3%) cases, respectively. At multivariate analysis, registration on clinicaltrials.gov and IF >10 were associated with improved OQS. No associations for KMS were observed. Conclusion: Phase II trial reporting is still poor even in journals with strict editorial policies. This may lead to biased interpretation of phase II trial results. Besides using a checklist during the preparation of their manuscript, authors should also provide reviewers and readers with the last version of the study’s protocol. Key words: phase II trials, oncology, quality of reporting, clinical trials
introduction In the field of oncology, many promising new drugs illustrate a new era of cancer drug development based on better knowledge of tumor biology. Phase II trials represent one of the key
*Correspondence to: Dr Antoine Italiano, Department of Medical Oncology, Institut Bergonié, 229 cours de l’Argonne, 33076 Bordeaux Cedex, France. Tel: +33-05-56-3332-44; Fax: +33-05-56-33-33-83; E-mail: [email protected]
elements of this progress. Their main objective is to demonstrate preliminary evidence of efficacy and acceptable toxicity. Accurate reporting of phase II trials is thus crucial. Indeed, such publications may represent the basis to test innovative clinical hypotheses and provide data for the rationale of future phase III, randomized, controlled trials (RCTs). In a study based on 1997 phase II published trials, Mariani et al. [1] showed that only 20% of phase II reports described the statistical design and the results clearly. Strikingly, the authors
© The Author 2014. Published by Oxford University Press on behalf of the European Society for Medical Oncology. All rights reserved. For permissions, please email: [email protected].
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
Quality of reporting of phase II trials: a focus on highly ranked oncology journals
original articles
Annals of Oncology
patients and methods study selection We selected all the peer-reviewed general oncology journals published in English with a 2011 5-year impact factor (IF) >4 [6]. They were Annals of Oncology (Ann Oncol), British Journal of Cancer (Br J Cancer), Clinical Cancer Research (Clin Cancer Res), European Journal of Cancer (Eur J Cancer), Journal of Clinical Oncology (J Clin Oncol), Journal of the National Cancer Institute (J Natl Cancer Inst), Lancet Oncology (Lancet Oncol), and The Oncologist (Oncologist). We reviewed table of contents for all volumes of the journals published between January and December 2011, identifying phase II trials assessing the efficacy and/or tolerance of anticancer drugs, radiotherapy, or surgery, as a single treatment modality or in combination. Appropriate abstracts were selected to identify articles for full-text reading and evaluation. Preclinical, phase I/II, phase II/III and biomarkers studies were excluded from this analysis. We also excluded updated reports from previously published trials, review articles, and studies related to nonmalignant disease or supportive care.
rating of global reporting quality Based on consensus among the authors, 44 items inspired from the revised CONSORT statement for randomized trials [7] and from the Guidelines for Neuro-Oncology: standards for investigational studies (GNOSIS)—reporting of phase I and phase II clinical trials [8] were compiled into a comprehensive list (Appendix 1). All items were categorized across nine subsections (Appendix 1). For each publication, we recorded the frequency of each item. Each item was scored 1 if reported and 0 if not reported or not clearly stated, giving an overall sum (the ‘overall quality score’: OQS) of 0–44.
rating of key methodological items The three items for the key methodological score were selected based on a consensus between the authors. Indeed, definitions of the first end point, definition of the evaluable population for each end point, and justification of sample size were considered as highly important for avoiding interpretation bias and thus distortion in the estimation of the effect. All end points should be identified and completely defined. If relevant, the use or not of previously validated scales or consensus guidelines such as RECIST [9] and of an independent central review [10] must be indicated.
Volume 25 | No. 2 | February 2014
Determination of the number of participants is made to detect clinically relevant treatment effects and must be determined carefully because it enables underlying power calculations. In contrast to phase III trials, ‘intention-to-treat’ analysis is not necessarily required in phase II studies where analysis can be restricted to participants who fulfill the protocol. Whatever the statistical plan used, the population assessable for each end point should be given allowing easy identification of the characteristics of the population to whom the trial results and treatment tested applies. A score of 1 is given for each of the three following variables ‘first end point’, ‘sample size’, and ‘evaluable population’ if clearly provided. The KMS was calculated for each publication by combining the scores of these three factors (range: 0–3).
statistical analysis A standardized data collection form was created to extract data from each article. The data collection form was first tested by four coauthors (TG, APM, SMP, and AI), using a common set of five articles. After finalization and approval of the form, all articles were evaluated by two reviewers (TG and APM). Reproducibility of the results was estimated using the Cohen’s kappa (κ) index for 20 randomly selected articles. The κ coefficient ranged from 0.62 to 1 for all selected items evaluated indicating good agreement between the reviewers. Qualitative variables were described using rates and proportions. The OQS, its 9 subsections scores, and the KMS were described using the median, interquartile range, minimum, and maximum values. Univariate and multivariate regression analyses were carried out to identify factors significantly associated with the OQS (global and by section) and the KMS. The variables included in these analyses were: high (>10) versus moderate/low IF; clinical trial registration in clinicaltrials.gov; type of trial (industry or academic); geographic zone (US A, Europe, transcontinental, or other); population under study (adults only versus adults and children); cancer type (solid tumor or hematological neoplasm); type of treatment (systemic treatment versus other treatment); route of administration (oral or i.v.). Variables significant at a 10% level in the univariate models were included in a multivariate regression model. Variables significant at the 5% level in the final multivariate model were considered as significant predictors. Statistical analyses were carried out using the SAS software, version 9.2 (SAS Institute, Inc., Cary, NC).
results eligible studies characteristics One hundred fifty-eight articles were identified from table of contents. After abstract reading, two were excluded: one reporting on a phase I/II trial and the other on a biomarker-oriented study. A total of 156 articles were thus considered eligible and included in the final analysis (Table 1). The majority of the publications were from two journals: J Clin Oncol and Ann Oncol. No publications were found published in J Natl Cancer Inst. Most publications reported results of studies including adults with solid tumors, managed with i.v. systemic treatments and funded by the pharmaceutical industry. Reports were well balanced in terms of geographical distribution between North America and Europe. Only 27.6% (N = 43) of articles were reported as declared on the clinicaltrials.gov registry, and 28.2% (N = 44) of the trials were randomized.
overall quality rating (OQS) The median OQS was 28 on 44 items with a maximum score of 35 (Table 2). Analysis of the OQS subscores showed that
doi:10.1093/annonc/mdt550 |
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
found that positive findings emerged less frequently from trials reporting statistical design than from those not reporting. The dramatic increase in promising new agents and the large number of institutions involved in clinical research has led to an explosion of phase II trials. However, for most phase II trials, success does not accurately predict for phase III success [2]. This concern has influenced editorial policies of some oncology journals regarding the reporting of phase II trials [3]. Unlike phase III RCT for whom methodological recommendations for publication are available ( Consolidated Standards of Reporting Trials, CONSORT) [4] since the mid-1990s, which have improved reporting of phase III RCT [5], no recommendations are currently available for phase II trials. The aim of this study was to assess the quality of reporting of phase II clinical trials by focusing on publications in highly ranked oncology journals. We also investigate factors associated with better reporting quality and suggest guidelines for authors to improve their reports and to facilitate readers’ interpretation and critical comment of phase II trial results.
original articles
Annals of Oncology
Table 1. Checklist of items Item
Intention to treat Source of analysis RESULTS Dates of inclusion Patients accrued Patient ineligible Median follow-up Reason for discontinuation Table of patient characteristics Flow charts Distribution in analyses Confidence intervals Safety DISCUSSION Conclusion
| Grellety et al.
‘Phase II’ in the title Type of phase II indicated (single-arm, randomized, etc.) Type of intervention mentioned Statement of the disease setting (epidemiology, reference treatment) Scientific rationale with earlier results of the studying treatment explained Drug background (name, trademark name) and mechanisms of action indicated
Quote and definition of the first end point Quote and definition of secondary end points Age limit for eligibility indicated Performance status indicated Indication of the estimated survival Indication of the required levels for laboratory tests Description of the type of disease: histologies and/or specific molecular aberrations Indication of the disease evaluation criteria (RECIST or other…) If ≥2nd line setting, mention of criteria for determining progression at inclusion (radiological review, etc.) Prior treatments received before the study entrance and specified as allowed or not for inclusion Washout period from previous treatments received before the study entrance Co-morbidities that preclude inclusion Informed consent and IRB approval Planned doses of treatment indicated Description of treatment duration or number of cycle planned Criteria for removal of study Description of reason for dose modifications Forbidden concomitant medications or supportive care allowed in the trial
Type and frequency of the radiographic monitoring If relevant, indication of presence of central review imaging for first end point Frequency and type of laboratory monitoring Hypothesis and justification for determination of sample size Definition of rules to consider patients as evaluable for first and secondary end points Rules to define patient as nonassessable for analysis Statistical methods used for analyses of primary and secondary end points (information relevant to phase II study design, probability concerning null and alternative hypothesis) Specify if intention-to-treat analysis is used and assessment of number of patients in accordance with that analysis Statement of source of analysis (authors, institution, or industry) Dates defining accrual specified Number of patients included in analysis Number of patients deemed ineligible Median follow-up of the population reported Description of reasons for discontinuation of treatment Characteristics of patients in tabular form Details on the flow of participants through each stage of the trials (number of patients randomly assigned, receiving intended treatment, completing the protocol, and analyzed) Number of patients included in each analysis Confidence intervals indicated for first and secondary end points Description of all important adverse effects resulting from the intervention Conclusion about the first end point consistent with results provided by the study and with initial hypothesis
Volume 25 | No. 2 | February 2014
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
TITLE Phase II Type of phase II trial Intervention INTRODUCTION: study rationale State of the art Scientific rationale Mechanisms of action of the drug METHODS Study objectives First end point Secondary end point Patient selection Age Performance status Life expectancy Laboratory tests Histology Radiological criteria/RECIST Progression Previous treatments Recovery period Co-morbidities Informed consent Treatment study Dosing Duration (or number of cycles) Removal of study Criteria for dosing adaptation Concomitant medication/ supportive care Study procedures Imaging Centralized review of imaging assessments Biological assessment Statistical methods Sample size Evaluable population Nonassessable Statistical methods
Required detailed criteria
original articles
Annals of Oncology
Table 2. Characteristics of phase II reports published in highranking oncology journals (N = 156) Characteristics
Publications, N (%)
reporting of statistical methods (six items) was particularly low with a median of 3, and only 18.6% of the publications reporting more than three items. About 25% of the articles did not report any study objectives (24.4%). The highest ratio score for subsections was observed for the ‘study rationale’ section (three items) with a mean ratio of 80%. Only 31.8% of the articles published in very high IF journals (N = 66) had a OQG >30.
rating of key methodological items The median KMS was 2 with only 16.1% of the articles reporting all three items and almost a third reporting 10) were associated with improved OQS. These two factors remained independently associated with improved OQS on multivariate analysis {registration on clinicaltrials.gov: OR = 4.3 [95% confidence interval (CI) 1.8–10.1; P = 0.0008], IF >10: OR = 3.2 (95% CI 1.4–7.4; P = 0.006)}. No significant associations for KMS were observed (Table 4).
discussion Quality of reporting of phase II trials in oncology remains poor even in highly ranked journals. As well as describing a low OQS, our results also show that crucial points, mandatory for an accurate interpretation of data by readers, are described in only a fraction of papers. On average, only one of the two ‘end points description’ items and three of the six ‘statistical methods’ items were reported. Although a clear definition of primary (and secondary where appropriate) outcome measures are crucial for the interpretation of phase II trials, this item was reported accurately in only 35% of articles. Our results showed that only 16% of the articles reported the three KMS items. Among them, definition of the evaluable populations for the different end points was the most infrequently reported with only 33% of the papers clearly describing this aspect. This issue is clearly relevant in the area of targeted therapies. Indeed, the majority of such drugs are administered orally every day or several days a month. This raises the question of who is assessable for efficacy analyses. In the context of phase III trials, excluding patients after randomization may introduce noncomparability of characteristics across treatment groups leading to bias and is consequently not recommended. However, in the context of phase II studies, it may be justified, for example, to exclude from efficacy analysis patients who did not start treatment received only a few doses of an oral targeted therapy, or were ineligible because they were mistakenly enrolled. This has to be clearly planned in the study protocol and reported in the publication. Although the best design of phase II studies in oncology is still a matter of debate [11–14], our findings indicate that reporting of design remains to be improved. Reporting of clinicaltrials.gov items was found to be predictive of a good OQS. In 2005, the International Committee of Medical Journal Editors first published its statement requiring registration as a precondition for publication [15]. Such registration is supposed to prevent research misconduct, specifically selective reporting, by putting key protocol information in the public domain. However, the quality of the registered data is not guaranteed. Only 27% of the phase II papers published in 2011 and analyzed in our study indicated the registration of the study through the International Clinical Trials Registry Platform (ICTRP, clinicaltrials.gov) in their publications. Besides clinicaltrials.gov registration reporting, our analysis revealed that higher IF was significantly associated with better reporting of statistical methods and results even if phase II published in very high IF journals remain poorly reported. High IF has also already been found associated with better reporting quality of randomized trials [16, 17]. One explanation for this result may be a more
doi:10.1093/annonc/mdt550 |
Downloaded from http://annonc.oxfordjournals.org/ at East Carolina University on April 26, 2015
Journal title Annals of Oncology 39 (25) British Journal of Cancer 17 (10.9) Clinical Cancer Research 20 (12.8) European Journal of Cancer 10 (6.4) Journal of Clinical Oncology 58 (37.2) Journal of National Cancer Institute 0 (0) Lancet Oncology 8 (5.1) The Oncologist 4 (2.6) At least one author affiliated to an industrial sponsor Yes 68 (43.6) No 88 (56.4) Funding from pharmaceutical industry Yes 114 (73.1) No 42 (26.9) Trial reported to be registered on clinicaltrials.gov Yes 43 (27.6) No 113 (72.4) Population Adults 149 (95.5) Children 1 (0.6) Both 6 (3.9) Geographical zone North America 54 (34.6) Europe 57 (36.5) Other or transcontinental 45 (28.9) Type of tumor Solid 135 (86.5) Hematology 21 (13.5) Type of treatment Systemic only 130 (83.3) Radiotherapy 1 (0.6) Combined treatment 25 (16.1) Route of administration Intravenous 53 (34) Oral 42 (26.9) Other/combination 49 (31.4) Not specified 12 (7.7) Impact factor >10 66 (42.3) ≤10 90 (57.7)
exploratory analysis: factors associated with better reporting
original articles
Annals of Oncology
Table 3. Scoring of phase II reports quality (N = 156) Score
No. of items
Mean score (ratio %)
Median (range)
Overall Quality Score Subsections score Title Study rationale Patient selection Treatment study Study procedures Study objectives Statistical methods Results Discussion Key methodological score
44
27.5 (62.5)
28 (9–35)
3 3 11 5 3 2 6 10 1 3
2.2 (73.3) 2.4 (80) 6.9 (62.7) 3.1 (44.3) 1.5 (50) 1.1 (55) 2.5 (41.7) 7.3 (73) 0.3 (30) 1.8 (60)
2 (0–3) 2,5 (0–3) 7 (0–11) 3 (0–5) 2 (0–3) 1 (0–2) 3 (0–5) 7 (3–10) – 2 (0–3)
No. of articles with maximum score (%) 0 (0) 58 (37.2) 78 (50) 1 (0.6) 19 (12.2) 16 (10.3) 56 (35.9) 4 (2.6) 7 (4.5) 47 (30.1) 25 (16)
Items
No. of trials reporting this aspect
%
Primary end point Clear referencing and definition of the first end point Sample size Hypothesis and justification of sample size clearly explained Evaluable population Definition of rules to consider patients as evaluable for first and secondary end points
107
68.6
121
77.6
52
33.3
rigorous peer-review process or simply that better-written papers are more likely to be accepted for publication in higher impact journals. Moreover, a study showed that phase II studies were more likely to be published in low IF journals (IF
