meta-analyses in radiofrequency ablation versus hepatic resection for small hepatocellular carcinoma.

Journal of Evidence-Based Medicine ISSN 1756-5391

REVIEW ARTICLE

A systematic assessment of the quality of systematic reviews/meta-analyses in radiofrequency ablation versus hepatic resection for small hepatocellular carcinoma Yingqiang Wang1,2 , Qianqian Luo3 , Youping Li*1 , Shaolin Deng4,5 , Xianglian Li1 and Shiyou Wei4 1

The Chinese Evidence-based Medicine Center/The Chinese Cochrane Centre, West China Hospital, Sichuan University, Chengdu 610041, China 2 Department of Medical Administration, 363 Hospital, Chengdu 610041, China 3 National Chengdu Center for Safety Evaluation of Drugs, West China Hospital, Sichuan University, Chengdu 610041, China 4 West China Medical School/West China Hospital, Sichuan University, Chengdu 610041, China 5 West China Hospital, Sichuan University, Chengdu 610041, China

Keywords Hepatocellular carcinoma; hepatic resection; meta-analysis; quality assessment; radiofrequency ablation; systematic review. Correspondence Youping Li, The Chinese Evidence-based Medicine Center/The Chinese Cochrane centre, West China Hospital, Sichuan University, No. 37, Guoxue Xiang, Chengdu China, 610041, China. Tel: 86-028-8542-2052; Fax: 86-028-8542-3040; Email: [email protected] Financial Disclosure This study was funded by National Technology Support Program (2011BAI4B01). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. Received 19 March 2014; accepted for publication 1 April 2014.

Abstract Objectives: The systematic reviews (SRs) of radiofrequency ablation (RFA) versus hepatic resection (HR) for early hepatocellular carcinoma (HCC) are increasing with varies qualities. The aim of this study is to evaluate quality and their impacts on outcomes of these studies. Methods: We searched six databases and five official websites to find the SRs of RFA versus HR for early HCC. The Overview Quality Assessment Questionnaire (OQAQ), the Cochrane Collaboration’s tool, and modified MINORS score were applied to assess their quality for SRs, randomized (RCTs) and nonrandomized controlled trials (NRCTs), respectively. Results: Nineteen SRs were included. The results showed that the overall quality was poor, with a mean OQAQ score of 3.3 and 95%CI 2.6 to 4.1, only five (26.3%) SRs were good quality, six (31.6%) misused the statistical models, and three of them changed outcome direction after modification. Five SRs taken retrospective studies as RCT. In addition, a total of 39 primary studies referenced by these 19 SRs were included. The results showed that 3 RCTs were leveled grade B, and 35 NRCTs were of moderate quality, with an estimated mean MINORS score of 15.0 and 95%CI 14.6 to 15.4. Conclusions: The overall quality of SRs comparing the effects between RFA and HR for early HCC was poor. There was high heterogeneity and low evidence level. Physicians should take caution when applying the results from these studies to their clinical practice.

doi: 10.1111/jebm.12100

Introduction Currently, the global burden of cancer grows annually, with 10.9 million new cases of cancer and 67 million cancerrelated deaths each year (1). In 2008, an estimated 169.3 million years of healthy life were dead because of cancer worldwide. Seventy-five per cent of the overall burden of disability-adjusted life years (DALYs) was contributed by

Asia and Europe, with 25% of the total coming from China and 11% from India (2). Liver cancer ranked first among diseases in the burden of DALYs from cancer in male patients in 37 countries, which accounted 28% of the total worldwide (2). Liver cancer is the seventh most common cancer in the world, ranking fifth in males and seventh in females (3), and is also the third most common cause of death from cancer worldwide, with an overall ratio of mortality to incidence

C 2014 Chinese Cochrane Center, West China Hospital of Sichuan University and Wiley Publishing Asia Pty Ltd JEBM 7 (2014) 103–120

103

Quality assessment for small HCC

of 0.93 (4). The developing countries contributed 85% of the cases of liver cancer, with a ratio of male to female of 2.4 (3, 4). The incidence rate and mortality of liver cancer in China were 25.7 and 23.7 per 100,000 people, which rank fourth and third among cancers, respectively (4). The American Association for Study of Liver Disease (AASLD) (5), the European Association for Study of the Liver (EASL), and the European Organization for Research and Treatment of Cancer (EORTC) (1) guidelines recommend hepatic resection (HR) and liver transplantation for early stage hepatocellular carcinoma (HCC), in which the five-year survival rate can reach 50% to 70%. However, only 20% to 35% of HCC patients were suitable for hepatectomy due to poor hepatic reserve as well as inconspicuous symptoms and a low diagnosis rate in the early stage (6). In western countries, only 5% of patients with noncirrhotic HCC chose surgical therapy, while this group in Asia accounted for 40% of patients. However, the tumor recurrence rate, including dissemination and new tumors, at five years after resection exceeded 70% (7). Liver transplantation was only used as a supplementary treatment in China because of limited liver resources and high cost, as well as strict inclusion criteria (i.e., Milan criteria, UCSF criteria, Hangzhou criteria or Chengdu criteria) (7). Radiofrequency ablation (RFA) is considered as one of the best choices for patients with early HCC who are not suitable for hepatic resection or liver transplantation (8). For patients whose conditions are suitable for surgery or liver transplantation, RFA is also deemed as a safe and effective method (9). Recently, a randomized controlled trial (RCT) indicated that RFA was similar in effectiveness to HR for treatment of small HCC with tumor size less than 4 cm and up to two lesions (10). Wang et al. (11) also showed that there was no significant difference in overall survival between RFA and HR for HCC patients in Barcelona Clinic Liver Cancer (BCLC) very early/early stage, but that HR yielded better disease-free survival (DFS). A retrospective control study conducted by Peng et al. (12) demonstrated that RFA was superior to HR both in efficacy and safety for patients with a single HCC and a tumor smaller than 2 cm, especially for the central HCC. Most current meta-analyses have compared the effectiveness of RFA and HR, but it is still difficult to draw a consistent conclusion because of the variable quality among the included studies and the low evidence level (13–18). Therefore, choosing the best treatment for patients with early and resectable HCC is still controversial. The purpose of this study was to systematically search the currently available best evidence, make an evidence-based evaluation of the methodological quality of available systematic reviews/meta-analyses comparing the therapeutic effects between RFA and HR for early HCC and their impacts on outcome, and provide a reference for clinicians or patients in selecting the best clinical protocols.

104

Y. Wang et al.

Material and Methods Inclusion criteria Articles were included according to the elements of PICOS (19): Population: HCC patients met the Milan criteria (solitary HCC smaller than 5 cm in diameter or up to three nodules that were each smaller than 3 cm in diameter) or the UCSF criteria (single HCC smaller than 6.5 cm in diameter or up to three tumors that were each smaller than 4.5 cm and 8 cm in total diameter) with good liver function (Child-Pugh Class A or B), and without radiological evidence of invasion into the major portal/hepatic vein branches, no lymphatic or extrahepatic metastasis, and no previous treatment of HCC. Intervention: Radiofrequency ablation alone. Comparison: Hepatic resection alone. Outcome: O1 : Efficacy (Overall survival (OS) at one, three, and five years (Critical); Recurrence-free survival (RFS) at one, three, and five years (Critical); Disease-free survival (DFS) at one, three, and five years (Critical)). O2 : Safety (hospital mortality (Critical); Recurrence at one, three, and five years (Critical); Complication rate (Important but not critical)). Study design: Health technology assessment (HTA), Systematic review (SR), Meta-analysis and their included primary studies.

Exclusion criteria Conference abstracts, common overviews, and letters were excluded. Those with liver metastases from colorectal carcinoma or other positions, HCC recurrence after hepatectomy or unresectable HCC, as well as those mixed with other adjuvant therapy in either intervention group or control group (i.e., TACE) or follow-up period less than three years were also excluded.

Data sources and search strategy The electronic databases of PubMed, Web of Science, the Cochrane Library of Systematic Reviews, Chinese BioMedical Literature Database (CBM), Chinese National Knowledge Infrastructure (CNKI) and VIP were systematically searched, as well as the official websites of INAHTA (http://www.inahta.net/), HTAi (http://www. htai.org), OMHALTC (http://www.health.gov.on.ca/en/), ICES (http://www.ices.on.ca/index.html), and NIM (http://www.nlm.nih.gov/) through November 2012. The following Mesh search headings were used: (hepatic resection OR surgical resection OR liver resection OR hepatectomy) AND (radiofrequency OR radiofrequency ablation OR catheter ablation OR RFA). The type of literature was limited to health technology assessments,


Y. Wang et al.


systematic reviews, meta-analyses or literature reviews. There was no restriction of language.

Table 1 The secondary studies and their included primary ones in 2004 to 2012

Review selection and data extraction

Year

Metaanalysis

Systematic review

Health technology assessment

Primary study

2004 2005 2006 2007 2008 2009 2010 2011 2012 Total

0 0 0 0 1 2 3 5 4 15

0 0 0 0 0 1 0 2 0 3

0 0 0 0 0 0 1 0 0 1

2 5 3 7 8 6 4 4 0 39

We followed the PRISMA statement to search and select the literature (19). Two reviewers (Yingqiang Wang and Qianqian Luo) independently selected articles by browsing title and abstract based on predefined inclusion and exclusion criteria. If necessary, judgment was made by viewing the full text. For literature that met the inclusion criteria, data were extracted by two independent reviewers (Yingqiang Wang and Qianqian Luo) using standardized forms. Discrepancies between the two reviewers were resolved by discussion or with the third person (Youping Li). The following data were extracted: the first author, year of publication, study design, searched period, inclusion criteria, follow-up, and clinical outcomes.

Quality assessment The Overview Quality Assessment Questionnaire (OQAQ) (20, 21) was applied for assessing the methodological quality of a SR/meta-analysis by two independent reviewers (Yingqiang Wang and Qianqian Luo); with disagreements resolved by discussion or by resort to a third reviewer (Youping Li) if consensus could not be reached. There are 10 items in the OQAQ checklist, with 9 individual items relating to the methodological quality of the SR/meta-analysis, which can be scored by selecting either yes, partial/can’t tell, or no. The tenth item requires reviewers to give an overall quality score on a 7-point scale based on answers to the previous 9 items, with 7 showing no flaws (Exemplary), and a score of 5 indicating that the study has only minimal or minor flaws (Good). If the study is scored 3 to 4, it shows major flaws (Poor), and a score of 2 indicates extensive flaws (Very poor). The full details of the OQAQ scoring questionnaire are available as Supplement Material 1 (Table S1, Table S2). The Modified MINORS score questionnaire (22) was performed to evaluate the quality of nonrandomized controlled trials (NRCTs) referenced by the SRs/meta-analyses. The Risk of Bias Tool in the Cochrane Handbook for Systematic Reviews of Interventions (23) was applied to assess the quality of RCTs including randomization, allocation concealment, blinding, incomplete outcome, selective reporting, and other biases. The full details of this assessment are available as Supplement Material 1 (Table S3, Table S4).

The other quantitative data was analyzed by SPSS 13.0 software (SPSS, Chicago, IL, USA), mainly using frequency, mean and 95% CI. The fixed-effect model (Mantel-Haenszel) was used when the result of the heterogeneity test was P > 0.1 and I2 < 50%. Otherwise, the random-effect model was applied. P < 0.05 was considered as significant.

Results Literature search and selection After initial screening, 59 SRs/meta-analyses were identified. Of them, 12 studies focused on liver metastases from colorectal carcinoma or breast cancer, 12 only compared the therapeutic effectiveness of RFA with other interventions (i.e., percutaneous ethanol injection [PEI] and Transcatheter arterial chemoembolization [TACE]), four abstracts or fulltexts were not found, two were written in Spanish, one analyzed a Markov model, one showed a similar outcome with another one with low quality. These 32 studies were all excluded. In addition, eight studies were excluded for being only overviews or analyzing HR and RFA combined with other therapies. Thus, a total of 19 studies published between 2008 and 2012 were finally included (13–18, 24–35) (Figure 1). There were 1 HTA (18), 3 qualitative SRs (24–26), and 15 meta-analyses (13–17, 27–35). In addition, 39 primary studies referenced by the previous 19 studies published from 2004 to 2011 were also included. They were 1 comment (36), 3 RCTs (37–39), and 35 NRCTs (6, 9, 39–71). The characteristics of the included studies are shown in Table 1.

Review characteristics Statistical analysis Pooled odds ratios (ORs) with 95% confidence intervals (CIs) in the SRs/meta-analyses were standardized and integrated by Stata 10.0 software (Stata Co., College Station, TX, USA).

Within the 19 secondary studies, 4 included only RCTs (14, 27, 29, 34) and 15 included both RCTs and NRCTs. Only 8 of 19 (42.1%) evaluated the quality of included studies (13–16, 27, 29, 34, 35), mainly using the Jadad score (72)


105

106

Milan

NS

1991 to 2011

Up to 2011.3

2006.1 to 2009.2

–

NS

Sun (2011) (16)a

Li (2012) (17)a

Xie (2010) (18)c

Tiong (2011) (24)a Minami (2011) (25)b

Early stage HCC 5 cm (Child-Pugh A / B)

Milan, Child-Pugh A/B

Milan, Child-Pugh A/B; follow >3 year

Milan, liver cirrhosis Milan, Child-Pugh A/B; follow up >12 mo

1995.1 to 2007.12 1990.1 to 2008.12

Liu (2010) (14)a Liu (2010) (15)a

Early HCC

Included criteria

Up to 2011.12

Search period

Xu (2012) (13)a

Author (year)

NS

NS

NS

NRCT (4) RCT (1);

Cohort (5) Comparative (8) Prospective (1); RCT (2); Retrospective (8)

NS

£

B (2) , C (9)

7 (7) 9 (3)

NRCT (9) RCT(2);

Retrospective (7) RCT (2);

Cohort (3);

3 (1), 4 (2)d , 3 ( 8)d

NRCT (11) RCT (8)

1 (9), 2 (1),

Quality evaluation

RCT (2);

Type (No.)

Included study

NS

NS

NS

Exist

No

No

No

No

Publication biasf

Table 2 Baseline characteristic of the included meta-analyses or systematic reviews

928 vs.708 2046 vs. 1927

485 vs. 430

441 vs. 436

1459 vs. 1506

654 vs. 534 787 vs. 735

1302 vs. 1233

RFA vs. HR

Treatment

–

–

–

–

375/76§

–

HR

–

–

–

381/37/0

1414/81/1 1223/245/11

405/54/0

–

364/209§

–

RFA

Child-Pugh A/B/C

NS

NS

59.4 (53.9, 65.0)

62.4 (57.9, 66.9)

57.2 (53.3, 61.0)

NS

62.9 (59.6, 66.2)

63.1 (60.4, 65.7)

Age Mean (95%CI)(year)

3.4 (1.8, 5.0)

NS

3.0 (2.4, 3.5)

3.0 (2.4, 3.6)

NS

NS

3.3 (2.4, 4.1)

2.4 (2.2, 27)

Mean TumorSize (95%CI)(cm)

NS

NS

(Continued)

28.1 (24.7, 31.4)

33.8 (29.8, 37.8)

29.1 (24.6, 33.6)

NS

NS

NS

Mean follow-up (95%CI) (month.)

Quality assessment for small HCC Y. Wang et al.


CCT (6) RCT (2)

UCSF; Child-P A/B

NS

Milan, Child-P A/B

1966 to 2011

2000.1 to 2009.12

1990.1 to 2011.2 Up to 2010.4

–

1997.1 to 2009.11

1979.1 to 2008.8 1996 to 2008

You (2012) (28)a

Zhou (2011) (29)a Wang (2011) (30)a

Gravante (2011) (31)a

Zhou (2010) (32)a

Shi (2009) (33)a Du (2009) (34)a

Milan, Child-Pugh A/B; follow-up rate 95% HCC 5 cm, No. of tumor 3

Milan

Milan, Child-Pugh A/B Milan, Child-Pugh A/B; follow-up >1 year


B (2)

NS

NS

CC (8) RCT (1); NRCT (9)

NS

£

4 (2), 5 (1), 6 (1)d NS

NS

A (2), B £ (7)

RCT (2);

CCT/ Cohort (12)

NRCT (9) RCT (4)

RCT (6);

NRCT (4) RCT (9)

NS

RCT (1);

Huang (2012) (27)a

Milan, Resectable HCC

1997.1 to 2008.4

Quality evaluation

Type (No.)

Lau (2009) (26)b

Included criteria

Search period

Included study

Author (year)

Table 2 Continued

NS

No

NS

NS

No

No

NS

NS

NS

Publication biasf

403 vs. 333 122 vs. 144

744 vs. 667

3609 vs.3775

252 vs. 287 841 vs. 798

1213 vs. 1071

3829 vs. 3984

353 vs. 256

RFA vs. HR

Treatment

–

–

596/148/0

–

747/88/0

–

–

–

–

RFA

–

–

606/67/0

–

772/26/0

–

–

–

–

HR

Child-Pugh A/B/C

NS

NS

62.6 (59.2, 66.0)

61.9 (58.1, 65.8)

56.3 (53.5, 59.2)

56.4 (50.5, 62.2)

NS

NS

NS


NS

NS

3.0 (2.6, 3.4)

NS

NS

(Continued)

26.3 (22.9, 29.7)

NS

NS

5 cm

3.0 (2.0, 3.9)

23.8 (18.0, 29.7)

NS

NS

NS

NS

NS

6 cm

5 cm



Y. Wang et al. Quality assessment for small HCC

107

108

1966 to 2008

1995.1 to 2011.2

Chen (2008) (35)a

Cho (2011) (76)b

Milan

Milan, Child A/B; Follow-up rate > 95%

Included criteria

NS

CC (2); Cohort (3) RCT (2); Retrospective (6)

NS

6–8 (6)d

RCT (1);

NS

Publication biasf

Quality evaluation

Type (No.)

Included study

522 vs. 533

359 vs. 339

RFA vs. HR

Treatment

–

316/40/0

RFA

–

328/11/0

HR

Child-Pugh A/B/C

NRCT, nonrandomized controlled trial; CC, case-control; CCT, clinical control trial; RCT, randomized controlled trial; NS, not state. a Meta-analysis. b Systematic review. c Health technology assessment. £Grading of Cochrane quality evaluation (No. of studies). d Jadad score (No. of studies). e NRCT quality evaluation standard; UCSF: single up to 6.5 cm; or with up to 3 lesions, no larger than 4.5 cm. f Funnel plot evaluated by Egger’s and Begg’s test. § Child-Pugh A/B-C.

Search period

Author (year)

Table 2 Continued

NS

58.4 (53.9, 62.8)


5 cm

2.7 (2.3, 3.2)


NS

24.7 (22.2, 27.2)




Y. Wang et al.


Figure 1 PRISMA flowchart of searching and selecting guidelines.

or Risk of Bias Tool in the Cochrane Handbook for Systematic Reviews of Interventions (23). Eight (42.1%) reported the publication bias (13–17, 29, 30, 33). Only three (15.8%) studies fully reported the baseline characteristics of included patients (i.e., liver function, age, mean tumor size,

or follow-up period, etc.) (17, 32, 35). Most of these studies included patients with good liver function (Child-Pugh Class A or B), except for Sun et al. (16) who reported 11 and 1 patients with Child-Pugh Class C in the RFA group and HR group, respectively. The mean age ranged from


109


56.3 to 63.1 years, the mean tumor size was 3 cm, and the mean duration of follow-up ranged from 23.8 to 33.8 months (Table 2). When we analyzed the study designs of the primary studies referenced by the SRs/Meta-analyses, we found that only five studies were referenced by more than 10 SRs/Metaanalyses (9, 37, 40, 41, 45), while 11 were included by 5 to 9 secondary studies. Twelve primary studies were referenced by different SRs/meta-analyses, but their categories of study design varied. For instance, the study of Huang et al. (47) was included by Huang et al. (27) as an RCT, but Xu et al. (13) and Sun et al. (16) referenced it as an NRCT. Similar circumstances also occurred in other studies (9, 41, 44, 45, 51–54, 62). As another example, retrospective studies were included as RCTs by the authors of five SRs/Meta-analyses (39, 73).

Quality assessment of included studies As shown in Table 3, the overall quality of the 19 SRs/metaanalyses was found to be poor, with an estimated mean overall OQAQ score of 3.3 (95%CI, 2.6 to 4.1). Only 5 (26.3%) were considered as good quality, with minimal or minor flaws (score 5). Eight (42.1%) were thought to be of poor quality, with major flaws (scores of 3 to 4), while the remaining six studies were of very poor quality, with extensive flaws (scores 2). Six (31.6%) studies misused the fixed-effects model when the result of the heterogeneity test was P < 0.1 and I2 > 50%, and three of these studies changed the previous outcome direction after modification by the random-effects model. Table 4 shows the overall quality of the total 39 primary studies referenced by the 19 SRs/meta-analyses (Additional file 2, Table S2, Figure S1 and Figure S2). The quality of three RCTs were grade B because of the absence of blinding (37, 38, 73), and 35 NRCTs were found to be of moderate quality, with an estimated mean MINORS score (of a total possible score of 18) of 15.0 (95%CI 14.6 to 15.4). Only 13 (37.1%) were scored 16. In addition, one single article was a comment written in French, and its quality could not be assessed. Among the 39 primary studies, 17 (43.6%) did not meet the inclusion criteria of these SRs/meta-analyses (36, 39, 42, 48– 50, 55, 60, 63–67, 69–71, 73). Nine were mixed with other effective interventions in both groups (i.e., TACE, PEI). For example, Tashiro et al. (69), Takahashi et al. (66), and Yamakado et al. (63) all included patients who accepted RFA combined with TACE in the RFA group, while Wakai et al. (65), Peng et al. (60), and Chen et al. (39) included patients who accepted PEI therapy in the RFA group. The other four studies included patients with nonprimary HCC, of which 3 focused on the recurrence of HCC (48, 49, 64), one included 20 cases with liver metastases (70), and two

110

Y. Wang et al.

did not describe the baseline characteristics in detail (42, 71) (Table 4).

Clinical outcome Overall survival (OS) Twelve meta-analyses reported that OS at one year was more than 90% in the two groups. Only two showed that RFA was inferior to HR, with a pooled OR of 0.51 (95%CI, 0.29 to 0.86) and 0.60 (95%CI, 0.42 to 0.86), respectively. The other 10 studies showed no significant differences between the two groups (Figure 2). Thirteen studies reported OS at three years, which ranged from 62.5% to 77.9% in the RFA group and from 63.6% to 82.9% in the HR group. Of these, eight indicated that RFA was inferior to HR (P < 0.05), but six presented varying degrees of heterogeneity, with I2 ranging from 46.3% to 64.0%. Liu et al. misused the fixed-effect model when I2 = 53% (14). After modification for the random-effects model, there was no significant difference between the two groups (OR = 0.65, 95%CI 0.4 to 1.06, P = 0.08) (Figure 2). Seven studies showed OS at five years, which ranged from 41.3% to 58.5% in the RFA group and from 51.9% to 65.9% in the HR group. Of these, five reported that RFA was inferior to HR, but two of these showed high heterogeneity. Xu et al. (13) misused the fixed-effect model when I2 = 63.7%. Unfortunately, we could not modify his outcome because of lack of primary data (Figure 2).

Disease-free survival (DFS) Seven reported DFS at one year, which ranged from 54.3% to 93.0% in the RFA group and from 80.1% to 94.8% in the HR group. Of these, four showed that RFA was inferior to HR, with a pooled OR ranging from 0.54 to 0.8. However, Sun et al. (16) and Zhou et al. (32) showed evidence of high heterogeneity, with I2 equal to 57% and 59.2%, respectively (Figure 3). Seven reported DFS at three years, which ranged from 34.9% to 59% in the RFA group and from 45.1% to 73.6% in the HR group. Except for the results of Du et al. (34), all meta-analyses indicated that RFA was inferior to HR, but of these, five presented varying degrees of heterogeneity. Huang et al. (27) misused the fixed-effect model. After modifying for the random-effects model, there was no significant difference between the two groups (OR = 0.95, 95%CI 0.46 to 1.96, P = 0.88). Zhou et al. (29) also showed no significant difference between the two groups after modification (OR = 0.60, 95%CI 0.3 to 1.2, P = 0.15) (Figure 3). Only two meta-analyses reported DFS at five years; both showed that RFA was inferior to HR, with pooled ORs of 0.52 (95%CI 0.42 to 0.63) and 0.64 (95%CI 0.42 to 0.99), respectively (Figure 3).


Y Y Y Y Y Y Y N Y Y Y Y Y Y Y Y Y Y Y Y (18); N (1)

Xu (2012) (13) Liu (2010) (14) Liu (2010) (15) Sun (2011) (16) Li (2012) (17) Xie (2010) (18) Tiong (2011) (24) Minami (2011) (25) Lau (2009) (26) Huang (2012) (27) You (2012) (28) Zhou (2011) (29) Wang (2011) (30) Gravante (2011) (31) Zhou (2010) (32) Shi (2009) (33) Du (2009) (34) Chen (2008) (35) Cho (2011) (76) Total/Meana

Y Y Y Y Y Y Y N Y Y Y Y Y Y Y Y Y Y Y Y (18); N (1)

Q2 Y Y Y Y Y Y Y N N Y Y Y Y Y Y Y Y Y Y Y (17); N (2)

Q3 Y Y Y Y Y Y Y Can’t tell Can’t tell Y Y Y Y Y Y Y Y Y Y Y (17); Can’t tell (2)

Q4 Y Y Y Y Y N N N N Y N Y Partially Y N Partially Y Y N Y (10); N (7); Partially (2)

Q5 Partially Partially Partially Partially Partially N N N N Partially N Partially N Partially N N N Y N Y (1); N (10); Partially (8)

Q6 Y Y Y Y Y Y N N Y Y Y Y Y Y Y Y Y Y Y Y (17) N (2)

Q7 a

Partially Partiallya Y Y Y Y N Can’t tell Y Partiallya Partiallya Partiallya Y Y Y Partiallya Y Y Can’t tell Y (10); N (1); Can’t tell (2); Partially (6)

Q8

Y = yes; N = no. a Use fixed model when I2 > 50%, P < 0.1. Q1: Were the search methods used to find evidence on the primary question(s) stated? Q2: Was the search for evidence reasonably comprehensive? Q3: Were the criteria used for deciding which studies to include in the overview reported? Q4: Was bias in the selection of studies avoided? Q5: Were the criteria used for assessing the validity of the included studies reported? Q6: Was the validity of all the studies referred to in the text assessed using appropriate criteria? Q7: Were the methods used to combine the findings of the relevant (to reach a conclusion) reported? Q8: Were the findings of the relevant studies combined appropriately relative to the primary question of the overview? Q9: Were the conclusions made by the author(s) supported by the data and/or analysis reported in the overview? Q10: What was the overall scientific quality of the overview?

Q1

Author

Table 3 Quality assessment of included studies with OQAQ instrument

Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y (19)

Q9 4 4 5 5 5 2 1 1 1 4 3 4 3 5 3 2 3 6 2 3.3a

Q10

Major (Poor) Major (Poor) Minor (Good) Minor (Good) Minor (Good) Extensive (very Poor) Extensive (very Poor) Extensive (very Poor) Extensive (very Poor) Major (Poor) Major (Poor) Major (Poor) Major (Poor) Minor (Good) Major (Poor) Extensive (very Poor) Major (Poor) Minor (Good) Extensive (very Poor) Good (5); Poor (8); Very poor (6)

Flaws (Quality)

Y. Wang et al. Quality assessment for small HCC


111

112

•

• •

• •

•

•

•

•

•

¶

¶

¶

¶ ¶ £

£

¶ £

¶

•

•

• •

£

£

£

£

•

£

¶

¶

¶

¶

¶

¶

¶

• •

¶

Xu Liu Liu Sun Li Xie Tiong Minami (2012) (2010) (2010) (2011) (2012) (2010) (2011) (2011) (13) (14) (15) (16) (17) (18) (24) (25)

•

•

• •

•

•

•

•

•

•

•

•

• •

NS

NS

NS NS

NS

NS NS

NS NS

NS NS

NS

* *

*

* *

* *

• *

Lau Huang You Zhou Wang Gravante (2009) (2012) (2012) (2011) (2011) (2011) (26) (27) (28) (29) (30) (31)

•

•

NS

NS

NS

NS NS NS

•

•

*

£

£

* £

•

¶

¶

¶ ¶

¶

¶

•

×1 ×11 ×1 ×14 ×5 ×3 ×11 ×12 ×1 ×1 ×2 ×12 ×1 ×3 ×1 ×2 ×2 ×8 ×6 ×5 ×5 ×1 ×1 ×1 ×5 ×7 ×3 ×7 ×8 ×1 ×1 ×2 ×3 ×1 ×5 ×1 ×1 ×1 ×5

¶ • ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶ ¶

¶

16 15 B B 14 17 15 11 16 15 15 15 16 16 16 13 15 14 16 15 13 17 16 16 16 15 15 16 15 16 15 14 15 14 15 13 14 B

Zhou Shi Du Chen Cho (2010) (2009) (2009) (2008) (2011) (32) (33) (34) (35) (76) Total Type quality

Author judgment

NS, not stated; A, high risk of bias; B, moderate risk of bias; C: low risk of bias; •, RCT; ◦, NRCT (£, cohort study; *, case-control; ¶, retrospective; or , prospective); #, comment written in French.

Kobayashi (2009) (6) Guglielmi (2008) (6) Chagnon (2007) (36) Chen (2006) (37) Huang (2010) (38) Chen (2005) (39) Cho (2005) (40) Vivarelli (2004) (41) Li (2004) (42) Guo (20100) (43) Hasegawa (2008) (44) Hong (2005) (45) Bu (2009) (46) Huang (2011) (47) Shen (2007) (48) Ren (2008) (49) Zhang (2007) (50) Montorsi (2005) (51) Abu-Hila (2008) (52) Zhou (2007) (53) Ogihara (2005) (54) Nanashima (2010) (55) Hung (2011) (56) Nishikawa (2011 (57) Santambrogio (2009) (58) Ueno (2009) (59) Peng (2008) (60) Hiraoka (2008) (61) Lupo (2007) (62) Yamakado (2008) (63) Liang (2008) (64) Wakai (2006) (65) Takahashi (2007) (66) Farinati (2009) (67) Gao (2007) (68) Tashiro (2011) (69) Wong (2009) (70) Takayama (2010) (71) Lv (2006) (73)

Primary studies (Author, year)

Meta-analysis or systematic review

Table 4 The type of primary studies included in meta-analyses or systematic reviews



Y. Wang et al.


Author

RFA group(E/N)

HR group(E/N)

heterogenecity

Z-test(p-value)

OR (95% CI)

1-year OS Chen R,2008[35] Du J,2009[34] Huang Z,2012[27] Li L,2012[17] Liu J,2010[15] Liu Z,2010[14] Sun B,2011[16] Wang Z,2011[30] Xu G,2012[13] You W,2012[28] Zhou D,2011[29] Zhou Y,2010[32]

338/358 116/122 278/294 401/441 540/614 495/545 1311/1459 781/841 ˉˉ/ˉˉ 817/921 ˉˉ/ˉˉ ˉˉ/ˉˉ

320/339 133/144 355/379 414/436 492/555 402/443 1376/1506 731/798 ˉˉ/ˉˉ 770/858 ˉˉ/ˉˉ ˉˉ/ˉˉ

I-squared=0.0% I-squared=0.0% I-squared=0.0% I-squared=42.0% I-squared=0.0% I-squared=22.0% I-squared=14.0% I-squared=0.0% I-squared=14.6% I-squared=0.0% ˉ I-squared=0.0%

p=0.72 p=0.35 p=0.47 p=0.01 p=0.75 p=0.91 p=0.06 p=0.21 p=0.005 p=0.86 p=0.39 p=0.34

1.13 (0.58, 2.20) 1.63 (0.58, 4.54) 1.27 (0.66, 2.45) 0.51 (0.29, 0.86) 0.94 (0.65, 1.36) 0.97 (0.62, 1.53) 0.78 (0.61, 1.01) 1.27 (0.87, 1.86) 0.60 (0.42, 0.86) 0.97 (0.71, 1.33) 1.47 (0.61, 3.57) 0.84 (0.58, 1.21)

3-year OS Chen R,2008[35] Du J,2009[34] Gravante,2011[31] Huang Z,2012[27] Li L,2012[17] Liu J,2010[15] Liu Z,2010[14] Sun B,2011[16] Wang Z,2011[30] Xu G,2012[13] You W,2012[28] Zhou D,2011[29] Zhou Y,2010[32]

257/358 95/122 505/753 208/294 ˉˉ/ˉˉ 469/751 358/539 925/1459 582/841 ˉˉ/ˉˉ 792/1198 106/144 451/710

261/339 113/144 576/752 294/379 ˉˉ/ˉˉ 395/621 305/417 1166/1506 565/798 ˉˉ/ˉˉ 765/1042 130/165 471/633

I-squared=13% I-squared=0% ˉ I-squared=0% I-squared=64% I-squared=75.5% I-squared=52% I-squared=49% I-squared=0% I-squared=45.8% I-squared=46.4% I-squared=38% I-squared=46.3%

P=0.14 P=0.81 P 0.0001 P=0.04 P=0.03 P=0.73 P=0.002 P 0.00001 P=0.41 P=0.2 P=0.0004 P=0.26 P 0.00001

0.77 (0.54, 1.09) 0.93 (0.51, 1.68) 0.58 (0.46, 0.74) 0.68 (0.47, 0.98) 0.51 (0.28, 0.94) 0.92 (0.56, 1.51) 0.60 (0.44, 0.83) 0.49 (0.41, 0.57) 0.91 (0.73, 1.14) 0.49 (0.36, 0.65) 0.71 (0.58, 0.85) 0.74 (0.43, 1.26) 0.56 (0.44, 0.71)

5-year OS Chen R,2008[35] Gravante,2011[31] Li L,2012[17] Liu Z,2010[14] Sun B,2011[16] Xu G,2012[13] Zhou Y,2010[32]

107/182 165/378 ˉˉ/ˉˉ 245/494 449/728 ˉˉ/ˉˉ ˉˉ/ˉˉ

106/164 199/339 ˉˉ/ˉˉ 204/372 728/1039 ˉˉ/ˉˉ ˉˉ/ˉˉ

I-squared=7% ˉ I-squared=29% I-squared=41% I-squared=71% I-squared=63.7% I-squared=61.6%

P=0.37 P 0.0001 P=0.002 P=0.04 P=0.001 P=0.003 P=0.05

0.81 (0.51, 1.29) 0.48 (0.35, 0.66) 0.62 (0.45, 0.84) 0.73 (0.53, 0.99) 0.49 (0.32, 0.75) 0.60 (0.43, 0.84) 0.60 (0.36, 1.01)

.25

.5

1

Favors HR

2

4

Favors RFA

Figure 2 Overall survival (OS) rate at one, three, and five years after RFA versus HR in HCC.

Author

RFA group(E/N)

HR group(E/N)

heterogenecity

Z-test(p-value)

OR (95% CI)

1-year DFS Chen R,2008[35] Du J,2009[34] Huang Z,2012[27] Sun B,2011[16] Wang Z,2011[30] Zhou D,2011[29] Zhou Y,2010[32]

274/358 102/122 3548/3814 1084/1459 296/545 ˉˉ/ˉˉ ˉˉ/ˉˉ

272/339 122/144 3282/3463 1207/1506 463/564 ˉˉ/ˉˉ ˉˉ/ˉˉ

I-squared=0% I-squared=0% I-squared=19.5% I-squared=57% I-squared=0% ˉ I-squared=59.2%

P=0.32 P=0.84 P=0.04 P=0.01 P=0.005 P=0.55 P=0.006

0.96 (0.89, 1.04) 0.99 (0.89, 1.10) 0.80 (0.65, 0.99) 0.68 (0.50, 0.92) 0.68 (0.51, 0.90) 0.87 (0.55, 1.38) 0.54 (0.35, 0.84)

3-year DFS Chen R,2008[35] Du J,2009[34] Huang Z,2012[27] Sun B,2011[16] Wang Z,2011[30] Zhou D,2011[29] Zhou Y,2010[32]

153/358 72/122 393/910 628/1459 210/545 103/201 ˉˉ/ˉˉ

193/339 106/144 352/780 840/1506 299/564 145/233 ˉˉ/ˉˉ

I-squared=48.2% I-squared=78.6% I-squared=86.4% I-squared=84% I-squared=23.9% I-squared=59% I-squared=66.7%

P=0.02 P=0.21 P=0.05 P=0.007 P 0.0001 P=0.03 P 0.001

0.77 (0.62, 0.96) 0.46 (0.13, 1.57) 0.82 (0.67, 1.00) 0.56 (0.36, 0.85) 0.61 (0.47, 0.78) 0.65 (0.44, 0.95) 0.44 (0.28, 0.68)

5-year DFS Sun B,2011[16] Zhou Y,2010[32]

217/908 ˉˉ/ˉˉ

728/1039 ˉˉ/ˉˉ

I-squared=14% I-squared=47.2%

P 0.00001 P=0.05

0.52 (0.42, 0.63) 0.64 (0.42, 0.99)

.25

.5 Favors HR

1

2 Favors RFA

4

Figure 3 Disease-free survival (DS) at one, three, and five years after RFA versus HR in HCC.

Recurrence-free survival (RFS) Only two meta-analyses reported RFS at one, three, and five years indicating that RFA was inferior to HR. However, Liu et al. (14) misused the fixed-effects model with the heterogeneity test of I2 > 50%. After modification, RFS at one and

five years showed no significant difference between groups, with pooled ORs of 0.65 (95%CI 0.34 to 1.24) and 0.74 (95%CI 0.4 to 1.39), respectively, but the RFS difference at three years was still statistically significant between the two groups (OR = 0.46, 95%CI 0.27 to 0.79, P = 0.005) (Figure 4).


113


Y. Wang et al.

Author

RFA group(E/N)

HR group(E/N)

heterogenecity

Z-test(p-value)

OR (95% CI)

1-year RFS Li L,2012[17] Liu Z,2010[14]

ˉˉ/ˉˉ 296/398

ˉˉ/ˉˉ 253/316


P=0.03 P=0.02

0.65 (0.44, 0.97) 0.65 (0.45, 0.93)

3-year RFS Li L,2012[17] Liu Z,2010[14]

ˉˉ/ˉˉ 157/398

ˉˉ/ˉˉ 171/316


P=0.008 P 0.00001

0.65 (0.47, 0.89) 0.50 (0.36, 0.68)

5-year RFS Li L,2012[17] Liu Z,2010[14]

ˉˉ/ˉˉ 170/494

ˉˉ/ˉˉ 150/372


P=0.001 P=0.007

0.52 (0.35, 0.77) 0.66 (0.49, 0.89)

.25

.5 Favors HR

1

2 Favors RFA

4

Figure 4 Recurrence-free survival at one, three, and five years after RFA versus HR in HCC.

Author

RFA group(E/N)

HR group(E/N)

heterogenecity

Z-test(p-value)

OR (95% CI)

1-year recurernce Huang Z,2012[27] Xu G,2012[13] Liu J,2010[15] You W,2012[28]

869/3177 ˉˉ/ˉˉ 109/529 113/576

530/3050 ˉˉ/ˉˉ 87/397 94/462

I-squared=85.3% I-squared=63.4% I-squared=54.2% I-squared=43.5%

P 0.00001 P=0.025 P=0.8 P=0.73

1.81 (1.60, 2.05) 1.48 (1.05, 2.08) 0.96 (0.69, 1.33) 0.95 (0.69, 1.30)

3-year recurernce Xu G,2012[13] Liu J,2010[15] You W,2012[28]

ˉˉ/ˉˉ 379/638 404/736

ˉˉ/ˉˉ 295/488 320/607

I-squared=69.9% I-squared=82.4% I-squared=76.8%

P=0 P=0.59 P=0.86

1.76 (1.49, 2.08) 1.19 (0.63, 2.27) 1.02 (0.81, 1.28)

5-year recurernce Xu G,2012[13]

ˉˉ/ˉˉ

ˉˉ/ˉˉ

I-squared=54.5%

P=0.002

1.68 (1.21, 2.34)

.25

.5 Favors RFA

1

2 Favors HR

4

Figure 5 Recurrence rate at one, three, and five years after RFA versus HR in HCC.

Recurrence rate: Four studies reported recurrence at one year, which ranged from 19.6% to 27.4% in the RFA group and from 17.4% to 21.9% in the HR group. Two studies showed that the recurrence rate with RFA was higher than with HR, with pooled ORs of 1.81 (95%CI 1.6 to 2.05) and 1.48 (95%CI 1.05 to 2.08), respectively, but both presented varying degree of heterogeneity. However, Huang et al. (27) showed no significant difference when modifying for the random-effects model (OR 2.67, 95%CI 1.28 to 5.58, P = 0.09) (Figure 5).

114

Three reported recurrence at three years, which ranged from 54.9% to 62.2% in the RFA group and from 52.7% to 60.5% in the HR group. Only one study indicated that that the recurrence rate with RFA was higher than with HR, with a pooled OR of 1.76 (95%CI 1.49 to 2.08), but it presented high heterogeneity (I2 = 69.9%) (Figure 5). Only one meta-analysis showed that recurrence at five years in the RFA group was higher than in the HR group, with an OR of 1.68, and 95%CI of 1.21 to 2.34, I2 = 54.5% (Figure 5).


Y. Wang et al.


RFA

HR

Author

group(E/N)

group(E/N)

heterogenecity

Z-test(p-value)

OR (95% CI)

Huang Z,2012[27]

75/899

164/737

I-squared=94.2%

P 0.00001

0.40 (0.30, 0.54)

Li L,2012[17]

ˉˉ/ˉˉ

ˉˉ/ˉˉ

I-squared=84%

P=0.07

0.29 (0.08, 1.10)

Wang Z,2011[30]

59/633

167/586

I-squared=72.8%

P=0.0007

0.26 (0.12, 0.57) 0.36 (0.25, 0.51)

Sun B,2011[16]

45/815

131/979

I-squared=0%

P 0.00001

Zhou D,2011[29]

8/179

82/212

I-squared=53%

P 0.00001

0.14 (0.09, 0.22)

Liu Z,2010[14]

9/219

10/179

I-squared=11%

P=0.48

0.71 (0.27, 1.83)

Du J,2009[34]

7/122

66/144

I-squared=89.2%

P=0.22

0.21 (0.02, 2.55)

.25

.5 Favors RFA

1

2 Favors HR

4

Figure 6 Complicate rate on RFA versus HR.

Complications: Seven reported complication rates in the RFA and HR groups ranged from 54.9% to 62.2% and from 52.7% to 60.5%, respectively. Of these, four showed that the complication rate with RFA was significant lower than with HR (P < 0.05), but all studies presented varying degree of heterogeneity except Sun et al. (16). Huang et al. (27) and Zhou et al. (29) misused the fixed-effects model. However, Huang et al. (27) showed no significant difference between the two groups with modification for the randomeffects model (OR = 0.5, 95%CI 0.11 to 2.15, P = 0.35) (Figure 6). Mortality: Only two studies reported hospital mortality after RFA and HR, with the groups ranging from 0.1% to 0.3% and from 0.8% to 1.3%, respectively. Both showed that mortality after RFA was lower than after HR, but that it was not significantly different (P > 0.05) (Figure 7).

Subgroup analysis Overall survival in HCC patients with tumor size 3 cm Li et al. (17) reported that OS rates at one, three, and five years were 93.8%, 84%, and 59.9% in the RFA group, and 99%, 93.3%, and 69.2% in the HR group, but that only OS at three years showed a significant difference between the groups, with an OR of 0.56, 95%CI 0.37 to 0.84 (I2 = 55%). Xu et al. (13) showed that OS rates at one, three, and five years in the RFA group were lower than in the HR group (P < 0.05) (Figure 8). Unfortunately, he misused the statistical model in analyzing OS at five years, but we could not modify his outcome because of lack of primary data.

Discussion Systematic reviews and meta-analyses are considered to be among the most important sources of high-quality evidence, but they can be impacted by many other confounding factors, especially that of bias (74). Currently, the number of meta-analyses comparing RFA and HR for small HCC is increasing. However, the results of these studies without qualification raise questions when they are recommended as the best evidence to guide clinical practice, which may mislead clinical policy-making. The OQAQ scale checklist evaluates the key links which are prone to bias in systematic reviews (21, 74). We found that the overall quality of 19 SRs/meta-analyses was generally poor, with an estimated mean OQAQ score of 3.3 (95%CI 2.6 to 4.1). Only five (23.6%) studies had good methodological quality (15–17, 31, 35). Six (55.6%) meta-analyses misused statistical models when the results of heterogeneity tests indicated I2 values >50% (13, 14, 27–29, 33). These will not only lead an erroneous estimated effect, but also lead mistaken of clinical decision-making. For instance, Liu et al. (14) misused the fixed-effects model when he reported the pooled ORs of three-year overall survival and of one- to five-year recurrence rates, drawing the conclusion that HR was superior to RFA. However, there was no significant difference between the two groups after modifying with the random-effects model. A similar case also occurred in the research of Huang et al. (27), where there was shown to be no significant difference of three-year disease-free survival and of complication rates among groups after modification. The study design and quality of primary studies also are important factors which can impact the overall quality of meta-analyses. We found that most of the primary studies


115


Y. Wang et al.

RFA

HR

Author

group(E/N)

group(E/N)

heterogenecity

Z-test(p-value)

OR (95% CI)

Liu Z,2010[14]

2/666

7/534

I-squared=28%

p=0.07%

0.29 (0.08, 1.11)

Zhou Y,2010[32]

ˉˉ/ˉˉ

ˉˉ/ˉˉ

I-squared=0%

p=0.11%

0.36 (0.10, 1.27)

.25

.5 Favors RFA

1

2 Favors HR

4

Figure 7 Mortality after RFA versus HR in HCC.

Author

RFA

HR

group,%

group,%

heterogenecity

Z-test(p-value)

OR (95% CI)

93.8

99

I-squqre=0%

P=0.07

0.21 (0.04, 1.15)

1-year OS Li L,2012[17] Xu G,2012[13]

I-squqre=0%

0.34 (0.13, 0.89)

3-year OS Li L,2012[17]

84

93.3

Xu G,2012[13]

I-squqre=55%

P=0.03

0.38 (0.16, 0.89)

I-squqre=22.1%

0.56 (0.37, 0.84)

5-year OS Li L,2012[17] Xu G,2012[13]

59.9

69.2

I-squqre=70%

P=0.16

0.69 (0.41, 1.16)

I-squqre=75.6%

0.44 (0.31, 0.62)

.1

Favors HR

.5

1

2 Favors RFA

10

Figure 8 Survival rate at one, three, and five years after RFA versus HR in HCC 3 cm.

referenced by meta-analyses were NRCTs, with a generally poor quality of evidence (mean MINORS score of 15, with 95%CI of 14.6 to 15.4). There was some misunderstanding on the part of most of the authors of systematic reviews or meta-analyses when they judged the study design of the primary studies. First, retrospective clinical control studies were included as RCTs by most of the authors of the SRs/meta-analyses (14, 27, 29). Second, the same original studies were included by varying authors of SRs/metaanalyses, but their study designs were classified differently (45, 47, 53, 62). However, the situation where RCTs were in-

116

correctly classified as NRCTs was extremely rare, perhaps because RCTs are the standard and authors of systematic reviews focus on them. In addition, we found that the number of primary studies included in systematic reviews published in same years varied. The reasons may be either differences in inclusion criteria or insufficiency of search strategies. Unrestricted inclusion criteria are also an important factor that led to high heterogeneity among the meta-analyses. After carefully reading the full-text of the primary studies, we found that 17 of 39 (43.6%) studies did not meet the inclusion criteria of the systematic reviews, including 9 that were mixed


Y. Wang et al.

with other effective interventions (i.e., TACE, and PEI for example). To some extent, these will cause a certain bias for the estimated effect value and the credibility of the outcome. The quality of evidence (i.e., for a body of evidence) for each outcome is the key point that concerns clinicians and patients. Recently, the software of GRADE, which is “outcome centric,” can make a rating for each outcome, and then ultimately generate measures of the quality level of the evidence and the strength of the recommendations (75). Unfortunately, the related evaluation was not done in all of the included systematic reviews and meta-analyses.

Limitations In this study, we only systematically assessed the methodological quality and the effect values of outcomes. We were unable to make a quantitative analysis. Given the characteristics of the included studies, we modified the MINORS checklist (i.e., deleted the content of the prospective collection of data, the blinding evaluation of endpoint, and the prospective calculation of study size) for evaluating the methodological quality of NRCTs. The subgroups were not analyzed deeply due to the limited data, and we will improve this in our further study based on problems existing in previous meta-analyses.

Conclusion Current available evidence indicates that there was no significant difference of one-year overall survival and disease-free survival between the RFA group and the HR group for treating early resectable HCC. With prolonged periods of followup, the HR group had a higher overall survival rate, diseasefree survival, and recurrence-free survival, and a lower recurrence rate. However, the complication rate in the RFA group was significantly lower than that in the HR group. The hospital mortality in the RFA group was lower than in the HR group, but there was no significant difference between the two groups. For early HCC with tumor size smaller than 3 cm in diameter, the long-term survival in the RFA group was also lower than in the HR group. RFA is inferior to HR in improving long-term survival and reducing recurrence rate; however, as a minimally invasive, safe, effective interventional treatment, it can be considered as an important method for treatment of early HCC. However, most authors of systematic reviews or metaanalyses did not yet have an in-depth understanding of the methodology of meta-analysis, and they have not clearly known how to control bias to a minimum, resulting in poor overall quality, higher heterogeneity and a lower quality level of evidence. Therefore, physicians should exercise caution when applying the results of these systematic reviews or meta-analyses in their clinical practice. We suggested that it is


necessary to perform another meta-analysis based on the raw data and the currently updated studies, so as to reduce bias and improve the level of evidence to guide clinical decisionmaking correctly.

Competing Interests The authors have declared that no competing interests exist.

References 1. Llovet JM, Lencioni R, Di Bisceglie AM, et al. EASL-EORTC Clinical Practice Guidelines: management of hepatocellular carcinoma. J Hepatol 2012; 56(4): 908–43. 2. Soerjomataram I, Lortet-Tieulent J, Parkin DM, et al. Global burden of cancer in 2008: a systematic analysis of disability-adjusted life-years in 12 world regions. Lancet 2012; 380(9856): 1840–50. 3. Ferlay J, Shin HR, Bray F, Forman D, Mathers C, Parkin DM. GLOBOCAN 2008 v2.0,Cancer Incidence and Mortality Worldwise: IARC cancerBase No.10[Internet]. Lyon,France: International Agency for Research on Cancer 2010. Available from: http://globocan.iarc.fr. 4. Ferlay J, Shin HR, Bray F, Forman D, Mathers C, Parkin DM. Estimates of worldwide burden of cancer in 2008: GLOBOCAN 2008. Int J Cancer 2010; 127(12): 2893–917. 5. Bruix J, Sherman M. Management of hepatocellular carcinoma: an update. Hepatology 2011; 53(3): 1020–2. 6. Kobayashi M, Ikeda K, Kawamura Y, et al. High serum des-gamma-carboxy prothrombin level predicts poor prognosis after radiofrequency ablation of hepatocellular carcinoma. Cancer 2009; 115(3): 571–80. 7. Ye SL. Expert consensus on standardization of the management of primary liver cancer [Article in Chinese]. Zhonghua Gan Zang Bing Za Zhi 2009; 17(6): 403–10. 8. Crocetti L, de Baere T, Lencioni R. Quality improvement guidelines for radiofrequency ablation of liver tumours. Cardiovasc Intervent Radiol 2010; 33(1): 11–7. 9. Guglielmi A, Ruzzenente A, Valdegamberi A, et al. Radiofrequency ablation versus surgical resection for the treatment of hepatocellular carcinoma in cirrhosis. J Gastrointest Surg 2008; 12(1): 192–8. 10. Feng K, Yan J, Li X, et al. A randomized controlled trial of radiofrequency ablation and surgical resection in the treatment of small hepatocellular carcinoma. J Hepatol 2012; 57(4): 794–802. 11. Wang JH, Wang CC, Hung CH, Chen CL, Lu SN. Survival comparison between surgical resection and radiofrequency ablation for patients in BCLC very early/early stage hepatocellular carcinoma. J Hepatol 2012; 56(2): 412–8. 12. Peng ZW, Lin XJ, Zhang YJ, et al. Radiofrequency ablation versus hepatic resection for the treatment of hepatocellular carcinomas 2 cm or smaller: a retrospective comparative study. Radiology 2012; 262(3): 1022–33.


117


13. Xu G, Qi FZ, Zhang JH, Cheng GF, Cai Y, Miao Y. Meta-analysis of surgical resection and radiofrequency ablation for early hepatocellular carcinoma. World J Surg Oncol 2012; doi:10.1186/1477-7819-10-163. 14. Liu Z, Zhou Y, Zhang P, Qin H. Meta-analysis of the therapeutic effect of hepatectomy versus radiofrequency ablation for the treatment of hepatocellular carcinoma. Surg Laparosc Endosc Percutan Tech 2010; 20(3): 130–40. 15. Liu JG, Wang YJ, Du Z. Radiofrequency ablation in the treatment of small hepatocellular carcinoma: a meta analysis. World J Gastroenterol 2010; 16(27): 3450–6. 16. Sun B, Zheng CS, Feng GS, Wang Y, Xia XW, Kan XF. Radiofrequency ablation in the treatment of small hepatocellular carcinoma: a meta-analysis [Article in Chinese]. Shijie Huaren Xiaohua Zazhi 2011; 19(31): 3255–63. 17. Li L, Zhang JL, Liu XH, Li XH, Jiao BP, Kang TL. Clinical outcomes of radiofrequency ablation and surgical resection for small hepatocellular carcinoma: A meta-analysis. J Gastroenterol Hepatol 2012; 27(1): 51–8. 18. Xie XQ, Dendukuri N, McGregor M. Percutaneous radiofrequency ablation for the treatment of early stage hepatocellular carcinoma: a health technology assessment. Int J Technol Assess Health Care 2010; 26(4): 390–7. 19. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6(7): e1000097. 20. Oxman AD, Guyatt GH, Singer J, et al. Agreement among reviewers of review articles. J Clin Epidemiol 1991; 44(1): 91–8. 21. Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol 1991; 44(11): 1271–8. 22. Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ J Surg 2003; 73(9): 712–6. 23. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from: www.cochrane-handbook.org. 24. Tiong L, Maddern GJ. Systematic review and meta-analysis of survival and disease recurrence after radiofrequency ablation for hepatocellular carcinoma. Br J Surg 2011; 98(9): 1210–24. 25. Minami Y, Kudo M. Radiofrequency ablation of hepatocellular carcinoma: a literature review.Int J Hepatol 2011; doi: 10.4061/2011/104685. 26. Lau WY, Lai EC. The current role of radiofrequency ablation in the management of hepatocellular carcinoma: a systematic review. Ann Surg 2009; 249(1): 20–5. 27. Huang ZM, Zhang HY, Li ZJ, Zhao G, Xiao GF. Radiofrequency ablation versus surgical resection for the treatment of hepatocellular carcinoma: a meta-analysis [Article in Chinese]. Chin J Curr Adv Gen Surg 2012; (03): 211–5. 28. You W, Li XC. Effect of radiofrequency ablation and surgical resection on small hepatocellular carcinoma: a meta-analysis [Article in Chinese]. Acta Unversitatis Medicinalis Nanjing(Natural Science) 2012; 32(08): 1179–84.

118

Y. Wang et al.

29. Zhou DC, Geng XP, Zhu LX, Zhao HC, Liu FB, Zhao YJ. Percutaneous radiofrequency ablation versus hepatic resection for small hepatocellular carcinoma: a meta-analysis [Article in Chinese].Chin J Surg 2011; 49(12): 1132–6. 30. Wang Z, Qian YB. Treatment efficacy of radiofrequency ablation and surgical resection to small hepatocellular: a meta-analysis [Article in Chinese]. J Hepatobiliary Surg 2011; 19(01): 23–7. 31. Gravante G, Overton J, Sorge R, et al. Radiofrequency ablation versus resection for liver tumours: an evidence-based approach to retrospective comparative studies. J Gastrointest Surg 2011; 15(2): 378–87. 32. Zhou YM, Zhao YF, Li B, et al. Meta-analysis of radiofrequency ablation versus hepatic resection for small hepatocellular carcinoma. BMC Gastroenterol 2010; doi: 10.1186/1471-230X-10-78. 33. Shi LP, Wang Y. Meta-analysis of domestic and overseases three-year survival inpatients with liver cancer treated with radiofrequency ablation versus hepatectomy [Article in Chinese]. Med J Qilu 2009; 24(05): 413–5. 34. Du J, Tai QW, Cao J, Zhao DM. Comparison effects of percutanous thermal ablation versus surgical resection for small hepatocellular carcinoma: a systematic review [Article in Chinese]. ZhongHua XianDai WaiKe ZaZhi 2009; 6(10): 605–9. 35. Chen RF, Xiao DH, Zhou QB, Zhou JJ, Chen JS, Wang J. Therapeutic clinical effect of radiofrequency ablation for small hepatocellular carcinoma in cirrhotic patients: a meta-analysis [Article in Chinese]. Chin J Surg 2008; 46(18): 1413–8. 36. Chagnon S. Prospective randomized trial of radiofrequency ablation versus surgical resection for small hepatocellular carcinomas. J Radiol 2007; 88(12): 1842–4. 37. Chen MS, Li JQ, Zheng Y, et al. A prospective randomized trial comparing percutaneous local ablative therapy and partial hepatectomy for small hepatocellular carcinoma. Ann Surg 2006; 243(3): 321–8. 38. Huang J, Yan L, Cheng Z, et al. A randomized trial comparing radiofrequency ablation and surgical resection for HCC conforming to the Milan criteria. Ann Surg 2010; 252(6): 903–12. 39. Chen MS, Li JQ, Liang HH, et al. Comparison of effects of percutaneous radiofrequency ablation and surgical resection on small hepatocellular carcinoma [Article in Chinese]. Natl Med J China 2005; 85(2): 80–3. 40. Cho CM, Tak WY, Kweon YO, et al. The comparative results of radiofrequency ablation versus surgical resection for the treatment of hepatocellular carcinoma. Korean J Hepatol 2005; 11(1): 59–71. 41. Vivarelli M, Guglielmi A, Ruzzenente A, et al. Surgical resection versus percutaneous radiofrequency ablation in the treatment of hepatocellular carcinoma on cirrhotic liver. Ann Surg 2004; 240(1): 102–7. 42. Li LX, Ye SL, Yang BW, et al. The value of radiofrequency ablation and surgical resection for small hepatocellular carcinoma [Article in Chinese]. ZhongGuo LinChuang YiXue 2004; 11(6): 1011–3.


Y. Wang et al.

43. Guo WX, Zhai B, Lai EC, et al. Percutaneous radiofrequency ablation versus partial hepatectomy for multicentric small hepatocellular carcinomas: a nonrandomized comparative study. World J Surg 2010; 34(11): 2671–6. 44. Hasegawa K, Makuuchi M, Takayama T, et al. Surgical resection vs. percutaneous ablation for hepatocellular carcinoma: a preliminary report of the Japanese nationwide survey. J Hepatol 2008; 49(4): 589–94. 45. Hong SN, Lee SY, Choi MS, et al. Comparing the outcomes of radiofrequency ablation and surgery in patients with a single small hepatocellular carcinoma and well-preserved hepatic function. J Clin Gastroenterol 2005; 39(3): 247–52. 46. Bu XY, Wang Y, Ge Z, Wang ZC, Zhang DS.Comparison of radiofrequency ablation and surgical resection for small primary liver carcinoma [Article in Chinese]. Chin Arch Gen Surg (Electronic Edition) 2009; 3(02): 127–31. 47. Huang J, Hernandez-Alejandro R, Croome KP, et al. Radiofrequency ablation versus surgical resection for hepatocellular carcinoma in Childs A cirrhotics-a retrospective study of 1,061 cases. J Gastrointest Surg 2011; 15(2): 311–20. 48. Shen Q, Xue HZ, Jiang QF, et al. Comparison of the effects of percutaneous radiofrequency ablation and surgical re-resection on postoperative recurrence of hepatocellular carcinoma [Article in Chinese]. ZhongGuo ZhongLiu LinChuang 2008; 35(19): 1088–92. 49. Ren ZG, Gan YH, Fan J, et al. Treatment of postoperative recurrence of hepatocellular carcinoma with radiofrequency ablation comparing with repeated surgical resection[Article in Chinese]. Chin J Surg 2008; 46(21): 1614–6. 50. Zhang LQ, Wang JM. Clinical control study of radio frequency ablation and operation on complications and survival rate of liver cancer [Article in Chinese]. Acta Academiae Medicinae Militrris Tertiae 2007; 29(5): 457–9. 51. Montorsi M, Santambrogio R, Bianchi P, et al. Survival and recurrences after hepatic resection or radiofrequency for hepatocellular carcinoma in cirrhotic patients: a multivariate analysis. J Gastrointest Surg 2005; 9(1): 62–7; discussion 7–8. 52. Abu-Hilal M, Primrose JN, Casaril A, McPhail MJ, Pearce NW, Nicoli N. Surgical resection versus radiofrequency ablation in the treatment of small unifocal hepatocellular carcinoma. J Gastrointest Surg 2008; 12(9): 1521–6. 53. Zhou T, Qiu YD, Kong WT, Zhang WW, Ding YT. Comparison the effect of radiofrequency ablation and surgical resection for the treatment of small hepatocellular carcinoma [Article in Chinese]. J Hepatobiliary Surg 2007; 15(6): 424–7. 54. Ogihara M, Wong LL, Machi J. Radiofrequency ablation versus surgical resection for single nodule hepatocellular carcinoma: long-term outcomes. HPB (Oxford) 2005; 7(3): 214–21. 55. Nanashima A, Tobinaga S, Masuda J, et al. Selecting treatment for hepatocellular carcinoma based on the results of hepatic resection and local ablation therapy. J Surg Oncol 2010; 101(6): 481–5. 56. Hung HH, Chiou YY, Hsia CY, et al. Survival rates are comparable after radiofrequency ablation or surgery in patients


57.

58.

59.

60.

61.

62.

63.

64.

65.

66.

67.

68.

69.

with small hepatocellular carcinomas. Clin Gastroenterol Hepatol 2011; 9(1): 79–86. Nishikawa H, Inuzuka T, Takeda H, et al. Comparison of percutaneous radiofrequency thermal ablation and surgical resection for small hepatocellular carcinoma. BMC Gastroenterol 2011; 11: 143. Santambrogio R, Opocher E, Zuin M, et al. Surgical resection versus laparoscopic radiofrequency ablation in patients with hepatocellular carcinoma and Child-Pugh class a liver cirrhosis. Ann Surg Oncol 2009; 16(12): 3289–98. Ueno S, Sakoda M, Kubo F, et al. Surgical resection versus radiofrequency ablation for small hepatocellular carcinomas within the Milan criteria. J Hepatobiliary Pancreat Surg 2009; 16(3): 359–66. Peng ZW, Xu L, Chen MS, et al. Percutaneous radiofrequency ablation and surgical resection for small hepatocellular carcinoma: a comparative study [Article in Chinese]. ZhongGuo ShiYong WaiKe ZaZhi 2008; 28(08): 633–6. Hiraoka A, Horiike N, Yamashita Y, et al. Efficacy of radiofrequency ablation therapy compared to surgical resection in 164 patients in Japan with single hepatocellular carcinoma smaller than 3 cm, along with report of complications. Hepatogastroenterology 2008; 55(88): 2171–4. Lupo L, Panzera P, Giannelli G, Memeo M, Gentile A, Memeo V. Single hepatocellular carcinoma ranging from 3 to 5 cm: radiofrequency ablation or resection? HPB (Oxford) 2007; 9(6): 429–34. Yamakado K, Nakatsuka A, Takaki H, et al. Early-stage hepatocellular carcinoma: radiofrequency ablation combined with chemoembolization versus hepatectomy. Radiology 2008; 247(1): 260–6. Liang HH, Chen MS, Peng ZW, et al. Percutaneous radiofrequency ablation versus repeat hepatectomy for recurrent hepatocellular carcinoma: a retrospective study. Ann Surg Oncol 2008; 15(12): 3484–93. Wakai T, Shirai Y, Suda T, et al. Long-term outcomes of hepatectomy vs percutaneous ablation for treatment of hepatocellular carcinoma < or = 4 cm. World J Gastroenterol 2006; 12(4): 546–52. Takahashi S, Kudo M, Chung H, et al. Outcomes of nontransplant potentially curative therapy for early-stage hepatocellular carcinoma in Child-Pugh stage A cirrhosis is comparable with liver transplantation. Dig Dis 2007; 25(4): 303–9. Farinati F, Sergio A, Baldan A, et al. Early and very early hepatocellular carcinoma: when and how much do staging and choice of treatment really matter? A multi-center study. BMC Cancer 2009; 9: 33. Gao W, Chen MH, Yan K, Yang W, Sun Y, Xing BC. Therapeutic effect of radiofrequency ablation in unsuitable operative small hepatocellular carcinoma[Article in Chinese]. Chin J Med Imaging Technol 2007; 23(02): 254–7. Tashiro H, Aikata H, Waki K, et al. Treatment strategy for early hepatocellular carcinomas: comparison of radiofrequency ablation with or without transcatheter arterial


119


70.

71.

72.

73.

74.

75.

76.

120

chemoembolization and surgical resection. J Surg Oncol 2011; 104(1): 3–9. Wong J, Lee KF, Lee PS, et al. Radiofrequency ablation for 110 malignant liver tumours: preliminary results on percutaneous and surgical approaches. Asian J Surg 2009; 32(1): 13–20. Takayama T, Makuuchi M, Hasegawa K. Single HCC smaller than 2 cm: surgery or ablation?: surgeon’s perspective. J Hepatobiliary Pancreat Sci 2010; 17(4): 422–4. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17(1): 1–12. Lv MD, Kuang M, Liang LJ, et al. Surgical resection versus percutaneous thermal ablation for early-stage hepatocellular carcinoma: a randomized clinical trial[Article in Chinese]. Natl Med J China 2006; 86(12): 801–5. Zhang JH, Shang HC, Zhang BL. How to assess the quality of systematic review and meta-analysis [Article in Chinese]. Zhong Xi Yi Jie He Xue Bao 2008; 6(4): 337–40. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011; 64(4): 383–94. Cho YK, Rhim H, Noh S. Radiofrequency ablation versus

Y. Wang et al.

surgical resection as primary treatment of hepatocellular carcinoma meeting the Milan criteria: a systematic review. J Gastroenterol Hepatol 2011; 26(9): 1354–60.

Supporting Information Disclaimer: Supplementary materials have been peerreviewed but not copyedited. Table S1. Data collection form for the quality of metaanalyses Table S2. OQAQ score and quality grading Table S3. Modified MINORS score Table S4. The quality assessment of primary studies (nonrandomized controlled tries) included in metaanalysis/systematic review using the modified MINORS score Figure S1. Risk of bias summary: review authors’ judgments about each risk of bias item for each included RCTs. Figure S2. Risk of bias graph: review authors’ judgments about each risk of bias item presented as percentages across all included RCTs.


Radiofrequency ablation versus hepatic resection for small hepatocellular carcinoma: a meta-analysis of randomized controlled trials.

Repeated Hepatic Resection versus Radiofrequency Ablation for Recurrent Hepatocellular Carcinoma after Hepatic Resection: A Propensity Score Matching Study.

Comparative analysis of intraoperative radiofrequency ablation versus non-anatomical hepatic resection for small hepatocellular carcinoma: short-term result.

Small Hepatocellular Carcinoma: Radiofrequency Ablation versus Nonanatomic Resection--Propensity Score Analyses of Long-term Outcomes.

Hepatic resection versus transplantation for hepatocellular carcinoma.

Comparative study of percutaneous radiofrequency ablation and hepatic resection for small, poorly differentiated hepatocellular carcinomas.

Radiofrequency ablation versus hepatic resection for small hepatocellular carcinomas: a meta-analysis of randomized and nonrandomized controlled trials.

Surgical Resection Versus Radiofrequency Ablation for Single Hepatocellular Carcinoma ≤ 2 cm in a Propensity Score Model.

Outcome of salvage hepatic resection for recurrent hepatocellular carcinoma after radiofrequency ablation therapy.

Comparison of hepatic resection and radiofrequency ablation for small hepatocellular carcinoma: a meta-analysis of 16,103 patients.

Radiofrequency ablation versus resection for the treatment of early stage hepatocellular carcinoma: a multicenter Australian study.

Radiofrequency ablation versus surgical resection for intrahepatic hepatocellular carcinoma recurrence: a meta-analysis.

Efficacy and safety of percutaneous radiofrequency ablation versus surgical resection for small hepatocellular carcinoma: a meta-analysis of 23 studies.

Radiofrequency ablation versus stereotactic body radiotherapy for small hepatocellular carcinoma: a Markov model-based analysis.

Small single-nodule hepatocellular carcinoma: comparison of transarterial chemoembolization, radiofrequency ablation, and hepatic resection by using inverse probability weighting.

Radiofrequency ablation versus laser ablation for the treatment of small hepatocellular carcinoma in cirrhosis: a randomized trial.

Hepatic resection for hepatocellular carcinoma.

Recent progress in radiofrequency ablation therapy for hepatocellular carcinoma.

Combined resection with radiofrequency ablation for bilobar hepatocellular carcinoma: a single-center experience.

Hepatic abscess with hepatobronchial fistula following percutaneous radiofrequency ablation for hepatocellular carcinoma: A case report.

Hepatic resection for minute hepatocellular carcinoma.

Comparable Outcomes of Ultrasound versus Computed Tomography in the Guidance of Radiofrequency Ablation for Hepatocellular Carcinoma.

Comparison of Surgical Resection and Radiofrequency Ablation for Hepatocellular Carcinoma: Take Care Not to Neglect Radiofrequency Technic and Device.

Aggressive tumor recurrence after radiofrequency ablation for hepatocellular carcinoma.