P RO G RE SS IN CAR D IOV AS CU L A R D I SE A S ES 5 8 ( 2 01 5 ) 3 – 9
Available online at www.sciencedirect.com
ScienceDirect www.onlinepcd.com
Is the Guideline Process Replicable and, if Not, What Does This Mean? Allan Snidermana,⁎, Curt D. Furbergb , Peter P. Tothc , George Thanassoulisd a
Mike Rosenbloom Laboratory for Cardiovascular Research, McGill University Health Centre, Montreal, Quebec, Canada Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA c CH Medical Center, Sterling, Illinois and Ciccarone Center for Cardiovascular Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA d Preventive and Genomic Cardiology, Department of Medicine, Royal Victoria Hospital, McGill University Health Centre, Montreal, Quebec, Canada b
A R T I C LE I N F O
AB ST R A C T
Keywords:
Increasingly, guidelines determine how medical care will be provided. However there has been
Guidelines
limited study of the determinants of the reliability of the guideline process. Guidelines translate
LDL-C
evidence into recommendations. If only the evidence determines the recommendations, given
Cardiovascular prevention
the same evidence, different panels of experts should make the same recommendations. That
Evidence-based medicine
is, the process should be replicable, an essential characteristic of a valid scientific process. The
Cardiovascular risk
multiple recent cholesterol guidelines, which have considered the same evidence, offer an opportunity to examine guidelines from this perspective. Considerable discordance among the guideline recommendations is evident pointing to an important role for the participants, in addition to the evidence, in the development of guideline recommendations. Guideline recommendations, therefore, appear to be based on both evidence and expert opinion. © 2015 Elsevier Inc. All rights reserved.
Background and objective The practice of medicine is now governed by guidelines. The recommendations of guidelines have become the standard of care and, increasingly, reimbursement is linked to adherence. The knowledge base on which trainees are evaluated is the knowledge base defined by guidelines. The knowledge base on which recertification is based is the knowledge base defined by guidelines. To pretend otherwise is to deny the reality that the practice of medicine is now governed to an increasing degree by the paradigm of evidence-based medicine (EBM) and its product-guideline-based care.
We accept that values of EBM do not differ from the classical values of medical care of which the principal one is to ensure the best possible outcome for each individual patient.1 What is different is that EBM has become a method to decide how that should be determined, a method in which best care for an individual patient is based, to the greatest extent possible, on the results of what has occurred in groups of patients studied under the most controlled circumstances possible—the randomized clinical trial (RCT). The strengths and limitations of the RCT as an experimental tool are not the focus of this essay. Rather, our purpose is to examine this process: the translation of evidence by experts into recommendations.
Statement of Conflict of Interest: see page 8. ⁎ Address reprint requests to Allan D. Sniderman, MD, FRSC, Mike Rosenbloom Laboratory for Cardiovascular Research, Royal Victoria Hospital, McGill University Health Centre, 687 Pine Avenue West, Montreal, Quebec, H3A 1A1 Canada. E-mail address: [email protected] (A. Sniderman). http://dx.doi.org/10.1016/j.pcad.2015.05.002 0033-0620/© 2015 Elsevier Inc. All rights reserved.
4
P RO G R E SS IN CAR D IOV ASC U LA R D IS E A S ES 58 ( 20 1 5 ) 3 – 9
Abbreviations and Acronyms
The interpretation of any single piece of eviAHA/ACC = American Heart dence requires the evalAssociation/American uation of the methods College of Cardiology and conclusions of the Apo = apolipoprotein study that generated that evidence. Once there is CCS = Canadian more than one piece of Cardiovascular Society evidence, whatever there CHD = coronary heart disease is, must be assembled, evaluated and integrated CKD = chronic kidnet disease into a final recommenCV = cardiovascular dation by however many individuals make up DM = diabetes mellitus the panel. Accordingly, EAS/ESC = European guideline recommendaAtherosclerosis Society/European tions are the outcome Society of Cardiology of a complex process, a process that has not EBM = evidence based medicine been studied and valiHDL-C = high-density dated. If the outcome of liipoprotein cholesterol this complex process is IAS = International not automatic, if it is Atherosclerosis Society not certain and necessary, then the recomJBS3 = Joint British Society mendations that ensue LDL-C = low-density cannot simply be a prodlipoprotein cholesterol uct of the evidence, but must also be influenced NLA = National Lipid Association by the participants, by RCT = randomized their views and the intercontrolled trial actions among them. If so, the recommendations may be evidence-based, but they are not entirely evidence-determined. Therefore whatever emerges is only one of many possible outcomes. If this is the case, the recommendations of any guideline process cannot be assumed to be valid simply because an agreed-on deliberative process has been followed. Replicability is an essential feature of a scientifically valid process, although this does not ensure that the conclusions are valid, but merely that the process by which the conclusions were reached is reproducible. Accordingly, to assess the replicability of the guideline development process, we will examine the replicability of the recent cholesterol guidelines. These were selected for two reasons. First, in this domain of research, there is an abundance of the highest quality evidence. Many RCTs that are accepted as well designed and well conducted have been performed and several meta-analyses that are accepted as well designed and well conducted have been performed on these RCTs. Second, there are numerous recent cholesterol guidelines, which have been conducted by prominent national and professional groups. The recommendations from these groups have been based on almost the same body of evidence. To the extent their recommendations are the same or similar, the process is replicable. To the extent, they are not, the guideline process is not replicable and one would conclude that the outcome has been determined by subjective as well as objective forces.
Cholesterol guidelines The recommendations of six recent major cholesterol guidelines were compared. These were issued by the European Atherosclerosis Society/European Society of Cardiology (EAS/ESC) in 2011,2 by the Canadian Cardiovascular Society (CCS) in 2012,3 by the American College of Cardiology/American Heart Association (ACC/AHA) in 2013,4 by the Joint British Society (JBS3) in 2014,5 by the International Atherosclerosis Society (IAS) in 20146 and by the National Lipid Association (NLA) in 2014.7 Five items were selected on which to compare the 6 guidelines. These selections cover different aspects of the process of care. For each item, we determine whether the recommendations are concordant or discordant and, if the latter, whether in our view, the discordance is significant or not. The issue is not whether the guideline recommendations are concordant with our views but only whether they are concordant with each other: that is, do they or do they not reach the same judgments on the same issues?
Methods to select subjects for primary prevention All distinguished between primary and secondary prevention and all chose categorical indications for secondary prevention such as symptomatic vascular disease or diabetes or extreme elevations of low-density lipoprotein cholesterol (LDL-C) because all these clinical categories are acknowledged to be associated with subsequent high risk of a cardiovascular (CV) event. Specific definitions differed in some regards, such as when patients with type I diabetes mellitus (DM) were at high risk but these differences were minor. Accordingly, in this regard, the recommendations of all the guideline groups are concordant. However, considerable discordance was evident in the selection of subjects for primary prevention with lipid lowering therapy. (Table 1) Three of the six guidelines selected subjects for statin therapy for primary prevention based on calculated 10-year risk. The algorithms used differed but their operative features were the same. In SCORE, the algorithm used by EAS/ ESC, only fatal CV events were included, whereas in the others, non-fatal events were included as well. All the algorithms used had been validated although not necessarily in the populations to whom they were applied. Thus, the CCS guidelines for a multiethnic Canadian population calculated risk based on the Framingham algorithm, which was developed for a primarily white Caucasian population. Interestingly, none of the guidelines compared the performance of the algorithm they had
Table 1 – Primary Prevention is based on: Guideline
Primary
Supplementary
EAS/ESC
10 year—SCORE (Fatal CVD)
CCS
10 year Framinghahm + LDLC/non-HDL-C/apoB in Intermediate Risk 10 year pooled cohort equation 10 year QRISK2 Risk up to age 80 Risk factor Counting
Risk relative to a normal Cardiovascular age
ACC/AHA JBS3 IAS NLA
Lifetime Lifetime
P RO G RE SS IN CAR D IOV AS CU L A R D I SE A S ES 5 8 ( 2 01 5 ) 3 – 9
selected against the others. Three guidelines took significantly different approaches. The CCS guidelines divide subjects into low risk, intermediate risk and high risk based on calculated 10-year risk. LDL lowering therapy was not recommended in the former but was recommended in the latter. Intermediate risk was further stratified based on LDL-C: statin therapy was recommended in those with an LDL C ≥ 3.5 mmol/l and in those in whom either non- high-density lipoprotein cholesterol (nonHDL-C) or apolipoprotein (apo) B was increased above a specified level in those with an LDL C ≤ 3.5 mmol/l. By contrast, the IAS guidelines are based on risk up until age 80 rather than 10-year risk and the NLA guidelines were based on risk factor counting, not on risk as estimated by an algorithm that incorporates all the risk factors. Thus, CCS, IAS and NLA are each significantly discordant from each other and the rest. In contrast, all the guidelines acknowledged that for younger subjects, risk could be low in the short term but high in the longer term and five of the six identified secondary instruments, such as lifetime risk, that could be used for this purpose. These differed based on the guideline. Nevertheless, the majority recognized an important limitation of estimation of risk based on a 10-year window.
Non-invasive tests for risk assessment Table 2 lists the positions of the different guidelines on the merits of non-invasive tests to assess the risk of CV events. Based on consideration of costs, risks and benefits, the assessment of the evidence produced significantly discordant recommendations. At one end of the spectrum are the EAS/ESC guidelines with a positive stress echocardiogram or nuclear imaging or carotid plaque all denoting very high risk. Accordingly, all are taken as coronary heart disease (CHD) equivalents. Curiously, coronary calcification, the non-invasive test that arguably has been most extensively studied and validated is not listed. The CCS guidelines allow the results of carotid ultrasound, ankle brachial index and coronary calcification to be used for patients at intermediate risk. A positive coronary calcium is defined as an Agaston score ≥300 units whereas the NLA defines a positive Agaston score as ≥100 units. At the other end of the spectrum, three of the guidelines—ACC/AHA, JBS3 and IAS—recommend against the use at the present time of any non-invasive test to select subjects for preventive LDL-C lowering therapy. They judged that the balance of the evidence indicates such tests are either not worthwhile or too costly.
Table 2 – Non-invasive tests for risk assessment. Guideline
Test/Decision
EAS/ESC
Positive stress echo, nuclear imaging, carotid plaque: very high risk For intermediate risk patients, carotid ultrasound, ankle brachial index CAC ≥ 300 Agaston units None recommended None recommended None recommended CAC ≥ 100 Agaston units
CCS
ACC/AHA JBS3 IAS NLA
5
Accordingly, for this criterion, there is substantial discordance among the guidelines. Interestingly, no cost benefit analyses are cited by any of these guidelines for their conclusions. If the issue is treatment with statins of all at intermediate risk versus selection of those at intermediate risk with a positive calcium score, then broader statin therapy is the preferred choice.8,9 Neither was cited by any of the guidelines.
Is chronic kidney disease (CKD) a CHD risk equivalent? Sometime ago, diabetes mellitus was broadly accepted by guideline groups as equivalent in CV risk to symptomatic CV disease. The same argument is now being made for CKD. It is worth noting that whereas the evidence for clinical benefit with LDL-C lowering therapy is strong in patients with type 2 DM, that is not the case for those with end stage renal disease.10,11 Nevertheless, with relatively minor differences, three guidelines accept substantial reduction in glomerular filtration rate as a CHD risk equivalent (Table 3) while one, CCS, accepts albuminuria as well as a reduction in glomerular filtration rate as sufficient justification for statin LDL-C lowering therapy. Two, ACC/AHA and IAS, state the evidence is inadequate. None discuss whether statin therapy alone or the statin-ezetimibe regimen used in the SHARP trial12 should be the preferred regimen. Accordingly, overall, there is substantial discordance among the guidelines for this criterion also.
Which lipid measure should be used to determine the adequacy of the response to therapy? Four guideline panels chose LDL-C as the primary measure of the adequacy of the response to LDL lowering therapy whereas two chose non-HDL-C. (Table 4) Three of the four that chose LDL-C also chose non-HDL-C as an alternate to LDL-C and two of these also allow apoB as an additional option. None review in detail the evidence that determined their decision. There is significant discordance, therefore, for this recommendation also.
Consistency of treatment strategies among the guidelines The positions of the 6 guidelines are summarized in Table 5. Each takes a different position. CCS allows three markers to assess the adequacy of the treatment response with one level selected for each for subjects at intermediate and high risk
Table 3 – Is chronic CKD an indication for preventive statin therapy? Guideline
Renal Parameter
EAS/ESC CCS
CKD 3–5 CKD 30 (or ≥ 300 mg/day) Albuminuria, GFR of uncertain value CKD 3–5 Not worthwhile 3B (30–44 ml/min/1.73 m2 4 (15–29 ml/min/1.73 m2
ACC/AHA JBS3 IAS NLA
6
P RO G R E SS IN CAR D IOV ASC U LA R D IS E A S ES 58 ( 20 1 5 ) 3 – 9
whereas EAS/ESC, which also allows three markers, has three different targets for moderate risk, high risk and very high risk. Both also state a 50% reduction in LDL-C represents an adequate response although neither document the evidence in support of this recommendation. JBS3 selected a non-HDL-C target, but this target does not differ among the different risk categories whereas the NLA, which also favors non-HDL-C, mandates two intensities of therapy. The IAS favoured the concept of targets but did not select specific levels delegating that choice to national societies. The position of ACC/AHA is complex. The text at multiple points states that the statin randomized control trials were tests of regimens not of targets. Accordingly, almost all have concluded that ACC/AHA rejected targets, and this conclusion has attracted considerable attention and generated considerable controversy with several other guidelines rejecting this view and their participants or supporters confirming their own positions as to the validity of targets.13,14 The fact that recommendations that differ are always reaffirmed and never modified does not speak to the process as open and responsive. Importantly these replies do not note that the formal recommendation of ACC/AHA regarding targets—ACC/AHA Table 4 recommendation 1—is that they make no recommendation—either for or against—targets. This should mean that physicians are free either to use a target-based strategy or a regimen-based strategy. Nor do any note that the ACC/AHA guidelines do outline a strategy to assess adherence to therapy. Specific details are given as to when follow up lipid values should be obtained and what constitutes an adequate response (i.e. a 50% decrease in LDL-C from fasting levels) and what should be done if the response has been less than adequate (i.e. consider increasing the dose of the statin or adding an additional LDL lowering agent). This would appear to constitute a de facto target strategy. On this issue, therefore, not only is there substantial discordance among the guideline recommendations—and indeed, within the ACC/AHA guidelines—there is no evidence that this discordance can be analyzed and resolved.
Items the cholesterol guidelines agree about but without evidence So far, we have identified issues on which there is substantial discordance among the guidelines but there is another category of issue that should be briefly noted—namely, issues on which adequate evidence is absent but the guidelines agree and the absence of evidence is not noted. Such agreement points to Table 4 – Which lipid measure should be the primary index of the adequacy of the response to therapy? Guideline
Renal Parameter
EAS/ESC CCS ACC/AHA JBS3 IAS NLA
LDL-C primary; non-HDL-C/apoB alternate LDL-C primary; non-HDL-C/apoB alternate LDL-C Non-HDL-C LDL-C primary; non-HDL-C alternate non-HDL-C
shared core convictions among the participants. Among these are: (i) The statin RCTs were designed to test the benefit and risks of statin therapy for a limited period—generally not more than 5 years. Nevertheless, statin therapy is recommended without a time limit. As a practical matter that may be reasonable. However, such a recommendation is not evidence-based and this limitation is never noted. (ii) A number of guidelines recommend that LDL-C be directly measured rather than calculated by the Friedewald method, since LDL-C can be estimated no matter the triglyceride level. Not mentioned is that none of these methods are standardized and there is no evidence that that the direct measures provide superior care. Accordingly, there is no evidence they are cost-effective. Nevertheless, they are recommended. (iii) Maximal dose statin therapy is recommended for high risk subjects notwithstanding that there is no RCT evidence that maximal dose statin therapy is superior to moderate dose statin therapy. That is, atorvastatin 80 mg per day may produce a better clinical result than atorvastatin 10 mg per day but there is no evidence that atorvastatin 80 mg per day is superior to atorvastatin 40 mg per day or even to atorvastatin 20 mg per day.15 Doubling of statin dose produces only limited further reduction of LDL-C and thus would be expected to produce only limited further clinical benefit.15 Unfortunately, higher doses of statins are associated with more side effects, such as myalgia, and these side effects are associated with reduced adherence and increased mortality.16,17 Accordingly, these recommendations are based on opinion not evidence.
Conclusion and recommendations Notwithstanding the wealth of available RCT results and notwithstanding that all considered the same evidence and notwithstanding that all adopted an ‘evidence-based’ approach, the recommendations of these six cholesterol guidelines differ profoundly. With regard to these issues, the outcome of the guideline process was not replicable. This represents prima facie evidence that the process itself is not automatic and error-free. To be sure, we have compared only a few of the total number of recommendations but a review of each and every one would neither be practical nor necessary. Demonstrating discordance on several should suffice. That guideline recommendations are not replicable should not be surprising. They are the outcome of a deliberative process by a group of individuals, each with a history of achievements and failures, each with insights and biases, each with strengths and limitations of understanding and expression. The result is that the outcome—the recommendations that are generated—is not determined solely by the material that is examined but is also shaped by how the participants interpret that material, by how they interact with
7
P RO G RE SS IN CAR D IOV AS CU L A R D I SE A S ES 5 8 ( 2 01 5 ) 3 – 9
Table 5 – Guideline targets of therapy. Low Risk
Moderate Risk
CCS LDL-C
Available online at www.sciencedirect.com
ScienceDirect www.onlinepcd.com
Is the Guideline Process Replicable and, if Not, What Does This Mean? Allan Snidermana,⁎, Curt D. Furbergb , Peter P. Tothc , George Thanassoulisd a
Mike Rosenbloom Laboratory for Cardiovascular Research, McGill University Health Centre, Montreal, Quebec, Canada Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA c CH Medical Center, Sterling, Illinois and Ciccarone Center for Cardiovascular Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA d Preventive and Genomic Cardiology, Department of Medicine, Royal Victoria Hospital, McGill University Health Centre, Montreal, Quebec, Canada b
A R T I C LE I N F O
AB ST R A C T
Keywords:
Increasingly, guidelines determine how medical care will be provided. However there has been
Guidelines
limited study of the determinants of the reliability of the guideline process. Guidelines translate
LDL-C
evidence into recommendations. If only the evidence determines the recommendations, given
Cardiovascular prevention
the same evidence, different panels of experts should make the same recommendations. That
Evidence-based medicine
is, the process should be replicable, an essential characteristic of a valid scientific process. The
Cardiovascular risk
multiple recent cholesterol guidelines, which have considered the same evidence, offer an opportunity to examine guidelines from this perspective. Considerable discordance among the guideline recommendations is evident pointing to an important role for the participants, in addition to the evidence, in the development of guideline recommendations. Guideline recommendations, therefore, appear to be based on both evidence and expert opinion. © 2015 Elsevier Inc. All rights reserved.
Background and objective The practice of medicine is now governed by guidelines. The recommendations of guidelines have become the standard of care and, increasingly, reimbursement is linked to adherence. The knowledge base on which trainees are evaluated is the knowledge base defined by guidelines. The knowledge base on which recertification is based is the knowledge base defined by guidelines. To pretend otherwise is to deny the reality that the practice of medicine is now governed to an increasing degree by the paradigm of evidence-based medicine (EBM) and its product-guideline-based care.
We accept that values of EBM do not differ from the classical values of medical care of which the principal one is to ensure the best possible outcome for each individual patient.1 What is different is that EBM has become a method to decide how that should be determined, a method in which best care for an individual patient is based, to the greatest extent possible, on the results of what has occurred in groups of patients studied under the most controlled circumstances possible—the randomized clinical trial (RCT). The strengths and limitations of the RCT as an experimental tool are not the focus of this essay. Rather, our purpose is to examine this process: the translation of evidence by experts into recommendations.
Statement of Conflict of Interest: see page 8. ⁎ Address reprint requests to Allan D. Sniderman, MD, FRSC, Mike Rosenbloom Laboratory for Cardiovascular Research, Royal Victoria Hospital, McGill University Health Centre, 687 Pine Avenue West, Montreal, Quebec, H3A 1A1 Canada. E-mail address: [email protected] (A. Sniderman). http://dx.doi.org/10.1016/j.pcad.2015.05.002 0033-0620/© 2015 Elsevier Inc. All rights reserved.
4
P RO G R E SS IN CAR D IOV ASC U LA R D IS E A S ES 58 ( 20 1 5 ) 3 – 9
Abbreviations and Acronyms
The interpretation of any single piece of eviAHA/ACC = American Heart dence requires the evalAssociation/American uation of the methods College of Cardiology and conclusions of the Apo = apolipoprotein study that generated that evidence. Once there is CCS = Canadian more than one piece of Cardiovascular Society evidence, whatever there CHD = coronary heart disease is, must be assembled, evaluated and integrated CKD = chronic kidnet disease into a final recommenCV = cardiovascular dation by however many individuals make up DM = diabetes mellitus the panel. Accordingly, EAS/ESC = European guideline recommendaAtherosclerosis Society/European tions are the outcome Society of Cardiology of a complex process, a process that has not EBM = evidence based medicine been studied and valiHDL-C = high-density dated. If the outcome of liipoprotein cholesterol this complex process is IAS = International not automatic, if it is Atherosclerosis Society not certain and necessary, then the recomJBS3 = Joint British Society mendations that ensue LDL-C = low-density cannot simply be a prodlipoprotein cholesterol uct of the evidence, but must also be influenced NLA = National Lipid Association by the participants, by RCT = randomized their views and the intercontrolled trial actions among them. If so, the recommendations may be evidence-based, but they are not entirely evidence-determined. Therefore whatever emerges is only one of many possible outcomes. If this is the case, the recommendations of any guideline process cannot be assumed to be valid simply because an agreed-on deliberative process has been followed. Replicability is an essential feature of a scientifically valid process, although this does not ensure that the conclusions are valid, but merely that the process by which the conclusions were reached is reproducible. Accordingly, to assess the replicability of the guideline development process, we will examine the replicability of the recent cholesterol guidelines. These were selected for two reasons. First, in this domain of research, there is an abundance of the highest quality evidence. Many RCTs that are accepted as well designed and well conducted have been performed and several meta-analyses that are accepted as well designed and well conducted have been performed on these RCTs. Second, there are numerous recent cholesterol guidelines, which have been conducted by prominent national and professional groups. The recommendations from these groups have been based on almost the same body of evidence. To the extent their recommendations are the same or similar, the process is replicable. To the extent, they are not, the guideline process is not replicable and one would conclude that the outcome has been determined by subjective as well as objective forces.
Cholesterol guidelines The recommendations of six recent major cholesterol guidelines were compared. These were issued by the European Atherosclerosis Society/European Society of Cardiology (EAS/ESC) in 2011,2 by the Canadian Cardiovascular Society (CCS) in 2012,3 by the American College of Cardiology/American Heart Association (ACC/AHA) in 2013,4 by the Joint British Society (JBS3) in 2014,5 by the International Atherosclerosis Society (IAS) in 20146 and by the National Lipid Association (NLA) in 2014.7 Five items were selected on which to compare the 6 guidelines. These selections cover different aspects of the process of care. For each item, we determine whether the recommendations are concordant or discordant and, if the latter, whether in our view, the discordance is significant or not. The issue is not whether the guideline recommendations are concordant with our views but only whether they are concordant with each other: that is, do they or do they not reach the same judgments on the same issues?
Methods to select subjects for primary prevention All distinguished between primary and secondary prevention and all chose categorical indications for secondary prevention such as symptomatic vascular disease or diabetes or extreme elevations of low-density lipoprotein cholesterol (LDL-C) because all these clinical categories are acknowledged to be associated with subsequent high risk of a cardiovascular (CV) event. Specific definitions differed in some regards, such as when patients with type I diabetes mellitus (DM) were at high risk but these differences were minor. Accordingly, in this regard, the recommendations of all the guideline groups are concordant. However, considerable discordance was evident in the selection of subjects for primary prevention with lipid lowering therapy. (Table 1) Three of the six guidelines selected subjects for statin therapy for primary prevention based on calculated 10-year risk. The algorithms used differed but their operative features were the same. In SCORE, the algorithm used by EAS/ ESC, only fatal CV events were included, whereas in the others, non-fatal events were included as well. All the algorithms used had been validated although not necessarily in the populations to whom they were applied. Thus, the CCS guidelines for a multiethnic Canadian population calculated risk based on the Framingham algorithm, which was developed for a primarily white Caucasian population. Interestingly, none of the guidelines compared the performance of the algorithm they had
Table 1 – Primary Prevention is based on: Guideline
Primary
Supplementary
EAS/ESC
10 year—SCORE (Fatal CVD)
CCS
10 year Framinghahm + LDLC/non-HDL-C/apoB in Intermediate Risk 10 year pooled cohort equation 10 year QRISK2 Risk up to age 80 Risk factor Counting
Risk relative to a normal Cardiovascular age
ACC/AHA JBS3 IAS NLA
Lifetime Lifetime
P RO G RE SS IN CAR D IOV AS CU L A R D I SE A S ES 5 8 ( 2 01 5 ) 3 – 9
selected against the others. Three guidelines took significantly different approaches. The CCS guidelines divide subjects into low risk, intermediate risk and high risk based on calculated 10-year risk. LDL lowering therapy was not recommended in the former but was recommended in the latter. Intermediate risk was further stratified based on LDL-C: statin therapy was recommended in those with an LDL C ≥ 3.5 mmol/l and in those in whom either non- high-density lipoprotein cholesterol (nonHDL-C) or apolipoprotein (apo) B was increased above a specified level in those with an LDL C ≤ 3.5 mmol/l. By contrast, the IAS guidelines are based on risk up until age 80 rather than 10-year risk and the NLA guidelines were based on risk factor counting, not on risk as estimated by an algorithm that incorporates all the risk factors. Thus, CCS, IAS and NLA are each significantly discordant from each other and the rest. In contrast, all the guidelines acknowledged that for younger subjects, risk could be low in the short term but high in the longer term and five of the six identified secondary instruments, such as lifetime risk, that could be used for this purpose. These differed based on the guideline. Nevertheless, the majority recognized an important limitation of estimation of risk based on a 10-year window.
Non-invasive tests for risk assessment Table 2 lists the positions of the different guidelines on the merits of non-invasive tests to assess the risk of CV events. Based on consideration of costs, risks and benefits, the assessment of the evidence produced significantly discordant recommendations. At one end of the spectrum are the EAS/ESC guidelines with a positive stress echocardiogram or nuclear imaging or carotid plaque all denoting very high risk. Accordingly, all are taken as coronary heart disease (CHD) equivalents. Curiously, coronary calcification, the non-invasive test that arguably has been most extensively studied and validated is not listed. The CCS guidelines allow the results of carotid ultrasound, ankle brachial index and coronary calcification to be used for patients at intermediate risk. A positive coronary calcium is defined as an Agaston score ≥300 units whereas the NLA defines a positive Agaston score as ≥100 units. At the other end of the spectrum, three of the guidelines—ACC/AHA, JBS3 and IAS—recommend against the use at the present time of any non-invasive test to select subjects for preventive LDL-C lowering therapy. They judged that the balance of the evidence indicates such tests are either not worthwhile or too costly.
Table 2 – Non-invasive tests for risk assessment. Guideline
Test/Decision
EAS/ESC
Positive stress echo, nuclear imaging, carotid plaque: very high risk For intermediate risk patients, carotid ultrasound, ankle brachial index CAC ≥ 300 Agaston units None recommended None recommended None recommended CAC ≥ 100 Agaston units
CCS
ACC/AHA JBS3 IAS NLA
5
Accordingly, for this criterion, there is substantial discordance among the guidelines. Interestingly, no cost benefit analyses are cited by any of these guidelines for their conclusions. If the issue is treatment with statins of all at intermediate risk versus selection of those at intermediate risk with a positive calcium score, then broader statin therapy is the preferred choice.8,9 Neither was cited by any of the guidelines.
Is chronic kidney disease (CKD) a CHD risk equivalent? Sometime ago, diabetes mellitus was broadly accepted by guideline groups as equivalent in CV risk to symptomatic CV disease. The same argument is now being made for CKD. It is worth noting that whereas the evidence for clinical benefit with LDL-C lowering therapy is strong in patients with type 2 DM, that is not the case for those with end stage renal disease.10,11 Nevertheless, with relatively minor differences, three guidelines accept substantial reduction in glomerular filtration rate as a CHD risk equivalent (Table 3) while one, CCS, accepts albuminuria as well as a reduction in glomerular filtration rate as sufficient justification for statin LDL-C lowering therapy. Two, ACC/AHA and IAS, state the evidence is inadequate. None discuss whether statin therapy alone or the statin-ezetimibe regimen used in the SHARP trial12 should be the preferred regimen. Accordingly, overall, there is substantial discordance among the guidelines for this criterion also.
Which lipid measure should be used to determine the adequacy of the response to therapy? Four guideline panels chose LDL-C as the primary measure of the adequacy of the response to LDL lowering therapy whereas two chose non-HDL-C. (Table 4) Three of the four that chose LDL-C also chose non-HDL-C as an alternate to LDL-C and two of these also allow apoB as an additional option. None review in detail the evidence that determined their decision. There is significant discordance, therefore, for this recommendation also.
Consistency of treatment strategies among the guidelines The positions of the 6 guidelines are summarized in Table 5. Each takes a different position. CCS allows three markers to assess the adequacy of the treatment response with one level selected for each for subjects at intermediate and high risk
Table 3 – Is chronic CKD an indication for preventive statin therapy? Guideline
Renal Parameter
EAS/ESC CCS
CKD 3–5 CKD 30 (or ≥ 300 mg/day) Albuminuria, GFR of uncertain value CKD 3–5 Not worthwhile 3B (30–44 ml/min/1.73 m2 4 (15–29 ml/min/1.73 m2
ACC/AHA JBS3 IAS NLA
6
P RO G R E SS IN CAR D IOV ASC U LA R D IS E A S ES 58 ( 20 1 5 ) 3 – 9
whereas EAS/ESC, which also allows three markers, has three different targets for moderate risk, high risk and very high risk. Both also state a 50% reduction in LDL-C represents an adequate response although neither document the evidence in support of this recommendation. JBS3 selected a non-HDL-C target, but this target does not differ among the different risk categories whereas the NLA, which also favors non-HDL-C, mandates two intensities of therapy. The IAS favoured the concept of targets but did not select specific levels delegating that choice to national societies. The position of ACC/AHA is complex. The text at multiple points states that the statin randomized control trials were tests of regimens not of targets. Accordingly, almost all have concluded that ACC/AHA rejected targets, and this conclusion has attracted considerable attention and generated considerable controversy with several other guidelines rejecting this view and their participants or supporters confirming their own positions as to the validity of targets.13,14 The fact that recommendations that differ are always reaffirmed and never modified does not speak to the process as open and responsive. Importantly these replies do not note that the formal recommendation of ACC/AHA regarding targets—ACC/AHA Table 4 recommendation 1—is that they make no recommendation—either for or against—targets. This should mean that physicians are free either to use a target-based strategy or a regimen-based strategy. Nor do any note that the ACC/AHA guidelines do outline a strategy to assess adherence to therapy. Specific details are given as to when follow up lipid values should be obtained and what constitutes an adequate response (i.e. a 50% decrease in LDL-C from fasting levels) and what should be done if the response has been less than adequate (i.e. consider increasing the dose of the statin or adding an additional LDL lowering agent). This would appear to constitute a de facto target strategy. On this issue, therefore, not only is there substantial discordance among the guideline recommendations—and indeed, within the ACC/AHA guidelines—there is no evidence that this discordance can be analyzed and resolved.
Items the cholesterol guidelines agree about but without evidence So far, we have identified issues on which there is substantial discordance among the guidelines but there is another category of issue that should be briefly noted—namely, issues on which adequate evidence is absent but the guidelines agree and the absence of evidence is not noted. Such agreement points to Table 4 – Which lipid measure should be the primary index of the adequacy of the response to therapy? Guideline
Renal Parameter
EAS/ESC CCS ACC/AHA JBS3 IAS NLA
LDL-C primary; non-HDL-C/apoB alternate LDL-C primary; non-HDL-C/apoB alternate LDL-C Non-HDL-C LDL-C primary; non-HDL-C alternate non-HDL-C
shared core convictions among the participants. Among these are: (i) The statin RCTs were designed to test the benefit and risks of statin therapy for a limited period—generally not more than 5 years. Nevertheless, statin therapy is recommended without a time limit. As a practical matter that may be reasonable. However, such a recommendation is not evidence-based and this limitation is never noted. (ii) A number of guidelines recommend that LDL-C be directly measured rather than calculated by the Friedewald method, since LDL-C can be estimated no matter the triglyceride level. Not mentioned is that none of these methods are standardized and there is no evidence that that the direct measures provide superior care. Accordingly, there is no evidence they are cost-effective. Nevertheless, they are recommended. (iii) Maximal dose statin therapy is recommended for high risk subjects notwithstanding that there is no RCT evidence that maximal dose statin therapy is superior to moderate dose statin therapy. That is, atorvastatin 80 mg per day may produce a better clinical result than atorvastatin 10 mg per day but there is no evidence that atorvastatin 80 mg per day is superior to atorvastatin 40 mg per day or even to atorvastatin 20 mg per day.15 Doubling of statin dose produces only limited further reduction of LDL-C and thus would be expected to produce only limited further clinical benefit.15 Unfortunately, higher doses of statins are associated with more side effects, such as myalgia, and these side effects are associated with reduced adherence and increased mortality.16,17 Accordingly, these recommendations are based on opinion not evidence.
Conclusion and recommendations Notwithstanding the wealth of available RCT results and notwithstanding that all considered the same evidence and notwithstanding that all adopted an ‘evidence-based’ approach, the recommendations of these six cholesterol guidelines differ profoundly. With regard to these issues, the outcome of the guideline process was not replicable. This represents prima facie evidence that the process itself is not automatic and error-free. To be sure, we have compared only a few of the total number of recommendations but a review of each and every one would neither be practical nor necessary. Demonstrating discordance on several should suffice. That guideline recommendations are not replicable should not be surprising. They are the outcome of a deliberative process by a group of individuals, each with a history of achievements and failures, each with insights and biases, each with strengths and limitations of understanding and expression. The result is that the outcome—the recommendations that are generated—is not determined solely by the material that is examined but is also shaped by how the participants interpret that material, by how they interact with
7
P RO G RE SS IN CAR D IOV AS CU L A R D I SE A S ES 5 8 ( 2 01 5 ) 3 – 9
Table 5 – Guideline targets of therapy. Low Risk
Moderate Risk
CCS LDL-C
