Choosing Quality of Care Measures Based on the Expected Impact of Improved Care on Health Albert L. Siu, Elizabeth A. McGlynn, Hal Morgenstern, Mark H. Beers, David M. Carlisle, Emmett B. Keeler, Jerome Beloff, Kathleen Curtin, Jennifer Leaning, Bruce C. Perry, Harry P Selker, Warren Weiswasser, Andrew Wiesenthal, and Robert H. Brook This manuscript was collaboratively written with the HMO Quality of Care Consortium Ad Hoc Clinical Subcommittee. The Consortium consists of 12 managed care plans, in alphabetical order: AV-MED, CIGNA Health Plans, Community Health Care Plan, Group Health Cooperative of Puget Sound, Harvard Community Health Plan, The Health Care Plan of Buffalo, The Health Plan of America, Kaiser-Permanente Colorado Region, Kaiser-Permanente Northwest Region, Kaiser-Permanente Southern California Region, MedCenters Health Plan, and United Healthcare Corporation. This work was supported by the Hartford Foundation, the National Institute on Aging (Academic Award to Dr. Siu), and the participating plans. Albert L. Siu, M.D., M.S.P.H., lead author, is affiliated with the Health Sciences Program at RAND, Santa Monica, CA, and the UCLA School of Medicine, Department of Medicine, Los Angeles, CA. Elizabeth A. McGlynn, Ph.D. and Emmett B. Keeler, Ph.D. are in the Health Sciences Program at RAND. Mark H. Beers, M.D. and David M. Carlisle, M.D., M.P.H. are in the UCLA School of Medicine, Department of Medicine. Hal Morgenstern, Ph.D. is in the UCLA School of Public Health, Los Angeles, CA. Robert H. Brook, M.D., Sc.D. is in the Health Sciences Program at RAND; the UCLA School of Medicine, Department of Medicine; and the UCLA School of Public Health. Members of the Consortium Subcommittee are Jerome Beloff, M.D. of AV-MED Health Plan, Miami, FL; Kathleen Curtin, M.B.A. of The Health Care Plan, Buffalo, NY; Jennifer Leaning, M.D. of The Harvard Community Health Plan, Boston, MA; Bruce C. Perry, M.D. of Group Health Cooperative of Puget Sound, Seattle, WA; Harry P. Selker, M.D., M.S.P.H. of New England Medical Center, Boston, MA; Warren Weiswasser, M.D. (deceased) of The Community Health Care Plan, New Haven, CT; and Andrew Wiesenthal, M.D. of Kaiser-Permanente Colorado Region, Denver, CO. Address correspondence and requests for reprints to Albert L. Siu, M.D., M.S.P.H., RAND, P.O. Box 2138, Santa Monica, CA 90407-2138. This article, submitted to Health Services Research on July 3, 1990, was revised and accepted for publication on January 13, 1992.

620

HSR: Health Services Research 27:5 (December 1992)

Consumers, payers, and policymakers are demanding to know more about the quality of the services they are purchasing or might purchase. The information provided, however, is often driven by data availability rather than by epidemiologic and clinical considerations. In this article, we present an approach for selecting topics for measuring technical quality of care, based on the expected impact on health of improved quality. This approach employs data or estimates on disease burden, efficacy of available treatments, and the current quality of care being provided. We use this model to select measures that could be used to measure the quality of care in health plans, but the proposedframework could also be used to select quality of care measuresfor other purposes or in other contexts (for example, to select measures for hospitals). Given the limited resources available for quality assessment and the policy consequences of better information on provider quality, priorities for assessment efforts should focus on those areas where better quality translates into improved health.

Coinciding with policy initiatives to control health care costs through competition, policymakers have been increasing their attention in the last decade to the need to collect more information on the quality of care. Such information on quality could be used to regulate providers, or the information could be used by providers themselves to monitor and improve quality of care (Berwick 1989). Alternatively, the information could be used to stimulate competition by better informing consumers and major purchasers of health care (Office of Technology Assessment 1988; Davies and Ware 1988). This growing interest in measuring quality of care has spawned discussions about identifying the data that should be collected (Office of Technology Assessment 1988). A variety of approaches to measuring quality are available. One can use structural measures that describe the health care settings and providers of medical care, but these structural measures may not be sufficiently valid to detect clinically significant differences in quality. An alternative approach might be to measure patient satisfaction. Clearly, patient satisfaction is an important outcome of care; however, knowing about satisfaction alone yields only limited information about the appropriateness of resource-intensive technical aspects of care (the physician visits, diagnostic tests, and treatments). For information on these technical aspects of care, much attention has focused on quality of care indicators for which data are easily available. Thus, interest has concentrated on hospital mortality, read-

Choosing Quality of Care Measures

621

mission, and procedure volume as potential indicators of quality of care. While data availability or the feasibility of data collection are important, these should not be the central considerations in selecting indicators of quality. The selection of quality of care measures for public policy purposes should also be based on epidemiologic, medical, and health care delivery considerations. Ideally, the choice of measures should be based on knowledge of disease burden, efficacy of available treatments, the current type and amount of care being provided, the expected health impact of improvement in care, and the costs associated with making these improvements. Based on this information, one would then select those quality of care measures where improvement in care was associated with the greatest expected health impact for a given cost. Many of these considerations have been used in previous attempts to select topics for measuring the technical quality of care (Brook, Davies-Avery, Greenfield, et al. 1977). This earlier work is exemplified by Williamson's pioneering work on selecting priorities for quality assurance based on the expected health impairment associated with a medical condition, the preventability of this impairment through medical care, and the extent to which this prevention is not achieved (Williamson 1978; Williamson, Alexander, and Miller 1968). This article elaborates on such earlier work by proposing an approach that more explicitly specifies and quantifies the parameters and assumptions made in estimating the expected impact of improvements in care. We apply our approach to problems related to primary, secondary, and tertiary prevention, and we extend the approach to deal with overuse or inappropriate use of medical care. The methods also consider the expected impact of varying degrees of improved quality of care, and the approach can be extended to estimate the expected impact of compliance with specific quality of care indicators or criteria. The methods we describe can be applied in a variety of contexts. For illustration, we describe this process in the context of selecting quality of care measures that could be used in HMO health plans either for routine quality assurance activities, for continuous quality improvement (Berwick 1989), or for meeting the information needs of major purchasers. We use our estimates of the expected impact of improved care as our primary consideration in selecting topics for measuring quality. Then considering the context in which the measures will be used, we base a final choice of quality of care measures on other important considerations, such as cost effectiveness, relevance to the health care delivery setting, and feasibility.

622

HSR: Health Services Research 27:5 (December 1992)

CONCEPTUAL FRAMEWORK AND METHODS The quality of medical care has been defined in many ways. One definition considers quality of care as the difference "between efficacy and effectiveness that can be attributed to care providers" (Brook and Lohr 1985, 711). Efficacy here is defined as the benefits to patients achievable when medical technology is applied to a given dinical problem under ideal circumstances, and effectiveness is defined as the benefits achieved under more ordinary or actual circumstances. Considering this definition, interest in assessing the quality of care is sensible only when interventions of known efficacy exist for treating a given medical problem. If medical care for a given problem is known to be efficacious, reducing missed opportunities to apply the intervention (and/or improving the execution of the intervention) should lead to improvements in health. The missed opportunities to apply efficacious interventions are the major focus of this article, but other types of quality problems also merit consideration. If an intervention has no effects or potentially harmful effects, it would be more efficacious to desist from the intervention. Improved health would result from reducing these inappropriate interventions. Although reduced use of inappropriate care is not the major focus of this study, we illustrate the applicability of our approach to this situation as well. By these arguments, selection of clinical problems as topics for measuring quality of care should focus not only on the prevalence and health impact of a condition, but also on the efficacy (or lack thereof) of available treatments. Such information could be used to estimate the expected impact of improved care. To make such estimates, we began by identifying the major causes of mortality and morbidity in the United States. For each of the identified conditions, we asked a series of questions. Are there efficacious interventions for primary, secondary, or tertiary prevention of these causes of mortality and morbidity? If so, is the full efficacy of these interventions being realized? If not, what is the potential or expected impact of improved care for these conditions? Using data from Vital Statistics and the Health Interview Survey (Collins 1986), we identified the major sources of mortality and morbidity (limited activity days) for five age and gender groups: birth- 17, 18-44 (by gender), 45-64, and over 65 (Table 1). These data sources were chosen for their national scope and comprehensive detailed listing of medical problems. But the use of these surveys also had disadvan-

Choosing Quality of Care Measures

623

tages: for instance, because of reliance on physician- or self-reporting, these data sources underreport or unreliably classify a number of conditions (such as mental health problems, sexually transmitted diseases, dementia, incontinence, and osteoporosis). Additionally, we recognize that the impact of some important conditions (for example, sensory impairment) was reduced by the disaggregation of diseases in the surveys. With these considerations in mind, we added a number of additional conditions and health issues that were not originally identified (Table 1) in consultation with a panel of clinicians (authors J. B., K. C.,J. L., B. P., H. S., W. W., A. W.). For each of these conditions, we reviewed the medical literature to determine whether medical care or health-related behaviors were known to mitigate disease-specific outcomes. We first reviewed what was known about causation of each condition, what risk factors were associated with its development, and what was known about interventions for primary prevention -that is, the promotion of health by personal and communitywide efforts (Last 1983). We then considered the potential impact of secondary and tertiary prevention by reviewing Table 1: Major Causes of Mortality and Morbidity Identifiedfrom Vital Statistics and Household Interview Survey* Infant mortality and related conditions .(< 1) Otitis media (1-17) Asthma (1-17, 18-44) Accidents and injuries (1-17, 18-44, 45-64) Suicide (18-44) Acute respiratory conditions, including influenza (all ages) Breast cancer (18-44, 45-64) Back conditions (18-44, 45-64) Coronary artery disease (18-44, 45-64, 65+) Arthritis (18-44, 45-64, 65+) Chronic bronchitis and emphysema (45-64, 65+) Colorectal cancer (45-64, 65+) Lung cancer (45-64, 65+ ) Stroke and cerebrovascular disease (45-64, 65+) Diabetes (45-64, 65+) Pneumonia (65+) Additional Conditions Vaccine-preventable childhood infectious diseases Mental health problems Sexually transmitted diseases Dementia and incontinence Osteoporosis and hip fractures

Sensory impairment *Age categories in parentheses.

624

HSR: Health Services Research 27:5 (December 1992)

interventions known to reduce mortality or morbidity among those who have already developed the condition. Secondary prevention is the early detection and treatment of disease, and tertiary prevention is the reduction of impairment, disability, and suffering due to established illness (Last 1983). If the literature indicated that effective interventions existed for prevention, we estimated the impact of such interventions annually at the national level in the United States. Thus, for specified interventions, we estimated expected reductions in deaths, bed-days, or other adverse outcomes. Depending on the availability of data, similar estimates could be made at other population-based levels such a states, regions, or a given health plan as appropriate. These estimates were intended to illustrate the potential effect of individual medical interventions. Where more than one intervention was discussed for a given condition, the impact estimate was made independent of that for any other intervention(s) considered. Thus, attempts to combine impact estimates made for individual interventions may overestimate the magnitude of the total impact. Estimates of the potential impact of specific interventions were made using adaptations of previously described methods (Morgenstern and Bursic 1982; Ouellet et al. 1979; Browner 1986). For each intervention, we calculate an impact fraction (IF) or the expected proportional reduction in the occurrence of a disease or outcome following the intervention in a particular target population. For example, if the risk of an outcome event in the target population is reduced from 10 percent before an intervention to 8 percent after the intervention, the IF would be (0.10 - 0.08)/0.10 or 20 percent. (Note that the absolute reduction in risk is 0.10 - 0.08 or 2 percent.) From the vantage point of the target population, the IF summarizes the impact of a specific intervention given (a) how much of the population is exposed to a risk factor, (b) the risk associated with exposure, (c) the estimated incremental participation rate in the intervention, and (d) the efficacy of the intervention. If the intervention is designed to modify a causal risk factor (i.e., a factor that increases outcome risk), IF is estimated as A (1) f(P)(RR)(1 RR') IF=-(1 .ARR - 1)+1I -

where

f = the proportion of the population currently exposed to the risk factor; = P the proportion of the exposed population that is

Choosing Quality of Care Measures

625

expected to receive the intervention, that is, the participation rate; RR = the relative risk for a given outcome comparing the exposed to the nonexposed in the target population, that is, the risk in the exposed divided by the risk in the unexposed; and RR' = the relative risk (for the forementioned outcome) comparing exposed individuals participating in the intervention to exposed individuals not participating in the intervention, that is, the risk among exposed individuals participating in the intervention divided by the risk among exposed individuals not participating in the intervention.' If there is more than one exposure category (e.g., different categories of elevated serum cholesterol), we use an extension of this equation to estimate the impact of interventions that deal with exposure to more than one (k + 1) exposure category (indicated by the subscript i where i = 0 is the reference category) (Browner 1986): k

s f

(Pj)(RR5)(1 - RR'2) (2)

IF= k

fRRi If the risk factor under consideration is protective (i.e., RR < 1), equation 1 or 2 is used with the exposure categories reversed so thatf is the proportion of the population currently unexposed and P is the participation rate among the currently unexposed. Where the intervention involves the direct application of a protective risk factor (e.g., a vaccine), we assume that the intervention's benefit for newly treated patients is equal to the benefit for previously treated patients. In this special case, Equation 1 reduces to A

RR') RR') + RR'

f(P)(1

IF-f(1

-

-

(3)

where f is the proportion of the population currently unexposed (or untreated) and P is the participation rate among the currently unexposed or untreated. Apart from the validity of the four specified parameters, we make two additional assumptions. We assume that participation in the inter-

626

HSR: Health Services Research 27:5 (December 1992)

vention program is unrelated to outcome risk, and we assume that the only risk factor modified by the intervention is the one exposure under consideration. We make these assumptions for practical reasons. We have no information by which to estimate how participation might vary by risk category in most cases, and simultaneous consideration of multiple risk factors is limited by data availability in many cases. When these assumptions are not tenable, however, the multiple exposure category method can be modified to address these considerations. For example, the participation rate can be varied by risk category. Similarly, exposure to more than one risk factor can be addressed by separate exposure categories for combinations of risk factors. The estimated number of outcome events (per unit time) expected to be prevented by the intervention is the product of IF and the number of outcome events (e.g., incident cases or deaths) from that condition in the population. If only one subgroup of individuals (e.g., diabetics) with the given condition comprises candidates for the intervention, the expected number of outcome events prevented as a result is the product of the IF and the number of outcome events within that subgroup (e.g., the number of strokes in patients with diabetes). Estimates of exposure prevalences (f), risk-factor effects (RR), and intervention effects (RR') were obtained from the literature. If no estimate of these parameters could be obtained (primarily in the case of exposure prevalence and intervention effect), we made preliminary assumptions and tested the sensitivity of our results to alternative assumptions. After considering the expected impact of identified preventive strategies, we based decisions on the desirability of measuring quality of care on three issues that were context specific (i.e., in our particular case, specific to measuring quality of care in HMOs). First, if information was available in the literature, we considered whether or not the relevant improvements in medical care were known to be cost effective. Second, we considered whether the relevant improvements in quality of care could be influenced by the providers we were evaluating. Third, we considered the availability and feasibility of collecting quality of care information. This process was undertaken for all the conditions identified in Table 1. We present our overall conclusions and detailed discussions on four conditions that illustrate key issues.

Choosing Quality of Care Measures

627

RESULTS Based on the expected impact of improved care and on context-specific (i.e., measuring quality of care in HMOs) issues, we selected a number of conditions and topics for evaluating the quality of care within a group of HMO health plans (Table 2). We estimated that improved care for the conditions on this list could have a substantial impact on health, based on the prevalence of risk factors, the efficacy of relevant interventions, the level of current quality of care, and the incidence of adverse events (Table 3). A report with detailed discussions of all conditions is available from the authors (Siu, McGlynn, Beers, et al. 1992). The four conditions discussed now in detail illustrate the considerations that went into selecting this list of measures. INFANT MORTALITY AND RELATED CONDITIONS

Infant mortality is one of the traditional indicators of the health of the population. In the last decade, however, attention has focused on the problem of low birthweight (LBW) (less than 2,500 grams) infants, one of the major determinants of infant and particularly neonatal mortality and morbidity. The risk of mortality increases with decreasing birth weight so that LBW babies account for two-thirds of neonatal deaths (Behrman 1985). Surviving LBW infants are also at increased risk of neurodevelopmental handicaps, lower respiratory tract infections, learning disorders, behavior problems, and iatrogenic complications of neonatal intensive care unit interventions (McCormick 1985). Recent interest in reducing infant mortality is directed at the primary prevention of LBW births. Some maternal risk factors (such as race, education, and socioeconomic status) for LBW are not amenable to medical intervention. On the other hand, birth spacing and teenage pregnancy are examples of risk factors that might be modified by preconception planning. Further, the increased risks related to smoking, drinking, and failure to receive adequate prenatal care are thought be highly modifiable (Behrman 1985). Indeed, the early initiation of prenatal care is considered to be the most cost-effective strategy for improving pregnancy outcomes (Joyce, Corman, and Groosman 1988). The potential impact of interventions directed at primary prevention is likely to be large. Virtually unchanged since 1980, an estimated 20.6 percent of white and 38.9 percent of black women in the United States (1987) fail to receive prenatal care in the first trimester (National

628

HSR: Health Services Research 27.5 (December 1992)

Table 2: Topics Selected for HMO Quality Measurement Target Issue Prevention of low birth weight Childhood infectious disease Treatment of otitis media

Treatment of childhood asthma Breast cancer early detection Prevention of coronary artery disease Treatment of myocardial

infarction Treatment of diabetes mellitus Prevention of strokes Treatment of hip fractures Prevention of influenza Attention to the medical problems of the frail elderly Overuse of surgical procedures and prevention of

complications

General Content of the Quality of Care Measures Process measures to focus on the timeliness, frequency, and content of prenatal care Immunization rates (by age of 2) for measles, mumps, rubella, diphtheria, pertussis, tetanus, Haemophilus influenzae b Process measures to focus on the use of antibiotics for primary treatment and prophylaxis, use of ancillary diagnostic tests, and surgery Process measures to focus on timeliness and thoroughness of the treatment of asthma with special attention to counseling and self-care behaviors; functional outcomes (e.g., lost school days) adjusted for disease duration, age, and race Mammography rates among women over 50 stratified by age Rates of cholesterol screening by age and sex; adequacy of follow-up received for elevated cholesterols

Process measures to focus on timeliness of treatment, attention to recurrent angina, and appropriateness of medications Process and intermediate outcome measures to focus on access, glucose monitoring, patient education, eye screening, and cardiovascular risk factor modification Process measures to focus on hypertension screening, appropriateness of treatment, and blood pressure monitoring; blood pressure results adjusted for age, sex, initial blood pressure, comorbidity, and treatment duration Process measures to focus on preoperative evaluation, prevention of complications, and postoperative care; outcomes adjusted for age, race, sex, fracture type, and comorbidity Immunization rates among those over 65, stratified by age and residence Process measures to focus on appropriateness of medication regimens and attention to common geriatric problems (e.g., incontinence, confusion) Proportion of cardiac catheterizations, cholecystectomy, and hysterectomy performed for indications rated appropriate; complication rates adjusted for age, sex, and comorbidity

v

0

've

Cd

O.E:

O

>

CIS

0 $.. .0 4-0

0

biD 4 4) .*- 0 cd 0 v E C) 4) 0 0

o

0 C,4 4)

cd >

mu

cd

0 4-i -

W

N

4)

-Q.. O

'D cr

biO 0 bO

>-

o

bIO

O

OCO)

r. 0

4,

0

W

bO r. -

.8

0 10 --

5

(U

0 0

M

cd

.

(L) 0 v

4.j

I.-O -r-4

U u

0 0 bO - 0 0 V. 2 ->

0

00

0, u '.. 0 0 Lo

>

0

LO

u

0

C,' 0

u

0 >

0

0

1-4

1-1

O

v

V

2

*a

co

O

0 LO

E

la

W

4) > (u 0

Ca

L) &

c)

0

W

Cd

u

0 rn

4)

0

0 04 0

0 (U

0

-0 .2O

4.

0

:3

O

C02

'Z

-0

Cd

v O

rA

Cd

C. 4-i

cd

blo

4-1

Cld

4-J

0

U b4

CZ

CI4

0

0

cd

W

10

,O.(U

r.

ZZ,

4) >

.M

4-1

0 ;. 0

cn

V

>

:3

9

0

0

0

rA

0.

.0 U

U

.0 > u -0 u

O 4, 0

.:

4)

LL)

0

tjl Al

-0 0.)

g u

WI

u

8 Cd

C'I

LO C'4

Z

ul

u

Lto,

0 0

Cd

M-

tw

Cd

Lo

rA .0 -0 cd

M

cn

Al

Cd 0.

v 0 0

Cd

ce)

4)

-O

rA (=> 4-1

0

0

Cd

78

LO. C4. 0

-0

0-4

4)

u

U Z

0 I= O

0

0

0

0

0

.-

Cd u > -0 b4lo

cd

O >

-

co

>

cl

W

bO

C2 MC)

41

ro)

41

rn

0

O

4.1

rA

Q

.0 t:

0-0-4 E

0

-0

0

v 41

cd 0 O

S..

u

Cd

0 .-

0.

C)

2

cn

LO

(u

C.

X: 1. U 4-,

v 0 4

0

WO

CI4 0 4) C-4

Cd

0. 0

v

0

rA

>0

Its

0

Cd

v

00 CIS

C'4 s.

0 0

-4

wt

cd

bO SZ 0

co rA

.0 Cd

Cd

Cd

a

La. v 0 -

d zl, Q

o-

41

.50

5

rV

-

P

14

>s

~~~~~~~~O O

cn

Loo

.0i

o

>Z o

cd

2 t

O ;Ut

cdO

.o

Q8 e,= 4.

;3

X"

4q

i.-

_4 E

rA

0to

.o

E

E.

bi)

r. 0

r. >, 0

0 0

v

:s

'QrO

4

'-Zv

Q.

Cd

Cd

rn

r.

0

Cd

0.

4) u

LO

Cll Lo

v

4) rA

e

:

d

rn cn

00 C:

rA

O C

S..V12.

cd

CZ

C>

.

LO

'2

C>

cd

C,4

CZI

d

-0

u

d

Z

C;.

CZ N

(L)4

vo

-0 ct

o la

H

U.

r.)

td

W

r. 0

te:

10 0.) z r. -4 4-i r. 0

z z

.0

>

;3

>1

r.

S..

1,4

IVD

>dI >; -u *-0'

a

4d

E qd

C4.

r.

E E

W

0 0 °4°

*03 v Cw°C?3

e 0

54

4) 0 g) go

4.)E

a 4=.).

C0

o

r

C,

;0

0

4

'$¢

( 0

L)

0

V. 0LO

-0

0 .

4c -° 4

@ u= E

6;E 5 0C

w.E 0

-tEY

632

HSR: Health Services Research 27:5 (December 1992)

Center for Health Statistics 1990b). Based on a multivariate analysis of national data, timely and appropriate prenatal care is estimated to reduce the risk of LBW by 15 percent (i.e., RR' = 0.85) among white and by 12 percent among black women (Behrman 1985). Thus, we estimate that reducing the percentage of women failing to get prenatal care in the first trimester to 5 percent (P = .156/.206 = .757 for white women) could result in 2.7 percent2 or 4,521 fewer LBW births among white women, and 4.4 percent or 3,580 fewer LBW births among black women annually. A more modest goal of 10 percent failure in the first trimester would result in 3,050 fewer white and 3,050 fewer black LBW births. Methods for secondary prevention are also available. These methods include specialized treatment of high-risk infants in neonatal intensive care units and the establishment of specialized centers for the delivery of high-risk infants. The availability of such units is thought to be responsible for much of the reduction in neonatal mortality since 1960 (McCormick 1985). Most experts believe, however, that the major gains in secondary prevention have been achieved already, and that efforts should now be directed to primary prevention of LBW (Behrman 1985). Thus, in the prevention of infant mortality, quality of care measurement is best directed at the primary prevention of low birth weight and related problems, because the expected impact of improved neonatal care is likely to be small. Specifically, the proposed measures should focus on the timeliness, frequency, and content of prenatal care. Alternatively, one could measure outcomes such as rates of LBW births. Because these outcomes are known to be strongly affected by socioeconomic and other variables beyond the control of individual providers, evaluations of such outcomes need to be adjusted for complexity of the pregnancy (e.g., multiparity), maternal age, comorbidity, socioeconomic status, and ethnicity. CORONARY ARTERY DISEASE

Although mortality from coronary artery disease has declined in the United States in the last two decades, coronary disease and related diagnoses remain one of the major causes of mortality and morbidity among adults. Primary prevention of ischemic heart disease is based on modification of several well-established risk factors, including hypercholesterolemia, cigarette smoking, hypertension, obesity, and physical inactivity. Indeed, modification of risk factors (primarily changes in serum cholesterol and smoking) may have accounted for more than

Choosing Quality of Care Measures

633

half of the decline in mortality from 1968-1976 (Goldman and Cook 1984). Nevertheless, the potential for further improving health through risk factor modification remains large. An estimated 26.5 percent of American adults smoke (Centers for Disease Control 1987). Over onethird of the smokers seen in university internal medicine practices report never having been asked to stop smoking (Kosecoff, Fink, Brook, et al. 1985), and physicians (as recently as the mid-1980s) dismiss hypercholesterolemia as a problem in an estimated 71 percent of patients with cholesterol levels ranging from 221-260 mg/dl (Wynder, Field, and Haley 1986). The potential impact of improved primary prevention is substantial. Consider just one possible prevention strategy -reducing cholesterol by 10 percent for adults with values in the top quintile. Although there has been considerable controversy over the recommendations for broad-based cholesterol screening and treatment made by a recent national advisory body, screening and attempts to lower cholesterol in individuals with very high cholesterol is much less controversial (Olson 1989). Men in the top cholesterol quintile have a 3.4-fold relative risk of coronary death (i.e., RR = 3.4 andf = .2) compared to the lowest quintile (Martin, Hulley, Browner, et al. 1986), and reducing cholesterol by 10 percent is achievable and proportionally reduces the incidence of coronary heart disease by 20 percent (i.e., RR' = 0.80) (Lipid Research Clinics Program 1984; Lavie et al. 1988). If these results generalize to women as well, reducing cholesterol by 10 percent in half of high-risk individuals (i.e., P = .5) would reduce coronary mortality (510,000 cases in 1988 [National Center for Health Statistics 1990a]) by 3.5 percent3 or 17,850 cases annually (9,300 of which are men). Alternatively, getting just 10 percent of individuals in the top quintile to lower their cholesterol by 10 percent would also have important health effects, reducing coronary mortality by 0.7 percent or 3,570 cases (1,850 of which would be men). Progress has also been made in secondary and tertiary prevention where the potential for improvement in health exists at several levels. Lowering plasma cholesterol can slow the progression of established coronary artery disease (Blankenhorn, Nessim, Johnson, et al. 1987). Effective medications are available for the treatment of chronic (Strauss and Parisi 1988) and unstable angina (Theroux, Ouimet, McCans, et al. 1988). Improved prehospital care for acute myocardial infarctions has been shown to reduce mortality (Eisenberg, Bergner, and Hallstrom 1979, Crampton, Aldrich, Gascho, et al. 1975). Major advances have been made in both the acute (Wilcox, von der Lippe, Olsson, et al. 1988; ISIS Steering Committee 1987; ISIS-2 1988;

634

HSR: Health Seivices Research 27:5 (December 1992)

GISSI 1986, 1987; Yusuf, Peto, Lewis, et al. 1985; Goldman, Sia, Cook, et al. 1988) and postacute (Oldridge et al. 1988) treatment of myocardial infarction. Furthermore, a study suggests that some deaths among patients hospitalized for acute myocardial infarction may be preventable, perhaps by the more adequate treatment of postinfarction angina (Dubois and Brook 1988). Because of rapid technological changes, it is difficult to estimate the level of current use of some beneficial services. Nevertheless, the potential impact of secondary and tertiary prevention is also large. To illustrate, we consider the potential impact of two different treatments following myocardial infarction: thrombolytic therapy and betablockade. Studies have shown that thrombolytic therapy reduces mortality from hospitalized myocardial infarction (RR' = 0.76) (Yusuf, Wittes, and Friedman 1988). Thrombolytic therapy is perceived to be one of the more rapidly diffusing technologies ever introduced; however, it is probably still being underused, especially in smaller hospitals and in patients treated by generalist physicians (Tate and Dehmer 1989; Hlatky, Cotugno, O'Connor, et al. 1988). Inpatient mortality would otherwise be 12 percent (ISIS 1987) among the 15 percent of patients who are eligible for such treatment (Lee, Weisberg, Brand, et al. 1989). If we assume only 40 percent of eligible patients currently receive thrombolytic therapy (i.e., f = .6), we estimate that increasing use of thrombolytic treatment to 75 percent of eligible patients (i.e., P = (.60 - .25)/.60 = .58) would reduce mortality by 9.3 percent4 among eligible patients and reduce the annual number of deaths by 1,172. The expected impact of treating 75 percent of eligible patients with thrombolytic therapy is slightly larger or smaller depending on the level of current use (Table 4). Similarly, treatment with beta-blockers after myocardial infarction proportionally reduces the risk of death (RR' = 0.78) (Yusuf, Wittes, and Friedman 1988). Treatment with beta-blockers is contraindicated in some patients, and other patients will experience mild side effects from these drugs; however, approximately 50 percent of myocardial infarction patients are eligible for treatment (ISIS- 11986), and annual mortality is estimated to be 8 percent among these patients (Yusuf, Peto, Lewis, et al. 1985). If we assume that only 60 percent (i.e. ,f = .4) of eligible patients are currently treated,5 we estimate that 851 deaths6 could be averted annually by increasing the percentage of patients treated to 75 percent of eligibles [i.e., P = (.4 - .25)/.4 = .375]. The expected impact of improved use of beta-blockade is shown in Table 4 for different levels of current use. Thus, because the burden of coronary artery disease is great and

Choosing Quality of Care Measures

635

Table 4: The Expected Impact of Increasing Use of Thrombolysis or Beta-Blockade in Eligible Patients with Myocardial Infarction A Treating 75% of Eligible Patients with Thrombolytic Therapy Assumed percent of eligibles 20 currently receiving thrombolysis Impact fraction (%) 13.9 Prevented number of deaths 1,751 expected per year Treating 75% of Eligible Patients with Beta-Blockade Assumed percent of eligibles 40 currently receiving beta-blockade 8.4 Impact fraction (%) Prevented number of deaths 1,882 expected per year

Assumptions B

C

40

60

9.3 1,172

4.2 529

60

70

3.8 851

1.3 291

many clinical studies are available, we are able to estimate that the expected impact of improved care is large in primary prevention and in secondary and tertiary prevention; the choice of topics for this condition depends largely on the context for these quality assessment efforts. If one is interested in studying hospitals, the estimates just cited indicate that the potential impact of improved hospital care is sufficiently great to warrant such efforts. If, instead, one is interested in health plans, the choice is wider and one can focus on various aspects of either or both primary prevention and hospital care. BREAST CANCER

Breast cancer is one of the leading causes of death among women in the United States, with nearly 40,000 deaths occurring yearly. Unfortunately, primary prevention is not feasible. Although several risk factors for developing breast cancer have been identified, most of the important risk factors (previous breast cancer, family history of breast cancer, nulliparity and late first pregnancy) are not modifiable. On the other hand, early detection of breast cancer is feasible and beneficial. Evidence on the efficacy of breast self-examination is far from conclusive, but studies have shown that women who perform breast self-examination present with cancers at earlier stages (Hill et al. 1988) and have higher survival rates (Huguley et al. 1988). The efficacy of mammography in specific groups of women has been more

636

HSR: Health Services Research 27:5 (December 1992)

convincingly demonstrated in a number of studies (Tabar, Fagerberg,

Gad, et al. 1985; Verbeek, Hendriks, Holland, et al. 1984; Andersson, Aspergren, Janzon, et al. 1988; UK Trial of Early Detection of Breast Cancer Group 1988). The most widely cited work was a randomized trial at the Health Insurance Plan of New York (HIP) that showed that mammography reduced breast cancer mortality by 30 percent in women over 50 (Shapiro 1977). Although mammography may be effective in women under 50, the benefits are smaller and the decision to screen is likely to be based much more on a patient's risk factors and preferences (Eddy and Hasselbald 1988). With respect to tertiary prevention, the primary treatment of early breast cancer is surgical (axillary dissection with either mastectomy or local excision with radiation) (Veronesi, Saccozzi, Del Vecchio, et al. 1981; Fisher, Bauer, Margolese, et al. 1985). Although these surgical alternatives may have different effects on psychosocial function (Steinberg, Juliano, and Wise 1985), mortality is equivalent for either approach. After surgery, adjuvant therapy is useful in some patients. Available evidence indicates that adjuvant chemotherapy is effective in prolonging survival of premenopausal women with cancer extending to the axillary lymph nodes (Stockdale 1988; Bonadonna and Valagussa 1987), but while a number of recent trials have indicated that adjuvant therapy for node-negative cancer reduces recurrence, no impact has yet been found on overall survival (Ludwig Breast Cancer Group 1989; Fisher et al. 1989a; Fisher, Constantino, Redmond, et al. 1989; Mansour, Gray, Shatila, et al. 1989). Because of ongoing advances in adjuvant chemotherapy, it is difficult at this time to estimate the impact of improved treatment of breast cancer. If we consider the one subgroup where adjuvant therapy is most indicated, however, the impact of improved care can be shown to be relatively small on a national basis. Adjuvant chemotherapy proportionally reduces mortality by 24 percent (i.e., RR' = 0.76) in pre,menopausal women with involved axillary nodes (Bonadonna and Valagussa 1987). Increasing the proportion of these women getting appropriate adjuvant chemotherapy by 20 percent would reduce annual mortality from breast cancer by only 5.4 percent7 among women who presented with positive nodes, or by about 100 cases nationally (Table 5). By contrast, the potential impact of secondary prevention through early detection may be substantial on a national basis. Currently, only 50 percent of women seeing physicians report having been taught how to perform self-examination (Brook, Fink, Kosecoff, et al. 1987). Only 20 percent of women over 50 are screened regularly with mammogra-

Choosing Quality of Care Measures

637

Table 5: The Expected Impact of a 20 Percent Increase in Adjuvant Chemotherapy Use in Premenopausal Women with Breast Cancer Extending to the Axilla Assumptions

At

B C 70 50 30 Assumed percent currently receiving adjuvant treatment 5.4 5.8 5.2 Impact fraction (%) 103 111 99 Prevented number of deaths expected per year* *There are 3,826 breast cancer deaths among women under 45 (National Center for Health Statistics 1990a). We assume that 50 percent of these women present with positive nodes, because 50 percent of women who do not do breast self-examination present with positive nodes (Hill et al. 1988). tSimilar results are obtained for the use of adjuvant therapy in node-negative cancer. Extrapolating from National Surgical Adjuvant Breast Project (NSABP) trial results (Fisher, Redmond, Dimitrov, et al. 1989; Fisher, Costantino, Redmond, et al. 1989), increasing the proportion of women receiving adjuvant chemotherapy from 30 to 50 percent would prevent 85 recurrences among premenopausal women and 67 recurrences among postmenopausal women annually.

phy (i.e., f = .8) (Hayward et al. 1988), and increasing screening levels to over 50 percent is feasible (Nattinger, Panzer, and Janus 1989). Extrapolating from the HIP study (i.e., RR' = 0.7), we estimate that increasing to 50 percent the proportion of women over 50 who undergo screening mammography (i.e., P = (.8-.5)/.8 = .375) would reduce breast cancer mortality by 9.6 percent8 or 2,400 cases annually. Alternatively, increasing the proportion screened to 75 percent would reduce mortality by 17.6 percent or 4,400 cases. At first glance, the estimates of impact might suggest focusing quality measurement on the early detection of breast cancer. Because mammography involves more women, however, increasing mammography rates can require greater resources than increasing the use of adjuvant chemotherapy in the relatively smaller number of women diagnosed with breast cancer. Indeed, recent studies have suggested that adjuvant chemotherapy in some cases may be more cost effective than screening mammography (Hillner and Smith 1991; Eddy 1989). Thus, depending on the context and purpose of quality assessment, it might be reasonable to focus on either mammography or breast cancer treatment. A cancer hospital or referral center interested in monitoring quality might focus on the use of adjuvant therapy. On the other hand, a health plan with an enrollment of 100,000 would have only about 60 cases of breast cancer in a given year and only a

638

HSR: Health Services Research 27:5 (December 1992)

small fraction of these cases would be candidates for adjuvant chemotherapy (at least for those indications where it has been proved beneficial). In this case, the health plan might focus on mammography rates as a more routine quality assessment or continuous improvement activity. Meanwhile, the health plan could use a more ad hoc case-by-case peer review approach to monitoring quality in breast cancer treatment. COLORECTAL CANCER

Annually, 138,000 new cases of colorectal cancer are diagnosed, and 60,000 deaths occur. Commensurate with its public health impact, a large number of studies have looked at the prevention, early screening, and treatment of colon cancer. Although several dietary risk factors (Zaridze 1983) for colorectal cancer have been identified, much of the evidence is inconclusive and the efficacy of diet modification has yet to be demonstrated. Thus, primary prevention is theoretically possible but not scientifically confirmed at this time. Most efforts at secondary prevention of colorectal cancer have focused on screening and early detection of cancer or premalignant adenomas. Screening can be beneficial for both reducing disease incidence (by removal of premalignant lesions) and improving disease stage at presentation (and hence improving survival). Several screening strategies have been advocated; however, data on the efficacy of screening are inconclusive. Preliminary findings from trials of fecal occult-blood testing suggest that testing results in the diagnosis of a greater proportion of new cancers at an earlier stage (Winawer, Leidner, Miller, et al. 1977; Hardcastle, Thomas, Chamberlain, et al. 1989), but no results are available on mortality. The efficacy of screening sigmoidoscopy is suggested by the results of two large, nonrandomized studies (Gilbertsen and Nelms 1978; Crespi, Weissman, Gilbertsen, et al. 1984). In one study of 21,150 patients screened with sigmoidoscopy, Gilbertsen and Nelms (1978) found that the incidence of colorectal cancer was only 15 percent of what might have been anticipated in the absence of sigmoidoscopy (or an 85 percent reduction). However, this is probably an overestimate of the efficacy of screening: if one includes cases identified in the study's initial screening examination, the reduction in incidence is actually only 56 percent. Because of design and methodologic flaws, the true reduction in incidence is likely to be even smaller (Selby and Friedman 1989). Thus, estimates of the expected impact of improved screening for colorectal cancer would be highly speculative.

Choosing Quality of Care Measures

639

Table 6: The Expected Impact of Improved Care for Colorectal Cancer Increased Use of Adjuvant Chemotherapy Assumed percent to be treated Impact fraction (%o) Prevented number of cases expected per year

A

Assumptions B

C

30 10.2 1,020

50 16.5 1,650

70 23.8 2,380

With respect to tertiary prevention, the primary treatment of colorectal cancer is surgical. Attempts to improve outcomes after surgery have focused on testing various combinations of adjuvant radiotherapy and chemotherapy. A review of trials of adjuvant treatment suggests that benefits, if any, are likely to be quite small (Buyse, ZeleniuchJacquotte, and Chalmers 1988). Very recently, however, one form of adjuvant chemotherapy has been shown to reduce mortality proportionally by 33 percent (i.e., RR' = 0.67) for patients with Stage C colorectal cancer (i.e., cancer extending to the local lymph nodes) at the time of surgery (Laurie, Moertel, Fleming, et al. 1989; Moertel, Fleming, Macdonald, et al. 1990). The expected impact of the use of these adjuvant chemotherapy approaches is likely to be reasonably large. An estimated 21,000 patients have surgery for Stage C colorectal cancer annually (Moertel, Fleming, Macdonald, et al. 1990), and five-year survival for these patients ranges from 45-60 percent depending on the extent of lymph node involvement (Mayer et al. 1989). Hence, we estimate that patients initially diagnosed as Stage C account for approximately 10,000 of the 60,000 annual deaths from colorectal cancer. Administering appropriate adjuvant therapy to 50 percent of Stage C patients would reduce colorectal cancer deaths by 1,650 cases9 annually, assuming that only a negligible number of patients were receiving this relatively newly described treatment until very recently. The estimated impact of alternative assumptions of improvement are shown in Table 6. Thus, the expected impact of improved screening (by fecal occult blood testing or any other method) is very uncertain, and assessing quality of care relative to screening is not recommended at this time. On the other hand, if one is interested in measuring quality of care particularly as it relates to the diffusion of new therapies, assessing quality of care for newly diagnosed cases of Stage C colorectal cancer would be a reasonable approach.

640

HSR: Health Services Research 27:5 (December 1992)

COMMENTS ON OTHER CONDITIONS

On the subject of our final list of conditions (Table 2), a number of issues merit special comment. First, our illustrations thus far have focused on increasing the use of interventions that are underused, but the same framework could be extended to include overuse or performance of services for clinically inappropriate indications. For example, an estimated 32 percent of carotid endarterectomies are performed for inappropriate indications, and major postoperative complications (either death within 30 days or stroke with residual deficit) occur in 10.8 percent of these inappropriate procedures (Winslow, Solomon, Chaissin, et al. 1988). The volume of carotid endarterectomies performed in the United States reached a peak of 107,000 procedures in 1985 (Pokras 1987); hence, we estimate that 3,700 (107,000 x 0.108 x 0.32) major postoperative complications occurred in conjunction with inappropriate procedures in 1985. In many of these inappropriate surgeries, the short-term risk of death or stroke without surgery is very much lower than the risk with surgery (Chambers and Norris 1984; Warlow 1984), and we conservatively assume that the risk of shortterm complication reaches only as high as 5 percent (i.e., RR' = 0.05/ 0.108) with nonsurgical management, or about half the surgical complication rate. Receipt of appropriate nonsurgical management by half of the patients who would otherwise have had inappropriate surgery (i.e., P = .5 andf = 1) would prevent 26.8 percent or 992 "postoperative" strokes or deaths annually. If the short-term nonsurgical complication rate is only 2 percent, instead, eliminating half of inappropriate surgeries would prevent 1,508 strokes or deaths. Because many of the parameters required for making similar estimates are not available in the literature for numerous procedures and conditions, we could not make formal calculations of expected impact in many cases. Nevertheless, because we wished to balance considerations of underuse and overuse in our work, we included topics and measures focusing on appropriate use of procedures on our list of proposed measurement areas (Table 2). Second, a number of important health problems are not included in Table 2. In some cases, this exclusion is due to insufficient evidence on the efficacy of interventions to improve outcomes. For example, although suicide is one of the leading causes of death among young men and women, there is little evidence on the efficacy of methods to prevent suicide (Eddy, Wolpert, and Rosenberg 1987). Similarly, back pain is the most common cause of chronic physical limitation among adults aged 17 to 45 (HIS), but evidence on the efficacy of methods of

Choosing Quality of Care Measures

641

primary prevention or treatment of back pain is insufficient (Deyo 1983). Either because of feasibility or relevance, a few conditions were excluded from our final list even though the expected impact of improved care was potentially great. We did not include accidents and injuries because the health plans we wished to evaluate had limited or varying roles in the primary prevention of accidents and in prehospital care for victims of trauma. We did not include smoking cessation counseling to prevent lung cancer at this time, because of feasibility problems in ascertaining the occurrence of counseling by either chart review or patient recall. Finally, mental health problems and sexually transmitted diseases were not included because of problems (privacy concerns, among them) in identifying individuals with these conditions in a population.

DISCUSSION There is growing interest in holding providers more accountable for the quality of care they deliver. In this article, we propose an approach for selecting topics for measuring technical aspects of quality, and we apply this approach to selecting quality of care measures for possible use in HMO health plans. Based on epidemiologic considerations, the process we propose can be used to narrow the conditions and topics being considered. Then, depending on the goals of the quality measurement process, the population being measured, and the degree of feasibility, the list of possible topics can be further narrowed. One needs to consider whether the providers being evaluated have the power to provide the relevant interventions that might be expected to improve outcomes. For example, if an organization contracts with another group for inpatient services, it may not be appropriate to focus on topics related to inpatient care. On the other hand, if that organization has the ability to influence future contracting decisions, it may be more desirable to select topics dealing with inpatient care. Similarly, one needs to consider the cost effectiveness of the proposed improvements in care. Our work is limited in that we could not select those medical conditions where improvement in care has the greatest expected impact for a given cost. Unfortunately, this is difficult to do for two reasons. First, most clinical studies fail to use or give enough information to enable construction of a common measure of effectiveness (e.g., quality-adjusted life years) so that different types of outcomes can be evaluated across a variety of diseases and treatments.

642

HSR: Health Services Research 27:5 (December 1992)

We attempted in this project to use a common unit of benefit based on the Index of Well-Being (Kaplan and Anderson 1988), but we were not successful in most cases because of insufficient information in the clinical literature; sufficiently detailed information on symptoms frequently was unavailable. Rather than reporting results only for the most commonly reported outcome (i.e., mortality), we elected to report on a variety of disease-specific outcomes because mortality was not very relevant for a number of medical problems. Although different types of outcomes are admittedly sometimes difficult to compare, the outcome estimates are explicit and available for others to consider, weigh, and discuss. In addition to difficulty in obtaining common units of effectiveness, data on the costs of treatment are rarely inclu5led in clinical trials. Although it is possible to perform cost-effectiveness analyses for many of these interventions using other data sources, this is an unreasonable undertaking for an activity encompassing a large number of medical diagnoses and conditions. Because of these problems, we relied on published studies of the cost effectiveness of various interventions as a guide to selecting conditions. To deal with cost effectiveness more explicitly we propose, as part of collecting quality of care data, that information be collected on the costs of the care provided. This information can then be used retrospectively to estimate the cost effectiveness of incremental improvements in quality of care. The process we advocate in this article is iterative. Estimates and the appropriateness of quality measurement topics need to be reassessed as information becomes available on the cost effectiveness of incremental improvements in quality. As new clinical studies become available (as in the colorectal cancer example), estimates must be revised. Indeed, our proposed approach provides a method and context for estimating the public health impact of newly reported therapies. For example, one can easily compare the relative expected impact of the newly advocated adjuvant chemotherapies for breast versus colon cancer. Estimates and choices need to be revised as new technologies and therapies are developed, but also as new information becomes available on established therapies. Even though we believe sufficiently strong evidence is available, there are those who question the efficacy of coronary disease prevention (McCormick and Skrabanek 1988), early detection of breast cancer (Warren and Skrabanek 1989), and aggressive diagnosis and treatment of hypertension (O'Brien and O'Malley 1988; Cruickshank, Thorp, and Zacharias 1987). Thus, the process for selecting quality of care measures must be as current as the

Choosing Quality of Care Measures

643

clinical evidence on which it is based and as responsive as the providers whose care is being evaluated. Finally, although we have illustrated the use of this epidemiologic model in choosing topics for quality measurement, the same methods could also be extended to choose the specific criteria and indicators used to assess technical quality. For example, one could estimate the expected effect of a two-hour delay by a provider in administering thrombolytic therapy, if one were considering the use of this issue as a specific quality of care criterion. Evidence from clinical studies to support efficacy estimates may be more difficult to find for these more detailed criteria, and subjective estimates (Robins et al. 1985) may need to be substituted. However, assumptions of efficacy are implicit whenever such specific criteria are selected, and the proposed approach only stipulates that these estimates be made explicit to aid in the discussion of the merits of specific quality of care criteria. Quality of care measurement is done to identify medical care delivery problems that can be remedied. Knowing this information presumably permits appropriate steps to be taken to remedy those problems. For this reason, we have argued in this article that one should select topics for measurement based on the expected impact of these quality assessment and assurance efforts. Although precise estimates of the expected impact of improved quality are not always possible, enough is known to incorporate epidemiologic and clinical evidence into public policy decisions about which aspects of quality of care should be measured.

NOTES 1. RR' reflects the efficacy of the intervention. Specifically, "efficacy" (i.e., the proportion of potential outcome events that is prevented by the intervention under ideal conditions) is 1 -RR'. "Relative efficacy" (i.e., the extent to which full participation in the intervention reduces risk to the level of those who were never exposed) is [RR (1 - RR')]/(RR - 1). 2. A 0.206(0.757)(1 - 0.85) IF =-=002 0.206(1 - 0.85) + 0.85 3. RR for coronary heart disease death for individuals in the five serum cholesterol quintiles (lowest to highest) is 1.0, 1.3, 1.7, 2.2, and 3.4, respectively (Martin, Hulley, Browner, et al. 1986). Thus, A 0 + 0 + 0 + = 0.2(0.5)(3.4)(1 - 0.8) 0.035 IF = . 01. 2

0.2(1.0) + 0.2(1.3) + 0.2(1.7) + 0.2(2.2) + 0.2(3.4)

644

HSR: Health Services Research 27:5 (December 1992)

4.

0.6(0.583)(1 - 0.76) = 0.093 0.6(1 - 0.76) + 0.76 5. In a nationally representative sample of Medicare patients with acute myocardial infarction in the mid-1980s, beta-blockers were received at some point during the hospitalization in 27 percent of patients (Rubenstein 19 ) (or 54 percent of eligible patients if we assume that half the patients had contraindications to treatment). Younger patients may be more likely to be treated with beta-blockers; thus, we assume that 60 percent of patients (both Medicare and younger patients) are currently receiving beta-blockers post-MI. 6. A 0.4(0.375)(1 - 0.78) = 0.038 IF = 0.4(1 - 0.76)+0.76 7. Assuming 50 percent are currently receiving treatment (i.e., P = (0.5 - 0.3)/0.5 = 0.4), A 0.5(0.4)(1 - 0.76) = 0.054 IF= 0.5(1 - 0.76) + 0.76 8. There are approximately 25,000 deaths from breast cancer in women aged 50-75 (National Center for Health Statistics 1990a) and A 0.8(0.375)(1 - 0.70) = 0.096 IF = 1.8(1 - 0.70) + 0.70 9. A 1.0(0.50)(1 - 0.67) = 0.165 IF 1.0(1 - 0.67) + 0.67 A

IF=

REFERENCES Andersson, I., K. Aspergren, L. Janzon, T. Landberg, K. Lindholm, F. Linell, 0. Liungberg, J. Ranstam, and B. Sigfusson. "Mammographic Screening and Mortality from Breast Cancer: The Malmo Mammographic Screening Trial." British MedicalJournal 297 (15 October 1988): 943-48. Behrman, R. E. Chairman, Committee to Study the Prevention of Low Birthweight. Preventing Low Birthweight. Washington, DC: Institute of Medicine, National Academy Press, 1985. Berwick, D. M. "Continuous Improvement as an Ideal in Health Care." New England Journal of Medicine 320, no. 1 (1989): 53-56. Blankenhorn, D. H., S. A. Nessim, R. L. Johnson, M. E. SanMarco, S. P. Azen, and L. Cashin-Hemphill. "Beneficial Effects of Combined Colestipol-Niacin Therapy on Coronary Atherosclerosis and Coronary Venous Bypass Grafts." Journal of the American Medical Association 257, no. 23 (1987): 3233-40.

Choosing Quality of Care Measures

645

Bonadonna, G., and P. Valagussa. "Current Status of Adjuvant Chemotherapy for Breast Cancer." Seminars in Oncology 14, no. 1 (1987): 8-22. Brook, R. H., A. Davies-Avery, S. Greenfield, L. J. Harris, T. Lelah, N. E. Solomon, and J. E. Ware, Jr. "Assessing the Quality of Medical Care Using Outcome Measures: An Overview of the Method." Medical Care 15 (1977 supplement): 1-84. Brook R. H., A. Fink, J. Kosecoff, L. S. Linn, W. E. Watson, A. D. Davies, V. A. Clark, and C. Kamberg. "Educating Physicians and Treating Patients in the Ambulatory Setting." Annals of Internal Medicine 107, no. 3 (1987): 392-98. Brook, R. H., and K. N. Lohr. "Efficacy, Effectiveness, Variations and Quality Boundary-Crossing Research." Medical Care 23, no. 5 (1985): 710-22. Browner, W. S. "Estimating the Impact of Risk Factor Modification Programs." American Journal of Epidemiology 123, no. 1 (1986): 143-51. Buyse, M., A. Zeleniuch-Jacquotte, and T. C. Chalmers. "Adjuvant Therapy of Colorectal Cancer. Why We Still Don't Know."Journal of the American Medical Association 259, no. 24 (1988): 3571-78. Centers for Disease Control. U.S. Department of Health and Human Services. "Cigarette Smoking in the United States, 1986." Morbidity and Mortality Weekly Report 36 (11 September 1987): 581-85. Chambers, B. R., andJ. W. Norris. "The Case against Surgery for Asymptomatic Carotid Stenosis." Stroke 15, no. 6 (1984): 964-67. Collins, J. G. "Prevalence of Selected Chronic Conditions: United States, 1979-81 " In Vital and Health Statistics Series 10, no. 155. DHHS Publication no. (PHS) 86-1583. National Center for Health Statistics, Public Health Service. Washington, DC: U.S. Government Printing Office, July 1986. Crampton, R. S., R. F. Aldrich, J. A. Gascho, J. R. Miles, Jr., and R. Stillerman. "Reduction of Prehospital, Ambulance and Community Coronary Death Rates by the Community-Wide Emergency Cardiac Care System." American Journal of Medicine 58 (February 1975): 151-65. Crespi, M., C. S. Weissman, V. A. Gilbertsen, S. J. Winawer, and P. Sherlock. "The Role of Proctosigmoidoscopy in Screening for Colorectal Neoplasia." CA-A CancerJournalfor Clinicians 34, no. 3 (1984): 158-66. Cruickshank, J. M., J. M. Thorp, and F. J. Zacharias. "Benefits and Potential Harm of Lowering High Blood Pressure." Lancet 1 (March 1987): 581-83. Davies, A. R., and J. E. Ware, Jr. "Involving Consumers in Quality of Care Assessment." Health Affairs 7, no. 1 (Spring 1988): 33-48. Deyo, R. A. "Conservative Therapy for Low Back Pain. Distinguishing Useful from Useless Therapy."Journal of the American Medical Association 250, no. 8 (1983): 1057-62. Dubois, R. W., and R. H. Brook. "Preventable Deaths: Who, How Often, and Why?" Annals of Internal Medicine 109 (October 1988): 582-89. Eddy, D. M. "Screening for Breast Cancer." Annals of Internal Medicine 111, no. 5 (1989): 389-99. Eddy, D. M., R. L. Wolpert, and M. L. Rosenberg. "Estimating the Effectiveness of Interventions to Prevent Youth Suicides." Medical Care 25, no. 12 (1987): S57-S65.

646

HSR: Health Services Research 27.5 (December 1992)

Eddy, D. M., and V. Hasselbald. "The Value of Mammography Screening in Women under Age 50 Years." Journal of the American Medical Association 259, no. 10 (1988): 1512-19. Eisenberg, M., L. Bergner, and A. Hallstrom. "Paramedic Programs and Out-of-Hospital Cardiac Arrest: II. Impact on Community Mortality." American Journal of Public Health 69, no. 1 (1979): 39-42. Fisher, B., C. Redmond, N. V. Dimitrov, D. Bowman, S. Legautt-Poisson, D. L. Wickerham, N. Wolmark, E. R. Fisher, R. Margolese, C. Sutherland, A. Glass, R. Foster, and R. Caplan. "A Randomized Clinical Trial Evaluating Sequential Methotrexate and Fluorouracil in the Treatment of Patients with Node-Negative Breast Cancer Who Have Estrogen-Receptor-Negative Tumors." New England Journal of Medicine 320, no. 8 (1989): 473-78. Fisher, B., J. Costantino, C. Redmond, R. Poisson, D. Bowman, C. Couture, N. V. Dimitrov, N. Wolmark, D. L. Wickerham, E. R. Fisher, R. Margolese, A. Robidoux, H. Shibata, J. Terz, A. H. G. Paterson, M. I. Feldman, W. Farrar, J. Evans, H. L. Lickley, and M. Ketner. "A Randomized Clinical Trial Evaluating Tamoxifen in the Treatment of Patients with Node-Negative Breast Cancer Who Have EstrogenReceptor-Positive Tumors." New England Journal of Medicine 320, no. 8 (1989): 479-84. Fisher, B., M. Bauer, R. Margolese, R. Poisson, Y. Pilch, M. Deutsch, E. Montague, E. Saffer, L. Wickerham, H. Lerner, C. Redmond, E. Fisher, N. Wolmark, A. Glass, H. Shibata, P. Deckers, A. Ketcham, R. Oishi, and I. Russell. "Five-Year Results of a Randomized Clinical Trial Comparing Total Mastectomy and Segmental Mastectomy with or without Radiation in the Treatment of Breast Cancer." New EnglandJournal of Medicine 312, no. 11 (1985): 665-73. Gilbertsen, V. A., and J. M. Nelms. "The Prevention of Invasive Cancer of the Rectum." Cancer 41, no. 3 (1978): 1137-39. Goldman, L., and E. F. Cook. "The Decline in Ischemic Heart Disease Mortality Rates: An Analysis of the Comparative Effects of Medical Interventions and Changes in Lifestyle." Annals of Internal Medicine 101, no. 6 (1984): 825-36. Goldman, L., S. T. B. Sia, E. F. Cook, J. D. Rutherford, and M. C. Weinstein. "Costs and Effectiveness of Routine Therapy with Long-Term Beta-Adrenergic Antagonists after Acute Myocardial Infarction." New EnglandJournal of Medicine 319, no. 3 (1988): 152-57. Gruppo Italiano per lo Studio della Streptochinasi Nell'infarto Miocardico (GISSI). "Effectiveness of Intravenous Thrombolytic Treatment in Acute Myocardial Infarction." Lancet 1, no. 22 (February 1986): 397-401. . "Long-Term Effects of Intravenous Thrombolysis in Acute Myocardial Infarction: Final Report of the GISSI Study." Lancet 1, no. 17 (October 1987): 871-74. Hardcastle, J. D., W. M. Thomas, J. Chamberlain, G. Pye, J. Sheffield, T. W. Balfour, N. C. Armitage, P. D. James, S. S. Amar, and S. M. Moss. "Randomised, Controlled Trial of Faecal Occult Blood Screening for Colorectal Cancer. Results for First 107,349 Subjects." Lancet (27 May 1989): 1160-64. Hayward, R. A., M. F. Shapiro, H. E. Freeman, and C. R. Corey. "Who

Choosing Quality of Care Measures

647

Gets Screened for Cervical and Breast Cancer?" Archives of Internal Medicine 148 (May 1988): 1177-81. Hill, D., V. White, D. Jolley, and K. Mapperson. "Self Examination of the Breast; Is It Beneficial? Meta-Analysis of Studies Investigating Breast Self Examination and Extent of Disease in Patients with Breast Cancer." British MedicalJournal 297 (23 July 1988): 271-75. Hillner, B. E., and T. J. Smith. "Efficacy and Cost Effectiveness of Adjuvant Chemotherapy in Women with Node-negative Breast Cancer." New England Journal of Medicine 324, no. 3 (1991): 160-68. Hlatky, M. A., H. Cotugno, C. O'Connor, D. B. Mark, D. B. Pryor, and R. M. Califf. "Adoption of Thrombolytic Therapy in the Management of Acute Myocardial Infarction." Amnican Journal of Cardiology 61 (1 March 1988): 510-14. Huguley, C. M., R. L. Brown, R. S. Greenberg, and W. S. Clark. "Breast Self-Examination and Survival from Breast Cancer." Cancer 62, no. 5 (1988): 1389-96. ISIS Steering Committee. "Intravenous Streptokinase Given within 0-4 Hours of Onset of Myocardial Infarction Reduced Mortality in ISIS-2." Lancet 1 (28 February 1987): 502. ISIS-1 (First International Study of Infarct Survival) Collaborative Group. "Randomized Trial of Intravenous Atenolol among 16,027 Cases of Suspected Acute Myocardial Infarction: ISIS-I." Lancet 1 (12 July 1986): 57-66. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. "Collaborative Group Randomized Trial of Intravenous Streptokinase, Oral Aspirin, Both, or Neither among 17,187 Cases of Suspected Acute Myocardial Infarction: ISIS-2." Lancet 13 (August 1988): 349-60. Joyce, T., H. Corman, and M. Groosman. "A Cost-Effectiveness Analysis of Strategies to Reduce Infant Mortality." Medical Care 26, no. 4 (1988): 348-60. Kaplan, R. M., and J. P. Anderson. "A General Health Policy Model: Update and Applications." Health Services Research 23, no. 2 (1988): 203-35. Kosecoff,J.,J. Fink, R. H. Brook, A. R. Davies, G. Goldberg, L. S. Linn, V. A. Clark, and P. C. Salisbury. "General Medical Care and the Education of Internists in University Hospitals: An Evaluation of the Teaching Hospital General Medicine Group Practice Program." Annals of Internal Medicine 102, no. 2 (1985): 250-56. Last, J. M. A Dictionary of Epidemiology. New York: Oxford University Press, 1983. Laurie, J. A., C. G. Moertel, T. R. Fleming, H. S. Wieand, J. E. Leigh, J. Rubin, G. W. McCormack, J. B. Gerstner, J. E. Malliard, D. I. Twito, R. F. Morton, L. K. Tschetter, and J. F. Barlow. "Surgical Adjuvant Therapy of Large-Bowel Carcinoma: An Evaluation of Levamisole and the Combination of Levamisole and Fluorouracil."Journal of Clinical Oncology 7, no. 10 (1989): 1447-56. Lavie, C. J., G. T. Gau, R. W. Squires, and B. A. Kottke. "Management of Lipids in Primary and Secondary Prevention of Cardiovascular Diseases." Mayo Clinical Proceedings 63 (June 1988): 605-21. Lee, T. H., M. C. Weisberg, D. A. Brand, G. W. Rouan, and L. Goldman.

648

HSR: Health Services Research 27:5 (December 1992)

"Candidates for Thrombolysis among Emergency Room Patients with Acute Chest Pain: Potential True- and False-Positive Rates." Annals of Intenal Medicine 110, no. 12 (1989): 957-62. Lipid Research Clinics Program. "The Lipid Research Clinics Coronary Primary Prevention Trial Results."Journal of the American Medical Association 251, no. 3 (1984): 351-74. Ludwig Breast Cancer Study Group. "Prolonged Disease-Free Survival after One Course of Perioperative Adjuvant Chemotherapy for NodeNegative Breast Cancer." New England Journal of Medicine 320, no. 8 (1989): 491-96. Mansour, E. G., R. Gray, A. H. Shatila, C. K. Osborne, D. C. Tormey, K. W. Gilchrist, M. R. Cooper, and G. Faulkson. "Efficacy of Adjuvant Chemotherapy in High-Risk Node-Negative Breast Cancer: An Intergroup Study." New EnglandJournal of Medicine 320, no. 8 (1989): 485-90. Martin, M. J., S. B. Hulley, W. S. Browner, L. H. Kuller, and D. Wentworth. "Serum Cholesterol, Blood Pressure, and Mortality: Implications from a Cohort of 361,622 Men." Lancet 25 (October 1986): 933-36. Mayer, R. J., M. J. O'Connell, J. E. Tepper, and N. Wolmark. "Status of Adjuvant Therapy for Colorectal Cancer."Journal of the National Cancer Institute 81, no. 18 (1989): 1359-64. McCormick, J., and P. Skrabanek. "Coronary Heart Disease Is Not Preventable by Population Interventions." Lancet 1 (27 October 1988): 839-41. McCormick, M. C. "The Contribution of Low Birth Weight to Infant Mortality and Childhood Morbidity." New EnglandJournal of Medicine 312 (10 January 1985): 82-90. Moertel, C. G., T. R. Fleming, J. S. Macdonald, D. G. Haller, J. A. Laurie, P. J. Goodman, J. S. Ungerleider, W. A. Emerson, D. C. Tormey, M. H. Veeder, and J. A. Mailliard. "Levamisole and Fluorouracil for Adjuvant Therapy of Resected Colon Carcinoma." New England Journal of Medicine 322, no. 6 (1990): 352-58. Morgenstern, H., and E. S. Bursic. "A Method for Using Epidemiologic Data to Estimate the Potential Impact of an Intervention on the Health Status of a Target Population." Journal of Community Health 7, no. 4 (1982): 292-309. National Center for Health Statistics. "Advance Report of Final Mortality Statistics, 1988." Monthly Vital Statistics Report 39, no. 7 (1990a): 1-48. . Prevention Profik. Health, United States, 1989. Hyattsville, MD: Public Health Service, 1990b. Nattinger, A. B., R. J. Panzer, and J. Janus. "Improving the Utilization of Screening Mammography in Primary Care Practices." Archives of Internal Medicine 149 (September 1989): 2087-92. O'Brien, E., and K. O'Malley. "Overdiagnosing Hypertension." British Medical Journal 297 (November 1988): 1211-12. Office of Technology Assessment. The Quality of Medical Care: Information for Consumers. Summary. Washington, DC: U.S. Government Printing Office, 1988. Oldridge, N. B., G. H. Guyatt, M. E. Fischer, and A. A. Rimm. "Cardiac Rehabilitation after Myocardial Infarction."Journal of the American Medical Association 260 (1988): 945-50. Olson, R. E. "A Critique of the Report of the National Institutes of Health

Choosing Quality of Care Measures

649

Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol." Archives of Internal Medicine 149, no. 7 (1989): 1501-1503. Ouellet, R. P., A. Y. Apostolides, G. Entwisle, andJ. R. Hebel. "Estimated Community Impact of Hypertension Control in a High Risk Population." American Journal of Epidemiology 109, no. 5 (1979): 531-38. Pokras, R. "Detailed Diagnoses and Procedures for Patients Discharged from Short-Stay Hospitals, United States, 1985." In Vital and Health Statistics Series 13, no. 90. DHHS Publication no. (PHS) 87-1751. National Center for Health Statistics. Public Health Service. Washington, DC: U.S. Government Printing Office, April 1987. Robins, J. M., P. J. Landrigan, T. J. Robins, and L. J. Fine. "Decision Making under Uncertainty in the Setting of Environmental Health Policy." Journal of Public Health Policy 6 (September 1985): 322-28. Rubstein, L. V. Telephone conversation. 19 . Selby, J. V., and G. D. Friedman. "Sigmoidoscopy in the Periodic Health Examination of Asymptomatic Adults." Journal of the American Medical Association 261, no. 4 (1989): 595-601. Shapiro, S. "Evidence on Screening for Breast Cancer from a Randomized Trial." Cancer 39 (June 1977, supplement): 2772-82. Siu, A. L., E. McGlynn, M. H. Beers, H. Morgenstern, D. M. Carlisle, E. B. Keeler, R. H. Brook, J. Beloff, K. Wirtin, J. Leaning, B. C. Perry, H. P. Selker, W. Weiswasser, and A. Wiesenthal. The Expected Impact of Improved Quality of Carefor the Major Causes ofMortality and Morbidity. Santa Monica, CA: The RAND Corporation, 1992. Steinberg, M. D., M. A. Juliano, and L. Wise. "Psychological Outcome of Lumpectomy versus Mastectomy in the Treatment of Breast Cancer." American Journal of Psychiatry 142, no. 1 (1985): 34-39. Stockdale, F. E. "Breast Cancer." In Scientific American Medicine. Edited by E. Rubenstein and D. Federman. New York: Scientific American, Inc., 1988. Strauss, W. E., and A. F. Parisi. "Combined Use of Calcium-Channel and Beta-Adrenergic Blockers for the Treatment of Chronic Stable Angina. Rationale, Efficacy, and Adverse Effects." Annals of Internal Medicine 109, no. 7 (1988): 570-81. Tabar, L., C. J. G. Fagerberg, A. Gad, L. Baldetorp, L. H. Holmberg, 0. Gr6ntoft, U. Ljungquist, B. Lundstrom,J. C. Manson, G. Eklund, and N. E. Day. "Reduction in Mortality from Breast Cancer after Mass Screening with Mammography. Randomized Trial from the Breast Cancer Screening Working Group of the Swedish National Board of Health and Welfare." Lancet (13 April 1985): 829-32. Tate, D. A., and G. J. Dehmer. "New Challenges for Thrombolytic Therapy." Annals of Internal Medicine 110, no. 12 (1989): 953-55. Theroux, P., H. Ouimet, J. McCans, J. Latour, G. Levy, E. Pelletier, M. Juneau, J. Stasiak, P. DeGuise, G. B. Pelletier, D. Rinzler, and D. D. Waters. "Aspirin, Heparin, or Both to Treat Acute Unstable Angina." New England Journal of Medicine 319, no. 17 (1988): 1105-11. UK Trial of Early Detection of Breast Cancer Group. "First Results on Mortality Reduction in the UK Trial of Early Detection of Breast Cancer." Lancet (20 August 1988): 411-16. Verbeek, A. L. M., J. H. C. L. Hendriks, R. Holland, M. Mravunac, F.

650

HSR: Health Services Research 27:5 (December 1992)

Sturmans, and N. E. Day. "Reduction of Breast Cancer Mortality through Mass Screening with Modern Mammography. First Results of the Nijmegen Project, 1975-1981." Lancet (2 June 1984): 1222-24. Veronesi, U., R. Saccozzi, M. Del Vecchio, A. Banfi, C. Clemente, M. DeLena, G. Gallus, M. Greco, A. Luini, E. Marubini, G. Muscolino, F. Rilke, B. Salvadori, A. Zecchini, and R. Zucali. "Comparing Radical Mastectomy with Quadrantectomy, Axillary Dissection, and Radiotherapy in Patients with Small Cancers of the Breast." New EnglandJournal of Medicine 305, no. 1 (1981): 6-11. Warlow, C. "Carotid Endarterectomy: Does it Work?" Stroke 15, no. 6 (1984): 1068-76. Warren, R., and P. Skrabanek. "The Debate over Mass Mammography in Britain. The Case For, the Case Against." British Medical Journal 297 (October 1989): 969-72. Wilcox, R. G., G. von der Lippe, C. G. Olsson, G. Jensen, A. M. Skene, and J. R. Hampton. "Trial of Tissue Plasminogen Activator for Mortality Reduction in Acute Myocardial Infarction. Anglo-Scandinavian Study of Early Thrombosis (ASSET)." Lancet 3 (September 1988): 525-30. Williamson, J. W. "Formulating Priorities for Quality Assurance Activity. Description of a Method and Its Application." Journal of the American Medical Association 239, no. 7 (1978): 631-37. Williamson, J. W., M. Alexander, and G. E. Miller. "Priorities in PatientCare Research and Continuing Medical Education."Journal of the American Medical Association 204, no. 4 (1968): 303-308. Winawer, S. J., S. D. Leidner, D. G. Miller, D. Schottenfeld, B. Befler, R. C. Kurtz, P. Sherlock, and M. Stearns. "Results of a Screening Program for the Detection of Early Colon Cancer and Polyps Using Fecal Occult Blood Testing." Gastroenterology 72, no. 5 (1977): 1150. Winslow, C. M., D. H. Solomon, M. R. Chassin, J. Kosecoff, N. J. Merrick, and R. H. Brook. "The Appropriateness of Carotid Endarterectomy." New EnglandJournal of Medicine 318, no. 12 (1988): 721-27. Wynder, E. L., F. Field, and N. J. Haley. "Population Screening for Cholesterol Determination. A Pilot Study."Journal of the American Medical Association 256, no. 20 (1986): 2839-42. Yusuf, S., J. Wittes, and L. Friedman. "Overview of Results of Randomized Clinical Trials in Heart Disease. I: Treatments following Myocardial Infarction."Journal of the American Medical Association 260, no. 14 (1988): 2088-93. Yusuf, S., R. Peto, J. Lewis, R. Collins, and P. Sleight. "Beta Blockade during and after Myocardial Infarction: An Overview of the Randomized Trials." Progress in Cardiovascular Diseases 27 (1985): 335-71. Zaridze, D. G. "Environmental Etiology of Large-Bowel Cancer."Journal of the National Cancer Institute 70 (1983): 389-400.