NeuroRehabilitation An IntenlllClpliMry Journal

ELSEVIER

NcuroRehabilitation 5 (1995) 27-38

Clinical quality improvement: Measuring and managing quality in rehabilitation medicine Daniel R. Longo Department of Family and Community Medicine, University of Missouri·Columbia, Columbia, MO 65212, USA

Accepted 10 September 1994

Abstract In this paper clinical quality improvement, also referred to as total quality management, is described as a theoretical and methodological framework. Use of this approach is proposed and examined as a paradigm for quality of care measurement and management in rehabilitation medicine. While the field of rehabilitation has a long tradition in the area of outcomes measurement, in particular that of functional status assessment, the larger framework of clinical quality improvement offers a very promising and more complete framework for rehabilitation than those employed in the past. While the framework requires further testing and research, it has been useful in some areas of acute, as well as primary care. This paradigm is consistent with the approaches traditionally taken in measuring the quality of rehabilitation medicine. Recommendations are made for its application in rehabilitation medicine.

Keywords: Quality of care; Quality improvement; Rehabilitation quality

1. Background The measurement and management of the quality of medical care has been a subject of much debate in the clinical, administrative, and health services research literature. While there is a long tradition and literature in this area, it is only in the last decade that quality of care has moved from primarily a regulatory and accreditation concern to a much broader one included on the agenda of clinicians, administrators, and researchers. Consumers have also shown great interest in this area. While this change of interest and emphasis has been the result of a number of

factors, to a large degree it resulted from the recent application of industrial quality control measures and methods to health care delivery. This paper examines what is referred to as clinical quality improvement (CQ!). Its specific aims are to: (1) explain why quality of care is an issue, (2) define quality, (3) describe quality measures, (4) explore the possible clinical applications of CQI to rehabilitation medicine, and (5) offer recommendations for further work.

1.1. U'hy is quality of care an issue? Since the first reference in 2000

1053-8135/94/$09.50 © 1995 Elsevier Science Ireland Ltd. All rights reserved. SSDI 1053-8135(94) 00102-Z

B.c. to guide-

28

D.R. Longo / NeuroRehabilitation 5 (1995) 27-38

lines for physician billing and penalties for incompetence, [1] our system of rewards and penalties in medical care has dramatically changed. Yet society's interest in getting the best possible medical care has remained constant. Longo and Avant (1994) write that "the history of quality assurance is largely the history of physicians being asked for their best work by their regulators, church, state or other members of their profession" [2, p. 46]. Over the last several decades, the field of quality assurance and improvement has vastly changed in terms of its philosophical orientation and its methods of measurement. To a very large degree these changes have been the result of two interrelated factors. First, there has been discontent and frustration over the traditional quality assurance approach. Berwick (1989) refers to the 'bad apples' theory to illustrate these concerns [3]. From this perspective the quest of quality assurance is to identify and eliminate, or punish, health care providers whether they be clinicians or facilities that perform badly. Punishment may take various forms and includes, but is not limited to, sanctions, licensure termination, or public disgrace. This theory assumes that the elimination or curtailment of care by such providers would create a safe, or quality, health care system. Second, from at least the time of the American College of Surgeons' first Hospital Standardization Program, the forerunner of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), the standard approach to quality assurance was quality by inspection. It was measured through the use of standards by which the presence or absence of the standard fulfillment was inspected by external surveyors who decided whether or not the standard was in compliance. Through the inspection of standards it was believed that at least the most invasive and expensive of care settings, the hospital, would become standardized to ensure a standard of quality in the nation's hospitals. Similar approaches were taken by other groups in the health care field. Quality via inspection gave way to the assumption that if the inspection did not find a problem, it did not exist. Such an approach does not take into account the inevitable existence of false positives and negatives, fundamental

concerns in quality measurement. Inspection and identification of 'bad apples' are interrelated approaches. While elimination of 'bad apples' and quality by inspection may perhaps appear a bit archaic today, from a historic perspective it is understandable. By 1910, at the time of the Flexner Report on the Quality of Medical Education, neither medical education nor hospital care were well regarded [4]. In brief, medical education was substandard in many medical schools, and, the nation's hospitals were where the incurable poor died while the rich were cared for by private physicians at home. Fortunately, fundamental changes occurred in medical education as well as hospital care in response to the Flexner Report. In the next 40 years health care was transformed more than it had been in the previous 400 years. The germ theory and the scientific method were developed and influenced all aspects of health and medical care [5]. Infectious disease was largely conquered through the discovery of antibiotics. Numerous innovations in surgery and emergency care were brought to the civilian sector in the aftermath of medical experience gained during World War II. Further advances in medical diagnostic equipment and surgery, together with the widespread introduction of health insurance, made the hospital the preferred site for much of America's health care. The American College of Surgeons undertook the task of ensuring national standards of care through its Hospital Standardization Program [2]. For the next thirty years, the JCAHO largely carried on the vast amount of health care 'inspection' through hospital surveys. It received a tremendous amount of added clout in 1965 with the passage of Medicare and Medicaid when the concept of 'deemed status' was given to the JCAHO. The JCAHO became a major focus, next to government, in hospital inspection. This concept, in brief, accepted JCAHO accreditation as meeting the 'conditions of participation' required of all providers for payment under the Medicare program. With Medicare and Medicaid in 1965 also came the concept of utilization review, which began to integrate the concepts of quality of care and cost and utilization management.

D.R. Longo / NeuroRehabilitation 5 (1995) 27-38

In 1972 Professional Standards Review Organizations (PSROs) were established. These continued to strengthen the tie of quality to cost through government regulations and the creation of external review agencies for 'peer review' of Medicare hospital patients. PSRO and related laws placed greater emphasis on retrospective audit or medical care evaluation for review and improvement of care quality. These programs tied together, from payment, inspection, and sanction perspectives, the twin concerns of cost and quality. The period 1972 through 1986 experienced a number of innovations in quality of care including: (1) audits, problem identification and occurrence screening approaches [6], (2) quality assurance program through institutional-wide review [7], and (3) systematic monitoring and evaluation. While all these developments advanced the state of the art of quality of care measurement, unfortunately these innovations resulted in several serious side effects. First, they were viewed by many as an intrusion into clinical practice required simply for accreditation, licensure, and regulatory reasons. Second, implemented largely by administrators, the changes were seen as not pertinent to clinical practice. Third, the changes did nothing to stop the negative view of quality because of the continued emphasis on problem finding (alias for elimination of 'bad apples'), inspection and sanctions. Fourth, these changes were perceived as reflective of an inferior discipline in search of appropriate theories and methods rather than as advances or innovations. These perceptions continue in some circles today. Ironically, perhaps the major impetus to a perception of real change in quality assurance came with the December 1986 public release of hospital mortality statistics by the Health Care Financing Administration (HCFA). This was the first public release of data about the experiences of hospitals treating Medicare patients, and served to heighten public awareness about the quality of health care. The release ushered in an era of public accountability that continues today and, while the release was not well received by the formal hospital and medical community, it resulted in substantive changes in the nature of quality assurance in this country.

29

From the HCFA mortality data release through the present there was a great explosion of quality of care projects that set the stage for the current health care reform debate. It also contributed in part to health care reform's inclusion as a major issue in the 1992 presidential election [2]. This election was driven by domestic economic concerns. Controlling rising health care costs was viewed as key to controlling the national debt [8]. The healthcare reform debate was concerned with issues of under-care, over-care, the uninsured, and the provision of basic health care services aimed at decreasing infant mortality and increasing childhood immunizations [9,10]. However, unlike previous attempts at reform, the cost issue fueled by the aforementioned concerns accelerated to the point that the American public raised questions such as: what return is society getting for its investment in health care? Are the dollars expended for health care resulting in a proportionate increase in good outcomes? Is care appropriate? Is it cost-effective? These questions continue to shape the health care reform debate [11-13]. 2. The definition of quality

Given the context of these questions, how is quality best defined today? To answer this fundamental question we must examine the work of Donabedian, the Institute of Medicine (10M), and the clinical quality improvement literature, especially that of James and Berwick. The foundation of a quality of care definition was initially outlined by the work of Donabedian [14-25]. Donabedian writes "that there are several definitions of quality, or several variants of a single definition, and that each definition or variant is legitimate in its appropriate context" [14, p. 27]. He proposes "that the balance of health benefits and harm is the essential core of a definition of quality" [14, p. 27], and that the definition incorporates both the technical and interpersonal aspects of care. His classic quality of care paradigm of structure, process, and outcome has become the framework for assessing quality. Structure asks the question - "Are necessary resources available to provide effective, efficient medical care?" Process asks the questions - "Have the processes

30

D.R Longo / NeuroRehabilitation 5 (1995) 27-38

necessary for providing effective, efficient medical and administrative services been established?" and "Are these processes in control, ensuring that outputs consistently meet requirements?" Outcome asks the questions - "How does the organization or provider assess whether resources and processes are used effectively and efficiently?", "What methods are used to measure effectiveness?" and "What are measurable benchmarks?" Numerous studies have utilized the Donabedian conceptual framework for assessing quality of care; these are described in detail elsewhere [21,23,26-34], One of the most notable is the Medical Outcomes Study conducted at RAND jUCLA which illustrates some of the quality of care 'methodologies' such as functional status, patient satisfaction, and other outcomes of care that may be used to measure quality [35]. This approach, although far more comprehensive, is very compatible with those taken in a variety of rehabilitation medicine outcome studies [36-39]. In a Congressionally mandated study of quality of care in Medicare conducted by the 10M, quality of care was defined as "the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge" [30]. Additionally it recommended that the definition of quality is multi-dimensional consistent with Donabedian's summary that: • • •

quality has many aspects; in any situation, attention usually will focus on some of these aspects in preference to others; and the specific outcomes chosen as indicators of quality will vary in some degree depending on the aspect of quality being assessed, but the role of outcomes in assessment will be the same [40. p. 356].

Industrial quality control theories as espoused by Deming, Juran, and others also offer guidance on the definition of quality [41,42]. The industrial quality control theories have been referred to in the literature by a variety of names including total quality management, continuous quality im-

provement, quality improvement process, and industrial quality management science [43]. While there are some subtle differences between each of these, their application to health care must be viewed as a paradigm shift where such differences in terminology are of little concern in comparison to their common theoretical framework. Initially used by the Japanese in the aftermath of the Second World War, and then by a number of American companies, they were later applied to American health care. Work at the National Demonstration Project at Harvard Community Health Plan and a multitude of projects launched in Health Maintenance Organizations and hospitals have applied these theories and methods to health care. To a large degree, these innovations challenged the definition, measurement, and management of quality in health care. The most important question these applications raise is "How can these theories and techniques developed largely in the manufacturing sector be applied, if at all, to health care?" Given this fundamental question, the term used, and specific application applied by James (1989) - clinical quality improvement (eQ!) - has the most appeal and applicability given its emphasis on clinical care as opposed to purely administrative and business concerns espoused by other theorists and methodologists [23]. In fact, it is James' and his colleagues' work that provides some of the best examples of the clinical application of industrial models and resulting improvement in patient outcomes through CQI [24,25]. CQI is the application of the industrial engineering approach of finding variations in the process of manufacturing in an effort to isolate and reduce those variations [44,45]. In so doing the quality of the product will be increased and re-work will be reduced, thus increasing profitability [46-49]. It includes a management philosophy that is appealing to hospital administrators who have grown tired of regulatory controls. It stresses employee empowerment and moves from the search for 'bad apples' to the continuous improvement of ~ll work processes and employees [50].

James writes that "High quality is achieved by continual improvement in terms of customer' ex-

D.R Longo / NeuroRehabilitation 5 (1995) 27-38

pectations" [23, p. 10]. This mandates the inclusion of patient satisfaction as a vital component of quality similar to the view of Oonabedian and the 10M. By further viewing health care delivery as a complex 'process,' James suggests that improvements in outcome will only result if the process of care is well understood and efforts are made to eliminate inappropriate variation in care. This is consistent with Kahn's emphasis upon the process of care: Process of care - what we do to patients - have been considered an essential component of quality of care ... Even if outcomes of care - what happens to patients - are the most meaningful measures of quality to the patient, we will be unable to develop clinical methods to improve outcomes unless we understand how processes and outcomes are related. Assessing quality of care by process also provides some measurement advantages over studying outcomes because not all patients who experience a poor process of care suffer a poor outcome [51, p. 1969]. He defines inappropriate variation as "variation that increases costs but does not lead to improvement in medical outcomes" [23, p. 22]. From this perspective the emphasis of CQI as opposed to traditional quality assurance is on the identification of variation in processes of care that impact upon clinical outcomes, rather than on the identification of 'bad apples.' All providers can do better and improve, not just the 'bad apples.' Further, the emphasis on ongoing or continuous improvement eliminates the necessity for inspection, quality ceilings, and thresholds of what may constitute good care. Finally, with the improvement of clinical care as outcome, CQI as espoused by James offers a clinically appealing model that negates the concerns of many of the past critics of quality assurance. Fig. 1 graphically depicts the impact of traditional quality assurance versus clinical quality improvement as described by James. The curves depict a distribution of quality outcomes. The two top curves are 'before' and 'after' an intervention to improve quality within the traditional QA

31

framework; the bottom curves illustrate the same intervention within a CQI framework. The QA model eliminates the outliers, or 'bad apples' with a minimum improvement for all other outcomes of care. The CQI model, however, through what James refers to as the "principle of elimination of inappropriate variation," improves all outcomes. As a result, the curve is now drawn "higher and tighter about its central point, and the principle of document continuous improvement shifts the entire curve to the right" [23, p. 37]. This is not to say that 'bad apples' are permitted to provide inferior care. Rather, it suggests that all providers are continually improving quality with resulting improvements in all outcomes. The best illustration of this is a study conducted at LOS Hospital, Salt Lake City, in which identifying that the administration of prophylactic antibiotics two hours prior to surgery reduced the surgical infection rate below the traditional threshold of 2% [24]. In summary, quality of care is the outcome of a series of interrelated processes that provide the highest level of care within available resources and current knowledge. It is multi-dimensional and includes such attributes as patient satisfaction, functional status, availability, and accessibility. Its measurement consequently will reflect these dimensions. 3. The measurement of quality The measurement of quality requires a knowledge of alternative methods and their objectives, and fundamental measurement issues. Table 1 details a variety of quality of care methods and/or approaches that are frequently used to measure quality. There are a number of issues revealed by a review of these methods / approaches. While there are a variety of methods, many are interrelated; for example, patient satisfaction and functional status measures are usually incorporated in CQI. Also, severity of illness adjustments should ideally be made for all appropriate measures. The use of mortality analysis, especially risk/severity adjustment, has also been widely used in the hospital field, especially after the HCFA mortality release [52,53]. Hospital mortal-

D.R. Longo I NeuroRehabilitation5 (1995) 27-38

32

A. Traditional quality assurance.

Increased

minimum

standard

Minimum

standard

-

Quality_

-

-

-

-

-

-

-

1- _ _ _ _ _ _ _

Quality_

B. Clinical quality improvement.

- - - - - - - - - .... - - - - - - - - , , - - - - - - - - -Quality - _

Quality_

---------.~--

Fig. 1. The impact of traditional quality assurance versus clinical quality improvement. Reprinted with permission: James Be. Quality management for health care delivery. Chicago: Hospital Research and Educational Trust, 1989.

ity is now perhaps the most commonly referenced 'outcome indicator' because it poses considerable intuitive appeal; however, scientific evidence concerning its appropriateness for purposes of assessing hospital quality is mixed [54]. There are two questions for consideration. First, does inhospital mortality measure quality (or the lack thereof)? and, if it is a valid measure, what is the most appropriate technique for risk/severity adjustment? The research literature contains exhaustive volumes debating these two points [55,56]. Finally, Table 1 lists CQI as a specific approach, recognizing it may incorporate a number of others, such as patient satisfaction and functional status assessment. In a similar way, medical audits and special studies may also incorporate a number of other approaches.

After considering the various methods/approaches, there are three fundamental questions that must be asked with regard to measuring the quality of care: "Will the method measure quality validly and reliably?," "Will the method be useful in developing approaches to improve quality?" and, "Will the method produce better care?" Validity is defined as the extent to which the measure actually measures what it purports to measure. Reliability refers to the ability of the measure to identify the same result after repeated measures, or when different individuals take the same measurement. It asks, if we measure the same set of objects again and again with the same or comparable measuring instruments, will we get the same or similar results? Unless a quality of care method explicitly addresses these

D.R. Longo / NeuroRehabilitation5 (J995) 27-38

Table 1 Quality of care methods and objectives

- - _..--_._-_._-_._-_ _-_ ....

Method Appropriateness review Functional status Generic screens/incident reporting Mortality review Integrative approaches Medical audit/,special studies' Patient satisfaction Record review/chart review 'Research studies' Severity of illness/case-mix adjustments Licensure/accreditation surveys Outcome studies/variation studies Clinical quality improvement/total quality management

....

33

_-_ _-_ _-_ _ - _ . _ - - - - - - - - - ..

..

....

..

Objective

. - - - - - - - - - - - - - . - - -....- - - - - - -..- - - -.. - - - -

To review and confirm the necessity for a given procedure, medication, or treatment. To assess the patient's view of activities of daily living. To identify through an early warning system potential litigation and/or sentinel events. To identify unexpected deaths and/or a greater than expected number of deaths. To merge together in one review method the interrelated quality management issues of quality assurance, risk management, utilization review, and infection control. To conduct in-depth medical record based studies to confirm or reject suspected quality problems. To assess the quality of care from a patient's perspective. To identify particular concerns in care through detailed review of the medical record. To address a variety of quality issues other than those identified in the other methods. To take into account the relative severity of illness, or mix of patients, to overall outcomes of care. To review and/or inspect organizations and their compliance with standards or regulations. To identify and analyze the outcome of patient care (may overlap with other approaches such as mortality review and CQI). To find variation in process in an effort to reduce it and improve the quality of the product (application of industrial engineering approach).

Adapted from: Longo DR, Daugird AJ. Measuring the quality of care and reforming the health care system [76].

two basic concerns, its utility should be seriously questioned. The literature well describes techniques for addressing issues of validity and reliability as they relate to the measurement of quality and offers suggestions on how to address these concerns [57-60]. Is COl useful in developing approaches to improve quality? and, will COl produce better care? To answer these questions a review of the COl theory and methods, and studies to date was reviewed as part of research conducted for the development of a quality of care strategy for the ShowMe Health Reform Taskforce organized by the Missouri Department of Health. The literature tells us that CQI has indeed been extremely successful in the industrial setting. In fact, it has been cited as a major factor in improving the Japanese economy since the end of World War II. In the United States it has also been well

received in the manufacturing sector of the economy. But, does COl work as well in health care? At this point there has been a great deal of enthusiasm and support for both the theory and methods. In fact there are reports that over sixty percent of U.S. hospitals have implemented at least some aspect of COL Thus, based on a number of considerations, it appears that COl can develop approaches that at least 'aim' to improve quality. The theory and methods are strong and are well tested in industry. They have been widely embraced by the health care community as a necessary condition for the adoption of an innovation as the degree of support for it and the changes it necessitates. Does COl produce better quality? At this point, there are reports of important administrative changes as the result of CQI. To the extent these administrative changes impact on outcomes of

34

D.R. Longo / NeuroRehabilitation 5 (J995) 27-38

care, there may be added benefit. Many of these reports come from the work of Berwick and his colleagues at Harvard Community Health Plan [61]. However, empirical findings from well conducted studies are sparse and largely limited to the work of Intermountain Health Care System in Salt Lake City [23,25,62]. COl provides an excellent theory on which to define, measure, and manage health care. Its methods have a great deal of appeal; however, prior to initiating widespread implementation of these methods further studies are needed in clinical settings to ensure the applicability of these methods to clinical medicine. At the hospital or clinic level it would be advisable to implement a small number of COl studies and evaluate their success prior to large scale adoption of COl for the entire organization. 4. Applications to rehabilitation medicine

COl offers a promising paradigm for application to rehabilitation medicine. There are, however, several cautions that must be assessed. First, outcomes do not directly assess quality, they only permit an inference about the quality of the process (and structure) of care. The degree of confidence in that inference depends on the strength of the predetermined causal relationship between process and outcome (and structure and process) (p. 358) [40]. Second, outcomes are influenced by a variety of factors, only one of which is the provision of medical care. For example, the following all contribute in some way to the outcome of care: health status, health behavior, the range and type of comorbid conditions, age, genetics, housing, education, income, and race [63,64]. Longo (1993) argues that Wennberg's concept of medical practice variation is incomplete, especially for chronic conditions where 'patient practice variation,' including patient culture, social class, ethnicity, and other factors under control of the patient that may contribute even more significantly to outcomes than medical care [65]. Third, two interrelated areas of inquiry, patient satisfaction and functional status, require further

examination [66-68]. They are joined in their inherent assumption that patient-centered approaches to quality assessment are critical. Patient satisfaction may serve as the most effective approach to monitoring the interpersonal and 'caring' dimensions of quality [69]. It measures the following domains: patient perception of treatment quality, patient perception of clinical and psychological outcomes, patient behaviors and attitudes regarding treatment, and patient's general impressions [68]. Functional status, from the patient's perspective, may also give insights regarding how well a care process improves outcome. One of the most widely used instruments is the short form 36 (SF36) developed by Ware. It includes 36 measures from the following domains: physical functioning, psychological distress/wellbeing and cognitive functioning measures, health perceptions, energy/fatigue, health distress and social activity limitations, family and marital functioning, role functioning, pain, sleep, and psychological symptoms [17,70]. Not surprisingly, many of these domains are similar to those currently used in rehabilitation. While there are a number of publications that describe in detail COl methods, [29,41,42,46,61, 71,72] there is one in particular that may be easily applicable to rehabilitation medicine. A 'control chart' is one of seven quality control tools that permit the graphic depiction of a measurement plotted against time. Fig. 2 illustrates how a control chart may be applied to rehabilitation medicine by plotting the self-care component of the Functional Independence Measures (FIM) from admission to discharge. FIM is consistently measured in most rehabilitation centers and is most commonly used to measure functional independence of persons with stroke. Rated on a seven-point scale, FIM is a good illustration of how a COl method may be applied within the context of measures already well accepted in rehabilitation. The chart illustrates for a group of patients assigned to specific clinicians the FIM score at admission and at discharge together with the mean score and upper and lower confidence limits for each. By convention, the control limits are set at three standard deviations above and

35

D.R. Longo / NeuroRehabilitation5 (1995) 27-38

Intervention I/)

III ~ ::::I

me

I/)

•

lII e 0 0 ell.

IIIE

"Co eU

~S! IIIIU

~~

-Qj

:!(J)

0 :;:: 0

e

::::I

u.

-

-

:i:1II

3.22

•

•

• •

D

-- --• --_._-- -- --• -- -- -- • • • • •

•

• .!....---.-

•

•

•

•

• • •

-- --

--.-- --

• ••

• -.------• •

Upper

limit __ -confidence e----

•

• • • Mean 4.64 •

•

•

•

• Lower confidence limit

...-----

•

Time (weeks) Admission - - - - - - - - - - - - - - - - - - - - . . . Discharge Fig. 2. Process control chart with measure of interest on the y-axis and time on the x-axis. Data from: Howard A. Rusk Rehabilitation Center Program Evaluation Report. Columbia, MO, July 1992-June 1993.

below the mean. This permits an indication of the degree of variation expected in a care process (in this case self-care in stroke patients) that is considered to be well established and stable. The control chart has several purposes. It allows a graphic display and assessment of the variation in a process of care. It identifies outliers that may represent particular 'problems' or 'successes' for further investigation. It permits an assessment of the influence of time as it relates to a particular measure. It may be used as part of an ongoing monitoring system to determine whether or not variations reflect random 'error' or particular threats to the integrity of the care process. The control chart is particularly useful as provider feedback through an analyses of practice patterns. Given this latter approach an individual clinician may compare his/her experience over time and in comparison to peers. When significant variation is found, either positive or negative, a discussion of the causes in variation should be included as part of the peer review process. This type of discussion can have dramatic impact on the peer review process as it identifies new techniques and approaches in the care process

that may promise improved outcomes for all patients. Obviously, if the variation found is negative, action may be taken to change the care process to ensure that variation is reduced and outcomes improved. Further discussion of physician profiling is found in Welch et al. [73] while discussion of CQI and peer review is found in Berwick [74] and Zusman and Berwick [75]. 5. Conclusions and recommendations The definition of quality is multi-dimensional. Consequently, a variety of measures are needed. A~ a framework clinical quality improvement offers an important approach to qualities that is consistent with the goals of rehabilitation medicine and offers great promise for improving the quality of care. The following guidelines are offered as the field of rehabilitation medicine advances its work in the quality of care. One, conceptualize quality of care as multi-dimensional. Two, assess the validity and reliability of measures. Three, demonstrate and test methods before wide-scale implementation. Four, be certain that results are

36

D.R. Longo / NeuroRehabilitation 5 (1995) 27-38

not only statistically significant but also clinically significant before taking any corrective action. CQI may assist in safeguarding the public trust for quality of care through an ongoing commitment to the highest quality of care within available resources and within current medical knowledge. References [1] Mayer ES. Academic support for rural practice: the role of area health education centers in the school of medicine. Acad Med 1990;65:S45-50. [2] Longo DR, Avant DW. Managing quality. In: Taylor R, Taylor S, eds. The AUPHA manual of health services management. Gaithersburg, MD: Aspen, 1994;45-69. [3] BeIWick DM. Continuous improvement as an ideal in health care [see comments]. N Engl J Med 1989;320:53-56. [4] Blumberg MS. Comments on HCFA hospital death rate statistical outliers. Health Serv Res 1987;21:715-739. [5] Fine DJ, Meyer ER. Quality assurance in historical perspective. Hosp Health Serv Adm 1983;28:94-121. [6] Craddick JW. Medical management analysis in 1986. In: Chapman-Cliburn G, ed. Risk management and quality assurance: issues and interactions. Chicago: Joint Commission on Accreditation of Hospitals, 1986;71-78. [7] Sanazaro PJ, Mills DH. A critique of the use of generic screening in quality assessment [see comments]. J Am Med Assoc 1991;265:1977-1981. [8] McNeil BJ. Socioeconomic forces affecting medicine: times of increased retrenchment and accountability. Semin Nucl Med 1993;23:3-8. [9] Dubois RW. Reducing unnecessary care: different approaches to the 'big ticket' and the 'little ticket' items [see comments]. J Ambulatory Care Manage 1991;14:30-37. [10] BeIWick DM, Hiatt HH. Who pays? [editorial; comment]. N Engl J Med 1989;321:541-542. [11] Dubois RW, Rogers WH, Moxley JH, et al. Hospital inpatient mortality. Is it a predictor of quality? N Engl J Med 1987;317:1674-1680. [12] Gustafson DH. Lessons learned from an early attempt to implement CQI principles in a regulatory system. Qual Rev Bull 1991;17:324-325. [13] Kasper JF, Mulley AGJ, Wennberg JE. Developing shared decision-making programs to improve the quality of health care [see comments]. Qual Rev Bull 1992;18:183-190. [14] Donabedian A. The definition of quality and approaches to its assessment. Ann Arbor, MI: Health Administration Press, 1980. [15] Donabedian A. The criteria and standards of quality. Ann Arbor, MI: Health Administration Press, 1982.

[16] Donabedian A. The methods and findings of quality assessment and monitoring: an illustrated analysis. Ann Arbor, MI: Health Administration Press, 1985. [17] Lohr KN. Advances in health status assessment. Overview of the conference. Med Care 1989;27:S1-S11. [18] Lohr KN, Donaldson MS, Harris-Wehling J. Medicare: a strategy for quality assurance. V. Quality of care in a changing health care environment. Qual Rev Bull 1992;18:120-126. [19] Lohr KN, Harris-Wehling J. Medicare: a strategy for quality assurance. I. A recapitulation of the study and a definition of quality of care. Qual Rev Bull 1991;17:6-9. [20] Lohr KN, Schroeder SA. A strategy for quality assurance in Medicare [see comments]. N Engl J Med 1990;322:707-712. [21] Lohr KN, ed. Medicare: a strategy for quality assurance, Vol. II. Washington, DC: National Academy Press, 1990. [22] Harris-Wehling J. Defining quality of care. In: Lohr KN, ed. Medicare: a strategy for quality assurance, Vol. II. Washington, DC: National Academy Press, 1990;116-139. [23] James Be. Quality management for healthcare delivery. Chicago: Hospital Research and Educational Trust, 1989. [24] James Be. Management by fact: quality improvement and clinical research. 1991; unpublished results. [25] James Be. TQM and clinical medicine [comment]. Front Health Serv Manage 1991;7:42-46. [26] Starfield B. Primary care: concept, evaluation, and policy. New York: Oxford University Press, 1992. [27] Benson DS, Townes PG Jr. Excellence in ambulatory care: a practical guide to developing effective quality assurance programs. San Francisco: Jossey-Bass, 1990. [28] German PS, Skinner EA, Shapiro S et al. Preventive and episodic health care of inner-city children. J Community Health 1976;2:92-106. [29] Ciccone KR, Lord JT. IQA-2: continuous performance improvement through integrated quality assessment. Chicago: American Hospital Publishing, Inc., 1989. [30] Lohr KN, ed. Medicare: a strategy for quality assurance, Vol. I. Washington, DC: National Academy Press, 1990. [31] Medicare: new directions in quality assurance. Washington, DC: National Academy Press, 1991. [32] Fuchs VR. The supply of surgeons and the demand for operations. J Hum Resources 1978;13(suppI):35-56. [33] Effectiveness and outcomes in health care. Washington, DC: National Academy Press, 1990. [34] Health care quality management for the 21st century. Tampa: American College of Physician Executives, 1991. [35] Tarlov AR, Ware JE Jr, Greenfield S, et al. The Medical Outcomes StUdy. An application of methods for monitoring the results of medical care. JAmMed Assoc 1989;262:925-930. [36] Rosenthal M, Millis S. Relating neuropsychological indi-

D.R. Longo / NeuroRehabilitation5 (1995) 27-38

[37] [38]

[39] [40] [41] [42] [43]

[44] [45]

[46]

[47] [48]

[49]

[50]

[51]

[52]

[53]

[54]

[55]

cators to psychosocial outcome after traumatic brain injury. NeuroRehabilitation 1994;2:1-8. Whiteneck GG. Outcome evaluation and spinal cord injury. NeuroRehabilitation 1992;2:31-41. Forer S. How to make program evaluation work for you: utilization for program service management. NeuroRehabilitation 1992;2:52-71. Quality rehabilitation: results oriented patient care. Chicago, IL: American Hospital Publishing, 1989. Donabedian A. The role of outcomes in quality assessment and assurance. Qual Rev Bull 1992;18:356-360. Walton M. The Deming management method. New York: Dodd, Mead and Company, 1986. Whetsell GW. Total quality management. Top Health Care Financing 1991;18:12-20. Blumenthal D. Total quality management and physicians' clinical decisions. JAmMed Assoc 1993; 269:2775-2778. Newman JA Jr. Lessons in quality from other industry experiences. Top Health Care Financing 1991;18:1-6. Schoenbaum SC. An attempt to manage variation in obstetrical practice. In: Effectiveness and outcomes in health care. Washington, DC: National Academy Press, 1990;190-200. Berwick DM. Controlling variation in health care: a consultation from Walter Shewhart. Med. 1991; 29:1212-1215. Berwick DM. Seeking systemness. Healthcare Forum J 1992;35:22-28. McKinlay JB, McKinlay SM. The questionable contribution of medical measures to the decline of mortality in the United States in the twentieth century. Milbank Mem Fund Q Health Soc 1977;55:405-28. Reinhardt UE. Perspectives on effectiveness and outcomes research: the social perspective. In: Effectiveness and outcomes in health care. Washington, DC: National Academy Press, 1990;34-37. Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data. J Am Med Assoc 1990;263:247-249. Kahn KL, Rogers WH, Rubenstein LV, et al. Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system [see comments]. JAmMed Assoc 1990;264:1969-1973. Fink A, Yano EM, Brook RH. The condition of the literature on differences in hospital mortality [see comments]. Med Care 1989;27:315-336. Charney E, Bynum R, Eldredge D et al. How well do patients take oral peniCillin? A collaborative study in private practice. Pediatrics 1967;40:188-195. Thomas JW, Holloway n, Guire KE. Validating risk-adjusted mortality as an indicator for quality of care. Inquiry 1993;30:6-22. Thomas JW, Longo DR. Application of severity mea-

[56]

[57] [58]

[59]

[60]

[61]

[62]

[63]

[64]

[65] [66]

[67] [68]

[69]

[70]

[71]

[72]

37

surement systems for hospital quality measurement. Hosp Health Serv Admin 1990;35:221-243. Iezzoni LI, Ash AS, Coffman GA, et al. Predicting in-hospital mortality. A comparison of severity measurement approaches. Med Care 1992;30:347-359. Iezzoni LI. Monitoring quality of care: what do we need to know? Inquiry 1993;30:112-114. Horn SD. Validity, reliability and implications of an index of inpatient severity of illness. Med Care 1981;19:354-362. McHorney CA, Ware JE Jr, Rogers W, et al. The validity and relative precision of MOS short- and longform health status scales and Dartmouth COOP charts. Results from the Medical Outcomes Study. Med Care 1992;30:MS253-MS265. Revicki DA, Klaucke DN, Brown RE, et al. Reliability of ratings of anesthesia's contribution to adverse surgical outcomes. Qual Rev Bull 1990;16:404-408. Berwick DM, Godfrey AB, Roessner J. Curing health care: new strategies for quality improvement. San Francisco: Jossey-Bass, 1991. Pauly MV. Effectiveness research and the impact of financial incentives on outcomes. In: Shortell SM, Reinhardt UE, eds. Improving health policy and management. Ann Arbor, MI: Health Administration Press, 1992;151-193. Ware JE Jr, Berwick DM. Patient judgments of hospital quality. Conclusions and recommendations. Med Care 1990;28:S39-S44. Wells KB, Rogers W, Burnam A, et al. How the medical comorbidity of depressed patients differs across health care settings: results from the Medical Outcomes StUdy. Am J Psychiatry 1991;148:1688-1696. Longo DR. Patient practice variation. A call for research. Med Care 1993;31:YS81-YS85. Wu AW, Rubin HR, Mathews WC, et al. A health status questionnaire using 30 items from the Medical Outcomes Study. Preliminary validation in persons with early HIV infection. Med Care 1991;29:786-798. Nelson EC, Berwick DM. The measurement of health status in clinical practice. Med Care 1989;27:S77-S90. Strasser S, Davis RM. Measuring patient satisfaction for improved patient services. Ann Arbor, MI: Health Administration Press, 1991. Holloway n, Thomas JW. Factors influencing readmission risk: implications for quality monitoring. Health Care Financing Rev 1989;11:19-32. Ware IE Jr, Sherbourne CD. The MOS 36-item shortform health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992;30:473-483. O'Brien JL, Hughes EF. The physician's role in quality assessment and improvement. Top Health Care Financing 1991;18:33-45. Steiber SR, Krowinski WJ. Measuring and managing patient satisfaction. Chicago: American Hospital Publishing, Inc., 1990.

38

D.R. Longo / NeuroRehabilitation5 (1995) 27-38

[73] Welch HG, Miller ME, Welch WP. Physician profiling. An analysis of inpatient practice patterns in Florida and Oregon. N Eng! J Med 1994;330:607-612. [74] Berwick OM. Peer review and quality management: are they compatible? Qual Rev Bull 1990;16:246-251. [75] Zusman J, Berwick OM. A letter and reply: "Peer

review and quality management: are they compatible?" Qual Rev Bull 1990;16:418-420. [76] Longo DR, Oaugird AJ. Measuring the quality of care: reforming the health care system. Am J Med Qual 1994;9(3):104-115.