Journal of Community Health Vol. 3, No. 3, Spring 1978
DIRECT AND I N D I R E C T MEASURES OF P A T I E N T S A T I S F A C T I O N W I T H PHYSICIANS' SERVICES Moira A. Stewart, Ph.D., a n d J a m e s Wanklin, Ph.D.
ABSTRACT: This research examines alternative measures of patient satisfaction. Three measures were compared: (1) a direct measure to evaluate how the patient felt about his own personal physician, (2) an indirect measure that assessed attitudes about physicians in general, and (3) a mneasure designed to be intermediate between these two. Responses to the three measures were found to differ: the levels of satisfaction increased with the directness of the measure used; indirect evidence that this relationship could not be attributed solely to a patient's reluctance to criticize his own physician is also provided. The three measures were compared in terms of their association with other assessments of outcome and with indicators of the process of care. Although none of the associations was statistically significant, high scores on the intermediate measure tended to correspond with better outcomes and higher scores on the process of care. These findings are at least compatible with the contention that an intermediate measure provides the most valid assessment of patient satisfaction. M e t h o d s for obtaining accurate assessments of the public's opinions a b o u t medical care are c u r r e n t l y attracting m o r e attention. I n the field o f health services research, patient satisfaction has b e c o m e an i m p o r t a n t m e a s u r e of this care. R e s e a r c h e r s seek answers to the question, H o w best should we assess what p e o p l e think o f their medical care? A l t h o u g h questions a b o u t what items should be included a n d h o w these items should be scaled are a m o n g the m a n y issues to be decided, the p r e s e n t p a p e r focuses on only one c o n c e r n - - w h e t h e r it is m o r e a p p r o p r i a t e to ask questions a b o u t physicians in general t h a n a b o u t the r e s p o n dent's own physician. It has recently b e e n p r o p o s e d that patients hesitate to criticize their own doctors b u t are less reticent w h e n asked a b o u t doctors in general. M o r e o v e r , it is a r g u e d , the patients' responses w h e n asked a b o u t doctors in general are actually b a s e d u p o n their experiences with their o w n doctors and, a s s u m i n g at the same time that reticence is minimized, the indirect m e a s u r e o f patient satisfaction (questions a b o u t doctors in general) m a y be p r e f e r a b l e to a direct m e a s u r e (questions a b o u t the r e s p o n d e n t ' s own doctor), t'z T h e alternative viewpoint can also be f o u n d in the literature: that the direct a n d indirect as-
Dr. Stewart, a National Health Research Scholar, was formerly with the Department of Epidemiology and Preventive Medicine, University of Western Ontario, and is now with the Department of Preventive Medicine, Dalhousie University, Halifax, Nova Scotia, Canada, B3H 4H7. Dr. Wanklin, is Professor in the Department of Epidemiology and Preventive Medicine, University of Western Ontario. Dr. Stewart has been supported by a Fellowship from the Physician's Services Incorporated Foundation (Ontario) and by a National Health Research Scholarship from Health and Welfare Canada. T h e authors wish to thank Drs. W. Weston, B. Halliday, D. Fuller, A. Hegde, P. Bartlett, and the staff and patients of the Tavistock Family Health Centre for their participation in this research. 0094-5145/78/1300-01955009501978 Human SciencesPress
195
196
JOURNAL OF COMMUNITY HEALTH
sessments actually measure different things, thereby making indirect measures invalid substitutes for direct measures? In an attempt to clarify this issue, a comparison was made of three measures of patient satisfaction: (1) a direct measure, that is, how the patient felt about his own personal physician, (2) an intermediate measure, and (3) an indirect measure about physicians in general. It was postulated that the levels of satisfaction reported would increase with the directness of the measure used. It was further postulated that the direct measure was more likely than the intermediate or indirect ones to show a positive association between satisfaction scores and patient recovery as well as indicators of the process of care. Clearly, the appropriateness of an indirect measure of patient satisfaction depends in large part upon the extent to which opinions concerning physicians in general differ from those held about a personal physician. Several studies have obtained assessments of both general and personal response patterns. However, none of these studies was designed to answer the question at hand; therefore no firm conclusions are possible. The first of these studies, by Cahal, found favorable opinions about personal physicians, but unfavorable opinions about doctors in general. 4 His results were not surprising, because the source of the data for the personal assessments was a survey based on patients' opinions, while the general assessment was based upon reports taken from newspapers and other news media. The second study, by Apostle and Oder, found little difference between personal and general response patterns. 5 Unfortunately, in this study the general and personal assessments were based upon different sets of questions. In a third study, by Fisher, only two of the many questions posed were the same and asked both about the personal physician and doctors in general. 6 One question showed a higher percentage of positive opinions about the personal physician than about doctors in general, while the second question resulted in similar opinions about personal physicians and doctors in general. These three studies showed no consistent evidence that personal and general response patterns differ markedly. More clear-cut differences, on the other hand, were found in two other studies. The first, a nationwide survey of households conducted by Andersen and associates, found 76% of the respondents agreeing that there is a crisis in health care in the United States today, while indicating only infrequent dissatisfaction with specific aspects of their own care, i.e., only 10% indicated dissatisfaction with the quality of the care they had recently received. 7 In the second, a study by Alpert and associates, 750 low-income families were allocated at random to three groups: (1) an experimental group that received family-focused pediatric care, (2) a control group that received the usual outpatient care, and (3) a non-attention control group. 8 At the end of the study period, the three groups were comparable in terms of their attitudes toward physicians in general, but they differed in terms of their satisfaction with the personal care they had received. The most satisfied group was the one that had received family-focused care. These observations suggest both that personal assessments are not necessarily
Stewart, Wanklin
19 7
invalidated by a blanket reticence on the part of patients to criticize their own doctors, and also that it may not be possible to elicit levels of personal satisfaction through questions about doctors in general. One approach to the measurement of patient satisfaction that is relevant to the present study is the one used by Hulka 1"2and Zyzanski2 This measure was designed to avoid reticence on the part of the respondents; but it does rely on the assumption that the respondentls feelings about doctors in general are based largely on his own personal experiences. Accordingly, the instructions ask the respondent to think about the care he is currently receiving as he responds to statements about doctors in general. Hulka and Zyzanski believe that the response patterns produced are probably intermediate between the general and the personal. They found support for the construct validity of their questionnaires and, when they tested its reliability, it was found to be 0.80. While we acknowledge a debt to these investigators for developing a scale to measure patient satisfaction, we were not encouraged on the basis of previous studies that had included both direct and indirect measures to accept the validity of an indirect approach. However, we could find no studies that had been designed specifically to assess the relative merits of the two approaches.
METHOD
The comparison reported here was a part of a larger study reported elsewhere. '° It was based on a sample of 319 patients, 25 years of age or over, who were found to be suffering from at least one chronic condition at the time of their initial study visit to one of the five participating family physicians. Three months after the visit, each patient was interviewed to elicit his personal assessment of the outcome and his level of satisfaction with the care received. An assessment was also made of the process of patient care during the three-month period. Measure o f Satisfaction
The items of satisfaction used were taken from Hulka and Zyzanski.x'~'s The Hulka-Zyzanski questionnaire consists of three categories of items: (1) 14 items relate to the physician's professional competence, (2) 14 items relate to the cost and convenience of the care received, and (3) 14 items relate to the physician's personal qualities. The third category was believed to bear most directly upon the issue under investigation and was the only one used in the present study. The 319 patients available for study were divided at random into three groups: Two of the groups were asked to respond to the items as if they referred to doctors in general. One of these groups was asked, as on the Hulka-Zyzanski questionnaire, to respond in the light of the personal care they were receiving
198
JOURNAL OF COMMUNITY HEALTH
(intermediate measure), while no such instruction was given to the second group (indirect measure). For the third group, the items were rephrased so as to refer directly to the patient's personal physician (direct measure). Satisfaction Scores
On the Hulka-Zyzanski questionnaire, a numerical value, derived by the Thurstone scaling method, is attached to each item. For every patient in the present study, a satisfaction score was calculated by averaging the scale values of all items with which the respondent agreed. Higher scores represented greater satisfaction. Two of the 14 items originally contained in the questionnaire (items 2 and 5) were omitted from the scoring procedure because a high percentage of our patients misinterpreted their meaning. Outcome Measures
Six subjective measures of outcome were elicited at the interview, which took place three months after the patient entered the study: (1) the patient's perception of the degree to which the doctor had helped, (2) the patient's general assessment of his progress, (3) the patient's perception of recovery from discomfort, (4) from worry, (5) from the disturbances of daily living, and (6) from social problems. Indicators of the Process of Care
The nine indicators of the process of patient care during the three-month study period are described in detail elsewhere. 1° Five were measures of the physician's knowledge of different kinds of patient problems and were based on a comparison of two sets of data: (1) information on the patient's problems ascertained through an interview with one of the investigators at the beginning of the study period, and (2) data from a questionnaire on each patient and his problems, completed by the physician. The remaining four indicators concerned the physician's management activity in response to the different kinds of patient problems. The patient's medical record was the source of the data; from it we ascertained the existence, rather than the kind, of physician responses made during the study period.
RESULTS
Of the 319 patients who were eligible for the follow-up interview, 20 patients, or 6.3%, were either missed or refused to be interviewed. The results are therefore based on the 299 patients for whom a complete set of data was obtained.
Stewart, W a n k l i n
199
40i
\
\
""..
NXN
S\\'1
\\\\\\~, t/1 t--
Z
x\\~l
30
t.tJ • "
Z 0
\\\ \\\
(1. ta.J
20
ew,
\\\
I.L
\\\ -\-\-\ x\ \
0
\\\%N\~l-,
I.--
Z
~ . \ X ' ~
"
" x
10
ev, i,i
" x
% X X N I x
~,\\X
",, ",, x
\\\'~
\\\,,
\\\
~, \ \'w
\\\',1
(1.
\ \ \ \\\',1 \\\\\\'~ imB**°*°*"
0
I
35
I
!
I
I
40 45 50 55 60 65 70 7 SCORE
I
I
80
85
K-~-~SCORES FROM PRESENT STUDY ..... SCORES FROM STUDY BY HULKA ET AL. FIGURE 1: Satisfaction scores for the present study compared with scores from Hulka and associates. 2 The distribution of scores from a more recent study by Hulka and associates 11 is not shown because it used transformed scale values not used in the present study. Figure 1 shows the overall distribution o f scores for the 299 patients. It can be seen that the distribution is similar to o n e obtained by H u l k a . 2 Both are positively skewed a n d h a v e c o m p a r a b l e ranges, in spite of the m a n y differences in the design of the two studies. As expected, the r a n d o m allocation created t h r e e g r o u p s o f patients who were c o m p a r a b l e in t e r m s o f their age, sex, level o f education, a n d the 16 o t h e r variables that were investigated. T h e Median T e s t was used to evaluate the statistical significance o f a n y differences in satisfaction scores a m o n g the t h r e e groups. F o r this test, "high" scores were d e f i n e d as those a b o v e 69.2, the m e d i a n for the t h r e e g r o u p s corn-
200
JOURNAL OF COMMUNITY HEALTH
TABLE 1 Percentage Distribution of High and Low Satisfaction Scores According to T y p e o f Questionnaire
Type of Questionnaire
Low 69.2
Total
No.
Personal Intermediate General
43.0 53.9 71.1
57.0 46.1 28.9
100.0 100.0 100.0
100 102 97
Total
55.9
44.1
100.0
299
Median test, chi square = 16.04, df = 8, p < 0.001.
b i n e d ; "low" s c o r e s w e r e t h o s e o n o r b e l o w t h e m e d i a n . T a b l e 1 s h o w s t h a t t h e differences in scores among the three groups were highly significant. The diff e r e n c e s , m o r e o v e r , w e r e in t h e d i r e c t i o n t h a t h a d b e e n p o s t u l a t e d , w i t h a n overrepresentation of high scores on the personal measure and an overrepres e n t a t i o n o f low s c o r e s o n t h e g e n e r a l m e a s u r e , w h i l e t h e i n t e r m e d i a t e m e a s u r e fell in b e t w e e n .
TABLE 2 Percentage o f Patients in A g r e e m e n t with Each Item, According to T y p e o f Questionnaire
Patients Agreeing (%) Personal
Intermediate
General
90
90
99
100
96
98
T o o many doctors think you cannot u n d e r s t a n d the medical explanation o f your illness, so they do not bother explaining
8
14
25
A lot of doctors do not care whether or not they h u r t you during the examination
0
2
5
Many doctors treat the disease, but have no feeling for the patient
1
3
11
Doctors should be a little m o r e friendly than they are
2
6
21
Item You cannot expect any one doctor to be perfect A doctor's j o b is to make people feel better
Most doctors let you talk out your problems
97
98
96
Doctors do their best to keep you from worrying
93
95
95
Doctors are devoted to their patients
94
85
79
0
0
3
42
48
67
100
99
99
Doctors have no feeling for tl~eir patients With so many patients to see, doctors cannot get to know them all Most doctors take a real interest in their patients
Stewart, Wanldin
201
Table 2 shows the principal source of these differences. The patients who were questioned in terms of the general measure of satisfaction seemed to agree more often than the patients who were questioned on the basis of the other two measures, particularly with the rather uncomplimentary items, notably items 3, 4, 5, and 6. On the whole, the patients did tend to be more critical when they were asked about doctors in general rather than about their own physician. There are two possible explanations for this. There may, on the one hand, simply have been a reluctance on the part of the patients to make personal criticisms. On the other hand, it is also possible that two distinct, separate sets of opinions were elicited--one concerning doctors in general and the other concerning the patient's own doctor. An attempt was made to assess the first possibility by evaluating the effect of the setting in which the interview took place upon the scores. Some of the interviews took place in the doctor's office, but most of them were conducted in the patient's own home. It seemed reasonable to expect, therefore, that any reluctance on the part of a patient to criticize his own doctor would be greater when the interview took place at the office. Table 3 shows that the percentage of high scores was essentially the same for the two settings, and that this similarity applied to all three measures. This finding would seem to lend some credibility to the argument that there are distinct sets of opinions toward physicians in general and one's personal physician. The alternative argument, that patients appear more critical of doctors in general simply because they are reluctant to criticize their personal physician, is not supported by our finding, although a reluctance to criticize cannot be entirely ruled out on the basis of such indirect evidence. One might reasonably expect that favorable outcomes, such as recovery from a recent illness, would be positively related to the patients' satisfaction. When the relationships between the subjective measures of outcome and the three measures of satisfaction were assessed, none was found to be statistically significant. However, in Table 4, we can see that, for some of the cross-
TABLE 3 Percentage of Patients with High Satisfaction Scores (> 69.2), According to Type of Questionnaire and Setting of Interview Setting of Interview
Type of Questionnaire No. Personal Intermediate General
75 70 75
Home
Office
% with High Scores No.
% with High Scores
57.3 41.4 30.7
25 32 22
56.0 56.2 22.7
202
JOURNAL OF COMMUNITY HEALTH TABLE 4 Percentage of Patients with High Satisfaction Scores According to Type of Questionnaire Shown for the Six Measures of Outcome
Measure of Outcome
Personal
Intermediate
General
Patient's perception of the degree to which the doctor helped A lot A little or not at all Ratio: lot/little or not
60.0 56.5 1.06
50.0 44,7 1,12
39.1 25.7 1.52
Patient's general assessment of progress Better Same or worse Ratio: better/same or worse
63.9 46.2 1.38
48.2 43,5 1.11
25,5 32.6 0.78
Perceived recovery from discomforts Better Same or worse Ratio: better/same or worse
55.8 57.6 0.97
56.1 44.8 1.25
21.4 36.8 0,54
Perceived recovery from worries Better Same or worse Ratio: better/same or worse
47,4 59.2 0.80
61.5 45.3 1.36
9.1 34.0 0.27
Perceived recovery from disturbances of daily living Better Same or worse Ratio: better/same or worse
70.6 50.0 1.41
81,0 43.8 1.85
27.8 31.8 0,87
Perceived recovery from social problems Better Same or worse Ratio: better/same or worse
55.6 54.5 1.02
39,1 50.0 0.78
25,0 26.2 0.95
tabulations at least, t h e r e was a t e n d e n c y t o w a r d h i g h e r satisfaction scores a m o n g those patients with favorable outcomes. This expected tendency, revealed by those ratios in T a b l e 4 that exceeded 1.00, were f o u n d in f o u r o f the six cross-tabulations o f outcomes with the personal m e a s u r e of satisfaction, five o f the six with the intermediate measure, a n d one o f the six with the general measure. T h e r e f o r e , the general m e a s u r e seemed the least consistent, while the i n t e r m e d i a t e one was most consistent with the outcomes. It was r e a s o n e d that w h e n the process o f care was given a p e r f e c t score the patients m i g h t be expected to feel m o r e satisfied than did the patients whose care did not receive a perfect score. Table 5 shows w h e r e this expectation was u p h e l d in those ratios that were g r e a t e r than 1.00. T h e expected trends were f o u n d in seven o f the nine relationships w h e n the intermediate m e a s u r e was used, b u t in
Stewart, Wanklin
TABLE 5 Percentage o f Patients with High Satisfaction Scores According to T y p e o f Questionnaire Shown for the Nine Indicators o f the Process o f Care
Indicator of the Process of Care
Personal
Intermediate
General
Physician's knowledge o f patient's complaints Perfect score O t h e r score Ratio: perfect/other
44.2 66.7 0.66
47.1 45.1 1.04
24.4 32.7 0.75
Physician's knowledge o f patient's discomforts Perfect score O t h e r score Ratio: perfect/other
48.8 64.0 0.76
58.5 40.8 1.43
25.0 40.5 0.62
Physician's knowledge of patient's wories Perfect score O t h e r score Ratio: perfect/other
41.7 71.9 0.58
60.5 32.1 1.88
24.2 35.7 0.68
Physician's knowledge o f patient's disturbances o f daily living Perfect score O t h e r score Ratio: perfect/other
50.0 63.0 0.79
57.4 50.0 1.15
26.3 37.5 0.70
Physician's knowledge o f patient's social problems Perfect score O t h e r score Ratio: perfect/other
71.4 49.2 1.45
44.4 47.5 0.93
28.6 25.0 1.14
Physician's response to discomforts Perfect score O t h e r score Ratio: perfect/other
57.0 57.1 1.00
50.6 33.3 1.52
33.8 18.2 1.86
Physician's response to worries Perfect score O t h e r score Ratio: perfect/other
42.1 61.2 0.69
48.1 48.7 0.99
21.1 33.3 0.63
Physician's response to disturbances o f daily living Perfect score O t h e r score Ratio: perfect/other
28.6 58.6 0.49
81.8 50.0 1.64
27.3 31.4 0.87
Physician's response to social problems Perfect score O t h e r score Ratio: perfect/other
47.1 56.9 0.83
58.8 43.3 1.36
33.3 23.6 1.41
203
204
JOURNAL OF COMMUNITY H E A L T H
only one o f the relationships w h e n the p e r s o n a l m e a s u r e was used, a n d in only t h r e e using the g e n e r a l m e a s u r e .
DISCUSSION
T h e findings o f this study suggest that r e s p o n s e p a t t e r n s are d i f f e r e n t w h e n b a s e d on general attitudes a n d on p e r s o n a l experience. N o s u p p o r t was f o u n d for the a s s u m p t i o n that patients r e s p o n d to questions a b o u t doctors in general as t h o u g h they are thinking a b o u t their own doctors, except p e r h a p s w h e n they are specifically instructed to d o so. T h e purely indirect m e a s u r e o f satisfaction (questions a b o u t physicians in general) was quite striking in its failure to attain positive associations with patients' r e c o v e r y a n d indicators o f the process o f care. O f the r e m a i n i n g two m e a s u r e s of satisfaction, the i n t e r m e d i a t e , r a t h e r t h a n the completely direct one, a p p e a r e d m o r e valid because it e v o k e d r e s p o n s e s that t e n d e d to be associated with the patient's r e c o v e r y a n d the kind o f care t h a t he h a d received. Reluctance on the p a r t of patients to criticize their own physicians s e e m e d to be the m o s t plausible e x p l a n a t i o n for the b e t t e r p e r f o r m a n c e o f the i n t e r m e d i a t e m e a s u r e , in spite o f o u r indirect evidence that such reluctance alone could not explain the differences a m o n g the t h r e e m e a s u r e s . Because the i n t e r m e d i a t e m e a s u r e was identical to the o n e d e v e l o p e d by H u l k a , la'9 the results o f the study a r e c o m p a t i b l e with the contention that H u l k a ' s a p p r o a c h is the m o s t valid o f the t h r e e alternatives.
REFERENCES
1, Hulka BS, Zyzanski SJ, CasselJC, et al: Scale for the measurement of attitudes toward physicians and primary medical care. Med Care 8: 429-436, 1970. 2. Hulka BS, Zyzanski SJ, Cassel JC, et al: Satisfaction with medical care in a low-income population.J Chronic Dis 24" 661-673, 1971. 3. Howell JR, Osterweis M, Huntley RR: Curing and caring--A proposed method for selfassessment in primary care organizations.J Community Health 1: 256-275, 1976. 4. Cahal MF: What the public thinks of the family doctor--folklore and facts. Gen Pract 25: 146-157, 1962. 5. Apostle DA, Oder, F: Factors that influence the public's view of medical care.JAMA 202: 140146, 1967. 6. Fisher AW: Patients' evaluation of outpatient medical care.J Med Educ 46: 238-244, 1971. 7. Andersen R, Kravits J, Anderson OW: The public's view of the crises in medical care: An impetus for changing delivery systems? Economic andBusiness Bulletin 24: 44-52, 1971. 8. Alpert JJ, Kosa J, Haggerty RJ, et al: Attitudes and satisfaction of low-income families receiving comprehensive pediatric care. Am J Public Health 60: 499-506, 1970. 9. Zyzanski SJ, Hulka BS, CasselJC: Scale for the measurement of "satisfaction" with medical care: Modifications in content, format and scoring. Med Care 12:611-620, 1974. 10. Stewart MA, Buck CW: Physicians' knowledge of and response to patients' problems. Med Care 15: 578-585, 1977. 11. Hulka BS. Kupper LL, Daly MB, et al: Correlates of satisfaction and dissatisfaction with medical care: A community perspective. Med Care 13: 648-658, 1975.
DIRECT AND I N D I R E C T MEASURES OF P A T I E N T S A T I S F A C T I O N W I T H PHYSICIANS' SERVICES Moira A. Stewart, Ph.D., a n d J a m e s Wanklin, Ph.D.
ABSTRACT: This research examines alternative measures of patient satisfaction. Three measures were compared: (1) a direct measure to evaluate how the patient felt about his own personal physician, (2) an indirect measure that assessed attitudes about physicians in general, and (3) a mneasure designed to be intermediate between these two. Responses to the three measures were found to differ: the levels of satisfaction increased with the directness of the measure used; indirect evidence that this relationship could not be attributed solely to a patient's reluctance to criticize his own physician is also provided. The three measures were compared in terms of their association with other assessments of outcome and with indicators of the process of care. Although none of the associations was statistically significant, high scores on the intermediate measure tended to correspond with better outcomes and higher scores on the process of care. These findings are at least compatible with the contention that an intermediate measure provides the most valid assessment of patient satisfaction. M e t h o d s for obtaining accurate assessments of the public's opinions a b o u t medical care are c u r r e n t l y attracting m o r e attention. I n the field o f health services research, patient satisfaction has b e c o m e an i m p o r t a n t m e a s u r e of this care. R e s e a r c h e r s seek answers to the question, H o w best should we assess what p e o p l e think o f their medical care? A l t h o u g h questions a b o u t what items should be included a n d h o w these items should be scaled are a m o n g the m a n y issues to be decided, the p r e s e n t p a p e r focuses on only one c o n c e r n - - w h e t h e r it is m o r e a p p r o p r i a t e to ask questions a b o u t physicians in general t h a n a b o u t the r e s p o n dent's own physician. It has recently b e e n p r o p o s e d that patients hesitate to criticize their own doctors b u t are less reticent w h e n asked a b o u t doctors in general. M o r e o v e r , it is a r g u e d , the patients' responses w h e n asked a b o u t doctors in general are actually b a s e d u p o n their experiences with their o w n doctors and, a s s u m i n g at the same time that reticence is minimized, the indirect m e a s u r e o f patient satisfaction (questions a b o u t doctors in general) m a y be p r e f e r a b l e to a direct m e a s u r e (questions a b o u t the r e s p o n d e n t ' s own doctor), t'z T h e alternative viewpoint can also be f o u n d in the literature: that the direct a n d indirect as-
Dr. Stewart, a National Health Research Scholar, was formerly with the Department of Epidemiology and Preventive Medicine, University of Western Ontario, and is now with the Department of Preventive Medicine, Dalhousie University, Halifax, Nova Scotia, Canada, B3H 4H7. Dr. Wanklin, is Professor in the Department of Epidemiology and Preventive Medicine, University of Western Ontario. Dr. Stewart has been supported by a Fellowship from the Physician's Services Incorporated Foundation (Ontario) and by a National Health Research Scholarship from Health and Welfare Canada. T h e authors wish to thank Drs. W. Weston, B. Halliday, D. Fuller, A. Hegde, P. Bartlett, and the staff and patients of the Tavistock Family Health Centre for their participation in this research. 0094-5145/78/1300-01955009501978 Human SciencesPress
195
196
JOURNAL OF COMMUNITY HEALTH
sessments actually measure different things, thereby making indirect measures invalid substitutes for direct measures? In an attempt to clarify this issue, a comparison was made of three measures of patient satisfaction: (1) a direct measure, that is, how the patient felt about his own personal physician, (2) an intermediate measure, and (3) an indirect measure about physicians in general. It was postulated that the levels of satisfaction reported would increase with the directness of the measure used. It was further postulated that the direct measure was more likely than the intermediate or indirect ones to show a positive association between satisfaction scores and patient recovery as well as indicators of the process of care. Clearly, the appropriateness of an indirect measure of patient satisfaction depends in large part upon the extent to which opinions concerning physicians in general differ from those held about a personal physician. Several studies have obtained assessments of both general and personal response patterns. However, none of these studies was designed to answer the question at hand; therefore no firm conclusions are possible. The first of these studies, by Cahal, found favorable opinions about personal physicians, but unfavorable opinions about doctors in general. 4 His results were not surprising, because the source of the data for the personal assessments was a survey based on patients' opinions, while the general assessment was based upon reports taken from newspapers and other news media. The second study, by Apostle and Oder, found little difference between personal and general response patterns. 5 Unfortunately, in this study the general and personal assessments were based upon different sets of questions. In a third study, by Fisher, only two of the many questions posed were the same and asked both about the personal physician and doctors in general. 6 One question showed a higher percentage of positive opinions about the personal physician than about doctors in general, while the second question resulted in similar opinions about personal physicians and doctors in general. These three studies showed no consistent evidence that personal and general response patterns differ markedly. More clear-cut differences, on the other hand, were found in two other studies. The first, a nationwide survey of households conducted by Andersen and associates, found 76% of the respondents agreeing that there is a crisis in health care in the United States today, while indicating only infrequent dissatisfaction with specific aspects of their own care, i.e., only 10% indicated dissatisfaction with the quality of the care they had recently received. 7 In the second, a study by Alpert and associates, 750 low-income families were allocated at random to three groups: (1) an experimental group that received family-focused pediatric care, (2) a control group that received the usual outpatient care, and (3) a non-attention control group. 8 At the end of the study period, the three groups were comparable in terms of their attitudes toward physicians in general, but they differed in terms of their satisfaction with the personal care they had received. The most satisfied group was the one that had received family-focused care. These observations suggest both that personal assessments are not necessarily
Stewart, Wanklin
19 7
invalidated by a blanket reticence on the part of patients to criticize their own doctors, and also that it may not be possible to elicit levels of personal satisfaction through questions about doctors in general. One approach to the measurement of patient satisfaction that is relevant to the present study is the one used by Hulka 1"2and Zyzanski2 This measure was designed to avoid reticence on the part of the respondents; but it does rely on the assumption that the respondentls feelings about doctors in general are based largely on his own personal experiences. Accordingly, the instructions ask the respondent to think about the care he is currently receiving as he responds to statements about doctors in general. Hulka and Zyzanski believe that the response patterns produced are probably intermediate between the general and the personal. They found support for the construct validity of their questionnaires and, when they tested its reliability, it was found to be 0.80. While we acknowledge a debt to these investigators for developing a scale to measure patient satisfaction, we were not encouraged on the basis of previous studies that had included both direct and indirect measures to accept the validity of an indirect approach. However, we could find no studies that had been designed specifically to assess the relative merits of the two approaches.
METHOD
The comparison reported here was a part of a larger study reported elsewhere. '° It was based on a sample of 319 patients, 25 years of age or over, who were found to be suffering from at least one chronic condition at the time of their initial study visit to one of the five participating family physicians. Three months after the visit, each patient was interviewed to elicit his personal assessment of the outcome and his level of satisfaction with the care received. An assessment was also made of the process of patient care during the three-month period. Measure o f Satisfaction
The items of satisfaction used were taken from Hulka and Zyzanski.x'~'s The Hulka-Zyzanski questionnaire consists of three categories of items: (1) 14 items relate to the physician's professional competence, (2) 14 items relate to the cost and convenience of the care received, and (3) 14 items relate to the physician's personal qualities. The third category was believed to bear most directly upon the issue under investigation and was the only one used in the present study. The 319 patients available for study were divided at random into three groups: Two of the groups were asked to respond to the items as if they referred to doctors in general. One of these groups was asked, as on the Hulka-Zyzanski questionnaire, to respond in the light of the personal care they were receiving
198
JOURNAL OF COMMUNITY HEALTH
(intermediate measure), while no such instruction was given to the second group (indirect measure). For the third group, the items were rephrased so as to refer directly to the patient's personal physician (direct measure). Satisfaction Scores
On the Hulka-Zyzanski questionnaire, a numerical value, derived by the Thurstone scaling method, is attached to each item. For every patient in the present study, a satisfaction score was calculated by averaging the scale values of all items with which the respondent agreed. Higher scores represented greater satisfaction. Two of the 14 items originally contained in the questionnaire (items 2 and 5) were omitted from the scoring procedure because a high percentage of our patients misinterpreted their meaning. Outcome Measures
Six subjective measures of outcome were elicited at the interview, which took place three months after the patient entered the study: (1) the patient's perception of the degree to which the doctor had helped, (2) the patient's general assessment of his progress, (3) the patient's perception of recovery from discomfort, (4) from worry, (5) from the disturbances of daily living, and (6) from social problems. Indicators of the Process of Care
The nine indicators of the process of patient care during the three-month study period are described in detail elsewhere. 1° Five were measures of the physician's knowledge of different kinds of patient problems and were based on a comparison of two sets of data: (1) information on the patient's problems ascertained through an interview with one of the investigators at the beginning of the study period, and (2) data from a questionnaire on each patient and his problems, completed by the physician. The remaining four indicators concerned the physician's management activity in response to the different kinds of patient problems. The patient's medical record was the source of the data; from it we ascertained the existence, rather than the kind, of physician responses made during the study period.
RESULTS
Of the 319 patients who were eligible for the follow-up interview, 20 patients, or 6.3%, were either missed or refused to be interviewed. The results are therefore based on the 299 patients for whom a complete set of data was obtained.
Stewart, W a n k l i n
199
40i
\
\
""..
NXN
S\\'1
\\\\\\~, t/1 t--
Z
x\\~l
30
t.tJ • "
Z 0
\\\ \\\
(1. ta.J
20
ew,
\\\
I.L
\\\ -\-\-\ x\ \
0
\\\%N\~l-,
I.--
Z
~ . \ X ' ~
"
" x
10
ev, i,i
" x
% X X N I x
~,\\X
",, ",, x
\\\'~
\\\,,
\\\
~, \ \'w
\\\',1
(1.
\ \ \ \\\',1 \\\\\\'~ imB**°*°*"
0
I
35
I
!
I
I
40 45 50 55 60 65 70 7 SCORE
I
I
80
85
K-~-~SCORES FROM PRESENT STUDY ..... SCORES FROM STUDY BY HULKA ET AL. FIGURE 1: Satisfaction scores for the present study compared with scores from Hulka and associates. 2 The distribution of scores from a more recent study by Hulka and associates 11 is not shown because it used transformed scale values not used in the present study. Figure 1 shows the overall distribution o f scores for the 299 patients. It can be seen that the distribution is similar to o n e obtained by H u l k a . 2 Both are positively skewed a n d h a v e c o m p a r a b l e ranges, in spite of the m a n y differences in the design of the two studies. As expected, the r a n d o m allocation created t h r e e g r o u p s o f patients who were c o m p a r a b l e in t e r m s o f their age, sex, level o f education, a n d the 16 o t h e r variables that were investigated. T h e Median T e s t was used to evaluate the statistical significance o f a n y differences in satisfaction scores a m o n g the t h r e e groups. F o r this test, "high" scores were d e f i n e d as those a b o v e 69.2, the m e d i a n for the t h r e e g r o u p s corn-
200
JOURNAL OF COMMUNITY HEALTH
TABLE 1 Percentage Distribution of High and Low Satisfaction Scores According to T y p e o f Questionnaire
Type of Questionnaire
Low 69.2
Total
No.
Personal Intermediate General
43.0 53.9 71.1
57.0 46.1 28.9
100.0 100.0 100.0
100 102 97
Total
55.9
44.1
100.0
299
Median test, chi square = 16.04, df = 8, p < 0.001.
b i n e d ; "low" s c o r e s w e r e t h o s e o n o r b e l o w t h e m e d i a n . T a b l e 1 s h o w s t h a t t h e differences in scores among the three groups were highly significant. The diff e r e n c e s , m o r e o v e r , w e r e in t h e d i r e c t i o n t h a t h a d b e e n p o s t u l a t e d , w i t h a n overrepresentation of high scores on the personal measure and an overrepres e n t a t i o n o f low s c o r e s o n t h e g e n e r a l m e a s u r e , w h i l e t h e i n t e r m e d i a t e m e a s u r e fell in b e t w e e n .
TABLE 2 Percentage o f Patients in A g r e e m e n t with Each Item, According to T y p e o f Questionnaire
Patients Agreeing (%) Personal
Intermediate
General
90
90
99
100
96
98
T o o many doctors think you cannot u n d e r s t a n d the medical explanation o f your illness, so they do not bother explaining
8
14
25
A lot of doctors do not care whether or not they h u r t you during the examination
0
2
5
Many doctors treat the disease, but have no feeling for the patient
1
3
11
Doctors should be a little m o r e friendly than they are
2
6
21
Item You cannot expect any one doctor to be perfect A doctor's j o b is to make people feel better
Most doctors let you talk out your problems
97
98
96
Doctors do their best to keep you from worrying
93
95
95
Doctors are devoted to their patients
94
85
79
0
0
3
42
48
67
100
99
99
Doctors have no feeling for tl~eir patients With so many patients to see, doctors cannot get to know them all Most doctors take a real interest in their patients
Stewart, Wanldin
201
Table 2 shows the principal source of these differences. The patients who were questioned in terms of the general measure of satisfaction seemed to agree more often than the patients who were questioned on the basis of the other two measures, particularly with the rather uncomplimentary items, notably items 3, 4, 5, and 6. On the whole, the patients did tend to be more critical when they were asked about doctors in general rather than about their own physician. There are two possible explanations for this. There may, on the one hand, simply have been a reluctance on the part of the patients to make personal criticisms. On the other hand, it is also possible that two distinct, separate sets of opinions were elicited--one concerning doctors in general and the other concerning the patient's own doctor. An attempt was made to assess the first possibility by evaluating the effect of the setting in which the interview took place upon the scores. Some of the interviews took place in the doctor's office, but most of them were conducted in the patient's own home. It seemed reasonable to expect, therefore, that any reluctance on the part of a patient to criticize his own doctor would be greater when the interview took place at the office. Table 3 shows that the percentage of high scores was essentially the same for the two settings, and that this similarity applied to all three measures. This finding would seem to lend some credibility to the argument that there are distinct sets of opinions toward physicians in general and one's personal physician. The alternative argument, that patients appear more critical of doctors in general simply because they are reluctant to criticize their personal physician, is not supported by our finding, although a reluctance to criticize cannot be entirely ruled out on the basis of such indirect evidence. One might reasonably expect that favorable outcomes, such as recovery from a recent illness, would be positively related to the patients' satisfaction. When the relationships between the subjective measures of outcome and the three measures of satisfaction were assessed, none was found to be statistically significant. However, in Table 4, we can see that, for some of the cross-
TABLE 3 Percentage of Patients with High Satisfaction Scores (> 69.2), According to Type of Questionnaire and Setting of Interview Setting of Interview
Type of Questionnaire No. Personal Intermediate General
75 70 75
Home
Office
% with High Scores No.
% with High Scores
57.3 41.4 30.7
25 32 22
56.0 56.2 22.7
202
JOURNAL OF COMMUNITY HEALTH TABLE 4 Percentage of Patients with High Satisfaction Scores According to Type of Questionnaire Shown for the Six Measures of Outcome
Measure of Outcome
Personal
Intermediate
General
Patient's perception of the degree to which the doctor helped A lot A little or not at all Ratio: lot/little or not
60.0 56.5 1.06
50.0 44,7 1,12
39.1 25.7 1.52
Patient's general assessment of progress Better Same or worse Ratio: better/same or worse
63.9 46.2 1.38
48.2 43,5 1.11
25,5 32.6 0.78
Perceived recovery from discomforts Better Same or worse Ratio: better/same or worse
55.8 57.6 0.97
56.1 44.8 1.25
21.4 36.8 0,54
Perceived recovery from worries Better Same or worse Ratio: better/same or worse
47,4 59.2 0.80
61.5 45.3 1.36
9.1 34.0 0.27
Perceived recovery from disturbances of daily living Better Same or worse Ratio: better/same or worse
70.6 50.0 1.41
81,0 43.8 1.85
27.8 31.8 0,87
Perceived recovery from social problems Better Same or worse Ratio: better/same or worse
55.6 54.5 1.02
39,1 50.0 0.78
25,0 26.2 0.95
tabulations at least, t h e r e was a t e n d e n c y t o w a r d h i g h e r satisfaction scores a m o n g those patients with favorable outcomes. This expected tendency, revealed by those ratios in T a b l e 4 that exceeded 1.00, were f o u n d in f o u r o f the six cross-tabulations o f outcomes with the personal m e a s u r e of satisfaction, five o f the six with the intermediate measure, a n d one o f the six with the general measure. T h e r e f o r e , the general m e a s u r e seemed the least consistent, while the i n t e r m e d i a t e one was most consistent with the outcomes. It was r e a s o n e d that w h e n the process o f care was given a p e r f e c t score the patients m i g h t be expected to feel m o r e satisfied than did the patients whose care did not receive a perfect score. Table 5 shows w h e r e this expectation was u p h e l d in those ratios that were g r e a t e r than 1.00. T h e expected trends were f o u n d in seven o f the nine relationships w h e n the intermediate m e a s u r e was used, b u t in
Stewart, Wanklin
TABLE 5 Percentage o f Patients with High Satisfaction Scores According to T y p e o f Questionnaire Shown for the Nine Indicators o f the Process o f Care
Indicator of the Process of Care
Personal
Intermediate
General
Physician's knowledge o f patient's complaints Perfect score O t h e r score Ratio: perfect/other
44.2 66.7 0.66
47.1 45.1 1.04
24.4 32.7 0.75
Physician's knowledge o f patient's discomforts Perfect score O t h e r score Ratio: perfect/other
48.8 64.0 0.76
58.5 40.8 1.43
25.0 40.5 0.62
Physician's knowledge of patient's wories Perfect score O t h e r score Ratio: perfect/other
41.7 71.9 0.58
60.5 32.1 1.88
24.2 35.7 0.68
Physician's knowledge o f patient's disturbances o f daily living Perfect score O t h e r score Ratio: perfect/other
50.0 63.0 0.79
57.4 50.0 1.15
26.3 37.5 0.70
Physician's knowledge o f patient's social problems Perfect score O t h e r score Ratio: perfect/other
71.4 49.2 1.45
44.4 47.5 0.93
28.6 25.0 1.14
Physician's response to discomforts Perfect score O t h e r score Ratio: perfect/other
57.0 57.1 1.00
50.6 33.3 1.52
33.8 18.2 1.86
Physician's response to worries Perfect score O t h e r score Ratio: perfect/other
42.1 61.2 0.69
48.1 48.7 0.99
21.1 33.3 0.63
Physician's response to disturbances o f daily living Perfect score O t h e r score Ratio: perfect/other
28.6 58.6 0.49
81.8 50.0 1.64
27.3 31.4 0.87
Physician's response to social problems Perfect score O t h e r score Ratio: perfect/other
47.1 56.9 0.83
58.8 43.3 1.36
33.3 23.6 1.41
203
204
JOURNAL OF COMMUNITY H E A L T H
only one o f the relationships w h e n the p e r s o n a l m e a s u r e was used, a n d in only t h r e e using the g e n e r a l m e a s u r e .
DISCUSSION
T h e findings o f this study suggest that r e s p o n s e p a t t e r n s are d i f f e r e n t w h e n b a s e d on general attitudes a n d on p e r s o n a l experience. N o s u p p o r t was f o u n d for the a s s u m p t i o n that patients r e s p o n d to questions a b o u t doctors in general as t h o u g h they are thinking a b o u t their own doctors, except p e r h a p s w h e n they are specifically instructed to d o so. T h e purely indirect m e a s u r e o f satisfaction (questions a b o u t physicians in general) was quite striking in its failure to attain positive associations with patients' r e c o v e r y a n d indicators o f the process o f care. O f the r e m a i n i n g two m e a s u r e s of satisfaction, the i n t e r m e d i a t e , r a t h e r t h a n the completely direct one, a p p e a r e d m o r e valid because it e v o k e d r e s p o n s e s that t e n d e d to be associated with the patient's r e c o v e r y a n d the kind o f care t h a t he h a d received. Reluctance on the p a r t of patients to criticize their own physicians s e e m e d to be the m o s t plausible e x p l a n a t i o n for the b e t t e r p e r f o r m a n c e o f the i n t e r m e d i a t e m e a s u r e , in spite o f o u r indirect evidence that such reluctance alone could not explain the differences a m o n g the t h r e e m e a s u r e s . Because the i n t e r m e d i a t e m e a s u r e was identical to the o n e d e v e l o p e d by H u l k a , la'9 the results o f the study a r e c o m p a t i b l e with the contention that H u l k a ' s a p p r o a c h is the m o s t valid o f the t h r e e alternatives.
REFERENCES
1, Hulka BS, Zyzanski SJ, CasselJC, et al: Scale for the measurement of attitudes toward physicians and primary medical care. Med Care 8: 429-436, 1970. 2. Hulka BS, Zyzanski SJ, Cassel JC, et al: Satisfaction with medical care in a low-income population.J Chronic Dis 24" 661-673, 1971. 3. Howell JR, Osterweis M, Huntley RR: Curing and caring--A proposed method for selfassessment in primary care organizations.J Community Health 1: 256-275, 1976. 4. Cahal MF: What the public thinks of the family doctor--folklore and facts. Gen Pract 25: 146-157, 1962. 5. Apostle DA, Oder, F: Factors that influence the public's view of medical care.JAMA 202: 140146, 1967. 6. Fisher AW: Patients' evaluation of outpatient medical care.J Med Educ 46: 238-244, 1971. 7. Andersen R, Kravits J, Anderson OW: The public's view of the crises in medical care: An impetus for changing delivery systems? Economic andBusiness Bulletin 24: 44-52, 1971. 8. Alpert JJ, Kosa J, Haggerty RJ, et al: Attitudes and satisfaction of low-income families receiving comprehensive pediatric care. Am J Public Health 60: 499-506, 1970. 9. Zyzanski SJ, Hulka BS, CasselJC: Scale for the measurement of "satisfaction" with medical care: Modifications in content, format and scoring. Med Care 12:611-620, 1974. 10. Stewart MA, Buck CW: Physicians' knowledge of and response to patients' problems. Med Care 15: 578-585, 1977. 11. Hulka BS. Kupper LL, Daly MB, et al: Correlates of satisfaction and dissatisfaction with medical care: A community perspective. Med Care 13: 648-658, 1975.
