J Clin Epidemiol Vol. 45, No. 7, pp. 715-720, 1992 Britain
0895-4356/92 $5.00 + 0.00
Printedin Great
PergamonPressLtd
COMPARISON OF “ROSE QUESTIONNAIRE ANGINA” TO EXERCISE THALLIUM SCINTIGRAPHY: DIFFERENT FINDINGS IN MALES AND FEMALES CAROL EWING GARBER.* RICHARD A. CARLETON and GARY V. HELLER Human Performance Laboratory, Department of Medicine (Division of Cardiology), Memorial Hospital of Rhode Island, Pawtucket, Brown University School of Medicine, Providence, RI 02912. U.S.A. (Received in revised form I I February 1992)
Abstract-Validation of the London School of Hygiene (Rose) Questionnaire with objective measures of myocardial ischemia is incomplete. Therefore, we compared the Rose Questionnaire with exercise thallium-201 myocardial scintigraphy in 147 male and 97 female patients with chest pain referred for clinical exercise testing. Of those with “Rose Questionnaire angina”, 26% of the females and 73% of the males had positive thallium-201 scans. Negative results on both the Rose Questionnaire and thallium-201 scintigraphy were observed in 71% of the females and 47% of the males. The sensitivity of the Rose Questionnaire was similar in females (41%) and males (44%). The specificity was 77% in males, while in females it was significantly lower at 56%. The specificity values reflect the higher (p < 0.05) prevalence of “false positive” Rose Questionnaire results in females (75%) compared with males (27%). In addition, males had a greater (p c 0.05) number of “false negative” results (53%) than females (29%). The accuracy of the Rose Questionnaire for myocardial ischemia was 0.19 in females, 0.48 in males, and 0.29 overall when including both males and females. Our results indicate a generally poor relationship between Rose Questionnaire angina and thallium-201 scintigraphy, an objective measure of myocardial ischemia in patients with chest pain referred to clinical exercise testing. Further, there are gender-specific differences in this relationship between the questionnaire and exercise thallium-201 imaging. Angina pectoris Epidemiology Thallium-201 myocardial naires
Gender scintigraphy
INTRODUCTION
The determination of the prevalence of angina pectoris in populations is important, especially for epidemiologic and large scale clinical trial research. The clinical diagnosis of angina pectoris by individual assessment is often impractical for population-based studies due to its labor-intensive nature and cost. Thus, an *All correspondence should be addressed to: Carol Ewing Garber, Ph.D., Division of Cardiology, Memorial Hospital of Rhode Island, 111 Brewster Street, Pawtucket, RI 02860, U.S.A. [Tel. 401-729-22611.
Myocardial
ischemia
Question-
easily-administered questionnaire which is reproducible and valid in comparison to wellaccepted clinical standards is desirable. Dr Geoffrey Rose developed the London School of Hygiene (Rose) Questionnaire for the standardized determination of angina pectoris [l-3]. Using male patients with known coronary heart disease, Rose identified several distinctive features defining angina pectoris which were highly specific and moderately sensitive in comparison with physician diagnoses [I]. Although on an individual basis the questionnaire was not highly reproducible, the
716
CAROLEWING GARBERef al.
prevalence of angina pectoris estimated by Rose Questionnaire within a population was very stable [2]. The Rose Questionnaire has been validated against physician diagnosis in both general and patient populations [l-7]. Based upon these studies, it has been generally well accepted as a valid means for measuring angina pectoris in epidemiologic study, with a sensitivity ranging from 25-83% and a specificity of 48-98%. The utility of the Rose Questionnaire for women is unknown. Although the questionnaire has been used extensively in Europe to evaluate women [8-l 11, validation is lacking. Most validation studies have used male subjects, and have either excluded women or included such a small number of female subjects to be statistically uninterpretable. Several studies have reported a higher prevalence of angina pectoris among women younger than 55 years of age, but this “Rose Questionnaire angina” was not predictive of coronary artery disease mortality [8, 12, 131. In older women, however, a predictive relationship was found. Wilcosky and associates found that the relationship between Rose Questionnaire angina and a number of coronary risk factors was gender-specific [12]. From these data and the lack of other studies, it is unclear whether the Rose Questionnaire can adequately assess the prevalence of ischemic coronary artery disease in females. The use of the Rose Questionnaire has extended beyond the assessment of the prevalence of angina pectoris within a general population. It has been used in clinical trials evaluating pharmacologic or other therapies [14] and also has been used to predict subsequent cardiovascular morbidity and mortality [3,7, 141. When the Rose Questionnaire is applied in these ways, it is assumed that angina pectoris reflects underlying myocardial ischemia and/or coronary artery disease. Although angina pectoris, myocardial ischemia and coronary artery stenoses they represent 3 distinct are associated, phenomena which are not necessarily directly related [15]. Thus, it is important to validate Rose Questionnaire angina with measures of myocardial ischemia and coronary anatomy before clinical applications of the Rose Questionnaire can be recommended. The purpose of the present study was to determine the validity of the Rose Questionnaire as a correlate of myocardial ischemia in males and females. We compared the results of the Rose Questionnaire with exercise thallium-
201 myocardial scintigraphy in patients referred for clinical exercise testing. METHODS
Subjects
Two hundred forty-four consecutive patients (147 males and 97 females; mean age 57 + 11 years) with self-reported chest pain or discomfort referred for exercise thallium-201 myocardial scintigraphy at two teaching hospitals completed a graded exercise test (reaching at least 85% of their predicted maximum heart rate) with thallium-201 myocardial scintigraphy and the Rose Questionnaire. Overall, the subjects had a pre-test probability of coronary artery disease of 58 (+ 1l)%, based upon the Diamond & Forrester classification [16]. Test procedures
Each patient filled out the version of the Rose Questionnaire shown in the Appendix which was modified and validated by Heyden et al. [5]. A symptom-limited graded exercise test using a Bruce protocol was then completed [17]. One minute prior to the end of exercise, an intravenous dose (8 1 MBq; 2.2 mCi) of thallium-201 was given. Within 10 minutes following exercise, planar myocardial imaging was obtained with a general all-purpose parallel hole collimator in three standard views: anterior, 45” left anterior oblique and the 70” left anterior oblique. Delayed images were obtained 4 hours later. Data interpretation
All thallium-201 scintigraphic images were interpreted by two experienced cardiologists without knowledge of the study protocol or Rose Questionnaire results. The images were read by consensus using standard criteria [18]. The left ventricular images were divided into 15 segments; each segment was scored on a scale of O-3 (0 = normal, 1 = slight photon emission reduction, 2 = moderate photon emission reduction, 3 = background or absent activity). This procedure has been validated and reported previously [19]. A thallium-201 scan was considered positive for myocardial ischemia when one or more of the images showed a photondeficient area with normalization on delayed images. The thallium-201 scan was negative if there were no photon-deficient areas. Angina pectoris was considered to be present when a subject provided the affirmative responses, as shown in the Appendix, to each of
Rose Questionnaire
the first six questions on the Rose Questionnaire. A negative response to one or more of these questions was classified as angina pectoris absent. The data were computer coded and analyzed using the BMDP statistical package, programs 4F and 3s [20]. Two by two contingency tables were generated and the independence of classification was tested using the Yates corrected Chi Square. Sensitivity, specificity, and the positive and negative predictive values of the Rose Questionnaire in comparison to thallium-201 imaging were determined from the contingency tables using standard equations [21]. The positive and negative likelihood ratios were derived from the sensitivity and specificity figures [22], and indicate the probability that an abnormal Rose Questionnaire result is true for ischemia or the probability of a negative Rose Questionnaire being negative for ischemia, respectively. Youden’s J index was calculated for males, females and the total group [23]. This index is an objective method for comparing the accuracy of two tests and ranges from 0 to 1. Zero indicates that the test yields the same proportion of positive tests in those with and without disease, while one is obtained when there are no false positive or false negative results. RESULTS
The prevalence of Rose Questionnaire angina in our patients was higher (43%) in women than in men (35%; p < 0.05). In contrast, 27% of the females and 59% of the males (p < 0.05) had ischemia on the thallium-201 scan.
717
The classification of patients by Rose Questionnaire and thallium-201 scintigraphy is presented in Table 1. Fifty-two percent (49/94) of the subjects with positive Rose Questionnaire results had evidence of myocardial ischemia by thallium-201 imaging. However, there were differences according to gender. Of males with Rose Questionnaire angina, 73% had a positive thallium-201 scan. In contrast, the agreement between positive results in females was significantly lower (26%; p < 0.05). Of subjects with a negative Rose Questionnaire, 57% had a negative thallium-201 scan. There were also gender differences in the negative results. Females had a significantly (p c 0.05) higher percentage of concurrently negative results (70%) when compared with the males (48%). Accepting the thallium-201 result as a true indicator of ischemia, the overall rates of false positive and false negative Rose Questionnaire results were 48 and 43%, respectively. Females had a significantly higher 0, < 0.05) rate of false positive results (74%) compared with males (27%). On the other hand, males had significantly more false negative results (53%) than females (29%). The sensitivity, specificity and positive predictive value of the Rose Questionnaire compared with myocardial imaging are presented in Table 2. The sensitivity of the Rose Questionnaire was similar for males and females, while the questionnaire was more specific in males than in females. Reflecting these values, both the positive and negative likelihood ratios were substantially lower in females than in males.
Table 1. Comparison of the Rose Questionnaire with exercise thallium-201 scintig raphy in 244 patients with chest pain Thallium-201 imaging Gender
Rose Questionnaire
Female
Positive
Positive
Negative
Total
(7.k) Negative
Male
Total Positive
(2;;) 27
(7:;) 70
(12%) 97
(7::)
(2::)
(Rz%)
(5::) 87
(4;:) 60
(lZ%) 147
(&)
(4:;)
(4;;) 114
,:gg
(KS?%) 150 (100%) 244
Negative Total Group
Total Positive Negative Total
Values are frequency with percent of row total in parentheses below. *p < 0.05 compared with females with same Rose Questionnaire result.
CAROLEWING GARBER et al.
718
Table 2. Sensitivity, specificity and predictive value of the Rose Questionnaire compared with thallium-201 scintigraphy Gender
Sensitivity (%) Positive likelihood ratio? Specificity (%) Negative likelihood ratio3 Positive predictive value (%) Negative predictive value (%) Youden’s J Index (+ SD)
Females
Males
Total group
40.7 0.9 55.7 0.9 26.2 70.9 0.19*0.11
43.8 1.9* 76.7* 1.4’ 73.1 48.4 0.48 + 0.07*
43.0 1.2 65.4 1.2 52.1 56.7 0.29 & 0.06
*p < 0.05 compared with females. TPositive likelihood ratio = sensitivity/l - specificity. SNegative likelihood ratio = specificity/l - sensitivity.
Youden’s J index was also significantly lower in females than in males.
DISCUSSION
The Rose Questionnaire is commonly employed to determine the prevalence of angina pectoris in epidemiologic studies. Our study, using patients referred for a clinical exercise test, found that the Rose Questionnaire had a low sensitivity for myocardial ischemia defined by abnormal thallium-201 myocardial scintigraphy in both males and females. The specificity and accuracy of the questionnaire was significantly higher in males than in females. The positive and negative likelihood ratios, although higher in males, were low in both sexes. Further, the prevalence figures for angina pectoris and myocardial ischemia as determined by Rose Questionnaire or by thallium-201 testing were widely different and varied by gender. These results suggest that the Rose Questionnaire may be of limited validity for the detection of myocardial ischemia in a clinical population, especially when utilized with women. Early validation studies by Rose and others [l-7] showed the Rose Questionnaire to have both a high sensitivity and specificity compared with physician diagnoses. More recent data in a clinical population from the Beta-Blocker Heart Attack Trial (BHAT) [14] reported a low sensitivity and high specificity in males compared with physician opinion. This may reflect either a lower prevalence of angina pectoris in these post-myocardial infarction patients or a lack of validity of the Rose Questionnaire in such patients. In keeping with the findings of the BHAT study, the current study, also completed with a clinical population, yielded relatively
low sensitivity and a moderate specificity in comparison with thallium-201 myocardial scintigraphy. The Rose Questionnaire was developed to assess the prevalence of angina pectoris in a general population. However, it has been used in clinical trials to assess treatment effects [14,24,25] and also to predict subsequent cardiovascular morbidity and mortality [3,7, 141. These uses of the Rose Questionnaire assume that angina pectoris reflects myocardial ischemia and/or coronary artery disease. As an editorial by Hlatky et al. [15] points out, this is not always a valid assumption. Although interrelated, the symptoms of angina pectoris are not always accompanied by myocardial ischemia and/or coronary artery stenoses. In order to utilize the Rose Questionnaire in this manner, it must be validated against measures of myocardial ischemia and coronary artery disease. There have been a few studies which have compared the Rose Questionnaire with coronary anatomy or clinical measures of ischemia. A study by Erikssen et al. [26] compared the Rose Questionnaire with coronary angiography in 106 male patients and found it to be highly sensitive and poorly specific. This same study also contrasted the Rose Questionnaire with exercise electrocardiography in 67 male patients who refused angiography, yielding a sensitivity of 57%, a specificity of 47% and a predictive value of 19%. The results of this study may be misleading, however, because of the low prevalence of abnormal electrocardiograms in the study sample and the only moderate sensitivity and specificity of the exercise electrocardiogram itself [27]. Thallium-201 myocardial scintigraphy is widely accepted as the best clinically-available measure of myocardial ischemia [28,29]. A
Rose Questionnaire
study by Bass and colleagues determined the sensitivity and specificity of a supplemented Rose Questionnaire in a clinical population of male and female patients referred for exercise thallium-201 imaging [30]. The authors reported a 26% sensitivity and a 74% specificity for the Rose Questionnaire compared with myocardial ischemia detected by thallium-201 imaging. Although completed in a similar clinical population, the Bass et al. study used a slightly different version of the Rose Questionnaire than employed in the current study. Another difference is in the prevalence of Rose Questionnaire angina and ischemia by thallium-201 in the two studies. The prevalence of Rose Questionnaire angina in the study by Bass and coworkers was 15% in males and 36% in females. This is in contrast to the present study, where 35% of the males and 43% of the females had Rose Questionnaire angina. Additionally, the prevalence of reversible thallium-201 defects in males was higher in the present study (59%) compared with the study by Bass (42%). In females, the prevalence figures for ischemia were similar in the two studies (24 vs 27%). These factors alone or in combination may explain, in part, the different sensitivity and specificity figures obtained by the two studies. The gender differences in the specificity of the Rose Questionnaire compared with thallium-201 imaging are of concern. The lower specificity in females may be due to well-known gender differences in the etiology of chest pain [9, 3&32]. Females, especially those under age of 55, are known to have a higher rate of exertional chest pain due to causes other than coronary artery disease [8, 11, 121. The females in our sample had a mean age of 57 f 11 years with a range of 28-81 years. The Rose Questionnaire has been shown to provide stable estimates of angina prevalence in populations [2]. Yet, the findings of the present study have important implications for the use of the Rose Questionnaire, both in epidemiologic and clinical trial research. The Rose Questionnaire does not differentiate well between patients with and without myocardial ischemia as demonstrated by thallium-201 myocardial scintigraphy. This is particularly true among females. Acknowledgements-We gratefully acknowledge secretarial assistance of Karen Poissant and the assistance of Mark J. Connolly, Lisa Warren Andrea Rousseau-Devol, Debra Messinger and Emaus.
the fine technical Swanson, Susan P.
719 REFERENCES
1. Rose GA. The diagnosis of ischemic heart pain and intermittent claudication in field surveys. Bull Wld Health Org 1962; 27: 645-658. Rose GA. Chest pain questionnaire. Milbank Mem Fund Q 1965; 43: 32-39. Rose GA, McCartney P, Reid DD. Self-administration of a questionnaire -on chest pain and intermittent claudication. Br J Prev See Med 1977: 31: 42-48. Cederlof R, Jonsson E, Lundman T. 6n the validity of mailed questionnaires in diagnosing angina pectoris and bronchitis. Arch Environ Health 1966; 13: 738-742. 5. Heyden S, Bartel AG, Tabest E, Cassel JC, Tyroler HA, Cornoni JC, Hames CG. Angina pectoris and the Rose questionnaire. Arch Intern Med 1971; 128: 961-964. 6. Hagman M, Jonsson D, Wilhelmsen L. Prevalence of angina pectoris and myocardial infarction in a general population sample of Swedish men. Acta Med Seand 1977; 201: 571-577. 7. Lundman T, Liljefors I, Cederlof R, Friberg L. The validity of the questionnaire diagnosis. Arch Environ Health 1971; 22: 597-599. T, 8. Feinleib M, Lambert PM, Zeiner-Henriksen Rogot E, Hunt BM, Ingster-Moore L. The BritishNorwegian Migrant Study-analysis of parameters of mortality differentials associated with angina. Biometrics 1982; 38 (Suppl): 55-71. 9. Laoidus L. Benntsson C. Lindauist 0. Sigurdsson JA: Rafnssbn V.-Prognosh for iomen with‘hifferent symptoms and signs suggesting ischaemic heart disease-a 12-year follow-up. J Chron Dis 1985; 38: 741-748. IO. Reunahen A, Aromaa A, Dyorala K, Punsar S, Maatela J, Knekt P. The Social Insurance Institutions Coronary Artery Disease Study. Acta Med Stand 1983; 67 (Suppl): 2-99. 11. Jensen G. Epidemiology of chest pain & angina pectoris with special reference to treatment needs. Acta Med Stand 1982; 682 (Suppl): 11-99. 12. Wilcosky T, Harris R, Weissfeld L. The prevalence and correlates of Rose questionnaire angina among women and men in the Lipid Research Clinics Program Prevalence Study Population. Am J Epidemiol 1987; 1250: 400-404. 13. Lacroix AZ, Haynes SG, Savage DD, Itavlick RJ. Rose questionnaire angina among United States Black, White and Mexican American women and men. Am J Epidemiol 1989; 129: 669686. 14. Friedman LM, Byington RP. Assessment of angina pectoris after myocardial infarction: comparison of Rose questionnaire with physician judgement in the Beta-Blocker Heart Attack Trial. Am J Epidemiol 1985; 121: 555-562. 15. Hlatky MA, Mark DB, Califf RM, Pryor DB. Angina, myocardial ischemia, and coronary artery disease: gold standards, operational definitions and correlations. J Clin Epidemiol 1989; 42: 381-384. 16. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary artery disease. N Engl J Med 1979; 300: 1350-1358. 17. Bruce RA, Hornstein TR. Exercise stress testing in evaluation of patients with ischemic heart disease. Prog Cardiovasc Dis 1969; 11: 371-381. 18. Okada RD. Boucher CA. Kirshenbaum HK. Kushner FG, Strausi WH, Block’ PC, McKusick Kk, Pohost GM. Improved diagnostic accuracy of thallium-201 stress test using multiple observers and criteria derived from interobserver Analysis of Variance. Am J Cardiol 1980; 46: 619424.
CAROLEWING GARBERet al.
720
19. Parker JA, Campbell CC, Heller GV, Silverman KJ, Markis JE, Royal HD, Paulin S, Kolodny G. Intracoronary thallium-201 assessment of thrombolysis in acute myocardial infarction: Validation of the method for imaging before and after therapy. Invest Radio1 1985; 20: 17-20. 20. Dixon WJ, Ed. BMDP Statistical Software. Berkeley, CA: University of California Press; 1983: 143-182, 437-446. S. Evidence & Reasoning in 21. Wassertheil-Smoller Health. New York: Albert Einstein College of Medicine; 1986. 22. Woolson RF. Statistical Methods for the Analysis of Biomedical Data. New York: John Wiley; 1987: 6&64. 23. Youden WJ. Index for rating diagnostic tests. Cancer 1950; 3: 32-35. 24. Langford HG. Further analysis of the hypertension detection and follow-up_. program. Drugs 1986; 31 (Suppl): 23-28. 25. Lipid Research Clinics. The Lipid Research Clinics Coronary Prevention Trial Results: I. Reduction Incidence of Coronary Heart Disease. JAMA 1984; 251: 351-364. 26. Erikssen J, Forfang K, Storstein 0. Angina pectoris in nresumablv healthv middle-aced men. Eur J Cardiol i977; 6: 285-298. -
27. 28.
29.
30.
31. 32.
33.
RiIkin RD, Hood WB. Bayesian analysis of electrocardiographic exercise stress testing. N Bngl J Med 1977; 297: 681-686. Okada RD, Boucher CA, Strauss HW, Pohost GM. Exercise radionuclide imaging approaches to coronary artery disease. Am J Cardiol 1980; 46: 1188-l 198. Gibson RS, Beller GA. Should exercise electrocardiographic testing be replaced by radioisotope methods. In: Rahimtoola SH, Ed. Controversies in Coronary Artery Disease: Philadelphia, PA: FA Davis Co; 1983: l-31. Bass EB, Follansbee WP, Orchard TJ. Comparison of a supplemented Rose Questionnaire to exercise thallium testing in men and women. J Clin Epidemiol 1989; 42: 385-394. Levine HJ. Difficult problems in the diagnosis of chest pain. Am Heart J 1980; 100: 108-118. Wu SC. Secchi MB. Radice M. Giaunoni G. Sachero A, Oltrana L, Mordsini PL, Foil G. Sex differences in the prevalence of ischemic heart disease and the response to a stress test in a working population. Eur Heart J 1981; 2: 461465. Kannell WB, Feinleib M. Natural history of angina pectoris in the Framingham Study. Am J Cardiol 1972; 29: 154-163.
APPENDIX Modified Rose Questionnaire and Criteria for a Positive Rose Questionnaire Positive Response(s)
Question
1. Have you ever had any pain or discomfort in your chest? 2. Do you get it when you walk uphill or hurry? 3. Do you get it when you walk at an ordinary pace on the level? 4. What do you do if you get it while you are walking? 5. If you stand still, what happens to it? 6. How soon? 7. Will vou show me where it is?
Yes Yes Yes or no
Stop, slow down, or take sublingual nitroglycerin Relieved 10 minutes or less Sternum or left anterior chest, left arm
0895-4356/92 $5.00 + 0.00
Printedin Great
PergamonPressLtd
COMPARISON OF “ROSE QUESTIONNAIRE ANGINA” TO EXERCISE THALLIUM SCINTIGRAPHY: DIFFERENT FINDINGS IN MALES AND FEMALES CAROL EWING GARBER.* RICHARD A. CARLETON and GARY V. HELLER Human Performance Laboratory, Department of Medicine (Division of Cardiology), Memorial Hospital of Rhode Island, Pawtucket, Brown University School of Medicine, Providence, RI 02912. U.S.A. (Received in revised form I I February 1992)
Abstract-Validation of the London School of Hygiene (Rose) Questionnaire with objective measures of myocardial ischemia is incomplete. Therefore, we compared the Rose Questionnaire with exercise thallium-201 myocardial scintigraphy in 147 male and 97 female patients with chest pain referred for clinical exercise testing. Of those with “Rose Questionnaire angina”, 26% of the females and 73% of the males had positive thallium-201 scans. Negative results on both the Rose Questionnaire and thallium-201 scintigraphy were observed in 71% of the females and 47% of the males. The sensitivity of the Rose Questionnaire was similar in females (41%) and males (44%). The specificity was 77% in males, while in females it was significantly lower at 56%. The specificity values reflect the higher (p < 0.05) prevalence of “false positive” Rose Questionnaire results in females (75%) compared with males (27%). In addition, males had a greater (p c 0.05) number of “false negative” results (53%) than females (29%). The accuracy of the Rose Questionnaire for myocardial ischemia was 0.19 in females, 0.48 in males, and 0.29 overall when including both males and females. Our results indicate a generally poor relationship between Rose Questionnaire angina and thallium-201 scintigraphy, an objective measure of myocardial ischemia in patients with chest pain referred to clinical exercise testing. Further, there are gender-specific differences in this relationship between the questionnaire and exercise thallium-201 imaging. Angina pectoris Epidemiology Thallium-201 myocardial naires
Gender scintigraphy
INTRODUCTION
The determination of the prevalence of angina pectoris in populations is important, especially for epidemiologic and large scale clinical trial research. The clinical diagnosis of angina pectoris by individual assessment is often impractical for population-based studies due to its labor-intensive nature and cost. Thus, an *All correspondence should be addressed to: Carol Ewing Garber, Ph.D., Division of Cardiology, Memorial Hospital of Rhode Island, 111 Brewster Street, Pawtucket, RI 02860, U.S.A. [Tel. 401-729-22611.
Myocardial
ischemia
Question-
easily-administered questionnaire which is reproducible and valid in comparison to wellaccepted clinical standards is desirable. Dr Geoffrey Rose developed the London School of Hygiene (Rose) Questionnaire for the standardized determination of angina pectoris [l-3]. Using male patients with known coronary heart disease, Rose identified several distinctive features defining angina pectoris which were highly specific and moderately sensitive in comparison with physician diagnoses [I]. Although on an individual basis the questionnaire was not highly reproducible, the
716
CAROLEWING GARBERef al.
prevalence of angina pectoris estimated by Rose Questionnaire within a population was very stable [2]. The Rose Questionnaire has been validated against physician diagnosis in both general and patient populations [l-7]. Based upon these studies, it has been generally well accepted as a valid means for measuring angina pectoris in epidemiologic study, with a sensitivity ranging from 25-83% and a specificity of 48-98%. The utility of the Rose Questionnaire for women is unknown. Although the questionnaire has been used extensively in Europe to evaluate women [8-l 11, validation is lacking. Most validation studies have used male subjects, and have either excluded women or included such a small number of female subjects to be statistically uninterpretable. Several studies have reported a higher prevalence of angina pectoris among women younger than 55 years of age, but this “Rose Questionnaire angina” was not predictive of coronary artery disease mortality [8, 12, 131. In older women, however, a predictive relationship was found. Wilcosky and associates found that the relationship between Rose Questionnaire angina and a number of coronary risk factors was gender-specific [12]. From these data and the lack of other studies, it is unclear whether the Rose Questionnaire can adequately assess the prevalence of ischemic coronary artery disease in females. The use of the Rose Questionnaire has extended beyond the assessment of the prevalence of angina pectoris within a general population. It has been used in clinical trials evaluating pharmacologic or other therapies [14] and also has been used to predict subsequent cardiovascular morbidity and mortality [3,7, 141. When the Rose Questionnaire is applied in these ways, it is assumed that angina pectoris reflects underlying myocardial ischemia and/or coronary artery disease. Although angina pectoris, myocardial ischemia and coronary artery stenoses they represent 3 distinct are associated, phenomena which are not necessarily directly related [15]. Thus, it is important to validate Rose Questionnaire angina with measures of myocardial ischemia and coronary anatomy before clinical applications of the Rose Questionnaire can be recommended. The purpose of the present study was to determine the validity of the Rose Questionnaire as a correlate of myocardial ischemia in males and females. We compared the results of the Rose Questionnaire with exercise thallium-
201 myocardial scintigraphy in patients referred for clinical exercise testing. METHODS
Subjects
Two hundred forty-four consecutive patients (147 males and 97 females; mean age 57 + 11 years) with self-reported chest pain or discomfort referred for exercise thallium-201 myocardial scintigraphy at two teaching hospitals completed a graded exercise test (reaching at least 85% of their predicted maximum heart rate) with thallium-201 myocardial scintigraphy and the Rose Questionnaire. Overall, the subjects had a pre-test probability of coronary artery disease of 58 (+ 1l)%, based upon the Diamond & Forrester classification [16]. Test procedures
Each patient filled out the version of the Rose Questionnaire shown in the Appendix which was modified and validated by Heyden et al. [5]. A symptom-limited graded exercise test using a Bruce protocol was then completed [17]. One minute prior to the end of exercise, an intravenous dose (8 1 MBq; 2.2 mCi) of thallium-201 was given. Within 10 minutes following exercise, planar myocardial imaging was obtained with a general all-purpose parallel hole collimator in three standard views: anterior, 45” left anterior oblique and the 70” left anterior oblique. Delayed images were obtained 4 hours later. Data interpretation
All thallium-201 scintigraphic images were interpreted by two experienced cardiologists without knowledge of the study protocol or Rose Questionnaire results. The images were read by consensus using standard criteria [18]. The left ventricular images were divided into 15 segments; each segment was scored on a scale of O-3 (0 = normal, 1 = slight photon emission reduction, 2 = moderate photon emission reduction, 3 = background or absent activity). This procedure has been validated and reported previously [19]. A thallium-201 scan was considered positive for myocardial ischemia when one or more of the images showed a photondeficient area with normalization on delayed images. The thallium-201 scan was negative if there were no photon-deficient areas. Angina pectoris was considered to be present when a subject provided the affirmative responses, as shown in the Appendix, to each of
Rose Questionnaire
the first six questions on the Rose Questionnaire. A negative response to one or more of these questions was classified as angina pectoris absent. The data were computer coded and analyzed using the BMDP statistical package, programs 4F and 3s [20]. Two by two contingency tables were generated and the independence of classification was tested using the Yates corrected Chi Square. Sensitivity, specificity, and the positive and negative predictive values of the Rose Questionnaire in comparison to thallium-201 imaging were determined from the contingency tables using standard equations [21]. The positive and negative likelihood ratios were derived from the sensitivity and specificity figures [22], and indicate the probability that an abnormal Rose Questionnaire result is true for ischemia or the probability of a negative Rose Questionnaire being negative for ischemia, respectively. Youden’s J index was calculated for males, females and the total group [23]. This index is an objective method for comparing the accuracy of two tests and ranges from 0 to 1. Zero indicates that the test yields the same proportion of positive tests in those with and without disease, while one is obtained when there are no false positive or false negative results. RESULTS
The prevalence of Rose Questionnaire angina in our patients was higher (43%) in women than in men (35%; p < 0.05). In contrast, 27% of the females and 59% of the males (p < 0.05) had ischemia on the thallium-201 scan.
717
The classification of patients by Rose Questionnaire and thallium-201 scintigraphy is presented in Table 1. Fifty-two percent (49/94) of the subjects with positive Rose Questionnaire results had evidence of myocardial ischemia by thallium-201 imaging. However, there were differences according to gender. Of males with Rose Questionnaire angina, 73% had a positive thallium-201 scan. In contrast, the agreement between positive results in females was significantly lower (26%; p < 0.05). Of subjects with a negative Rose Questionnaire, 57% had a negative thallium-201 scan. There were also gender differences in the negative results. Females had a significantly (p c 0.05) higher percentage of concurrently negative results (70%) when compared with the males (48%). Accepting the thallium-201 result as a true indicator of ischemia, the overall rates of false positive and false negative Rose Questionnaire results were 48 and 43%, respectively. Females had a significantly higher 0, < 0.05) rate of false positive results (74%) compared with males (27%). On the other hand, males had significantly more false negative results (53%) than females (29%). The sensitivity, specificity and positive predictive value of the Rose Questionnaire compared with myocardial imaging are presented in Table 2. The sensitivity of the Rose Questionnaire was similar for males and females, while the questionnaire was more specific in males than in females. Reflecting these values, both the positive and negative likelihood ratios were substantially lower in females than in males.
Table 1. Comparison of the Rose Questionnaire with exercise thallium-201 scintig raphy in 244 patients with chest pain Thallium-201 imaging Gender
Rose Questionnaire
Female
Positive
Positive
Negative
Total
(7.k) Negative
Male
Total Positive
(2;;) 27
(7:;) 70
(12%) 97
(7::)
(2::)
(Rz%)
(5::) 87
(4;:) 60
(lZ%) 147
(&)
(4:;)
(4;;) 114
,:gg
(KS?%) 150 (100%) 244
Negative Total Group
Total Positive Negative Total
Values are frequency with percent of row total in parentheses below. *p < 0.05 compared with females with same Rose Questionnaire result.
CAROLEWING GARBER et al.
718
Table 2. Sensitivity, specificity and predictive value of the Rose Questionnaire compared with thallium-201 scintigraphy Gender
Sensitivity (%) Positive likelihood ratio? Specificity (%) Negative likelihood ratio3 Positive predictive value (%) Negative predictive value (%) Youden’s J Index (+ SD)
Females
Males
Total group
40.7 0.9 55.7 0.9 26.2 70.9 0.19*0.11
43.8 1.9* 76.7* 1.4’ 73.1 48.4 0.48 + 0.07*
43.0 1.2 65.4 1.2 52.1 56.7 0.29 & 0.06
*p < 0.05 compared with females. TPositive likelihood ratio = sensitivity/l - specificity. SNegative likelihood ratio = specificity/l - sensitivity.
Youden’s J index was also significantly lower in females than in males.
DISCUSSION
The Rose Questionnaire is commonly employed to determine the prevalence of angina pectoris in epidemiologic studies. Our study, using patients referred for a clinical exercise test, found that the Rose Questionnaire had a low sensitivity for myocardial ischemia defined by abnormal thallium-201 myocardial scintigraphy in both males and females. The specificity and accuracy of the questionnaire was significantly higher in males than in females. The positive and negative likelihood ratios, although higher in males, were low in both sexes. Further, the prevalence figures for angina pectoris and myocardial ischemia as determined by Rose Questionnaire or by thallium-201 testing were widely different and varied by gender. These results suggest that the Rose Questionnaire may be of limited validity for the detection of myocardial ischemia in a clinical population, especially when utilized with women. Early validation studies by Rose and others [l-7] showed the Rose Questionnaire to have both a high sensitivity and specificity compared with physician diagnoses. More recent data in a clinical population from the Beta-Blocker Heart Attack Trial (BHAT) [14] reported a low sensitivity and high specificity in males compared with physician opinion. This may reflect either a lower prevalence of angina pectoris in these post-myocardial infarction patients or a lack of validity of the Rose Questionnaire in such patients. In keeping with the findings of the BHAT study, the current study, also completed with a clinical population, yielded relatively
low sensitivity and a moderate specificity in comparison with thallium-201 myocardial scintigraphy. The Rose Questionnaire was developed to assess the prevalence of angina pectoris in a general population. However, it has been used in clinical trials to assess treatment effects [14,24,25] and also to predict subsequent cardiovascular morbidity and mortality [3,7, 141. These uses of the Rose Questionnaire assume that angina pectoris reflects myocardial ischemia and/or coronary artery disease. As an editorial by Hlatky et al. [15] points out, this is not always a valid assumption. Although interrelated, the symptoms of angina pectoris are not always accompanied by myocardial ischemia and/or coronary artery stenoses. In order to utilize the Rose Questionnaire in this manner, it must be validated against measures of myocardial ischemia and coronary artery disease. There have been a few studies which have compared the Rose Questionnaire with coronary anatomy or clinical measures of ischemia. A study by Erikssen et al. [26] compared the Rose Questionnaire with coronary angiography in 106 male patients and found it to be highly sensitive and poorly specific. This same study also contrasted the Rose Questionnaire with exercise electrocardiography in 67 male patients who refused angiography, yielding a sensitivity of 57%, a specificity of 47% and a predictive value of 19%. The results of this study may be misleading, however, because of the low prevalence of abnormal electrocardiograms in the study sample and the only moderate sensitivity and specificity of the exercise electrocardiogram itself [27]. Thallium-201 myocardial scintigraphy is widely accepted as the best clinically-available measure of myocardial ischemia [28,29]. A
Rose Questionnaire
study by Bass and colleagues determined the sensitivity and specificity of a supplemented Rose Questionnaire in a clinical population of male and female patients referred for exercise thallium-201 imaging [30]. The authors reported a 26% sensitivity and a 74% specificity for the Rose Questionnaire compared with myocardial ischemia detected by thallium-201 imaging. Although completed in a similar clinical population, the Bass et al. study used a slightly different version of the Rose Questionnaire than employed in the current study. Another difference is in the prevalence of Rose Questionnaire angina and ischemia by thallium-201 in the two studies. The prevalence of Rose Questionnaire angina in the study by Bass and coworkers was 15% in males and 36% in females. This is in contrast to the present study, where 35% of the males and 43% of the females had Rose Questionnaire angina. Additionally, the prevalence of reversible thallium-201 defects in males was higher in the present study (59%) compared with the study by Bass (42%). In females, the prevalence figures for ischemia were similar in the two studies (24 vs 27%). These factors alone or in combination may explain, in part, the different sensitivity and specificity figures obtained by the two studies. The gender differences in the specificity of the Rose Questionnaire compared with thallium-201 imaging are of concern. The lower specificity in females may be due to well-known gender differences in the etiology of chest pain [9, 3&32]. Females, especially those under age of 55, are known to have a higher rate of exertional chest pain due to causes other than coronary artery disease [8, 11, 121. The females in our sample had a mean age of 57 f 11 years with a range of 28-81 years. The Rose Questionnaire has been shown to provide stable estimates of angina prevalence in populations [2]. Yet, the findings of the present study have important implications for the use of the Rose Questionnaire, both in epidemiologic and clinical trial research. The Rose Questionnaire does not differentiate well between patients with and without myocardial ischemia as demonstrated by thallium-201 myocardial scintigraphy. This is particularly true among females. Acknowledgements-We gratefully acknowledge secretarial assistance of Karen Poissant and the assistance of Mark J. Connolly, Lisa Warren Andrea Rousseau-Devol, Debra Messinger and Emaus.
the fine technical Swanson, Susan P.
719 REFERENCES
1. Rose GA. The diagnosis of ischemic heart pain and intermittent claudication in field surveys. Bull Wld Health Org 1962; 27: 645-658. Rose GA. Chest pain questionnaire. Milbank Mem Fund Q 1965; 43: 32-39. Rose GA, McCartney P, Reid DD. Self-administration of a questionnaire -on chest pain and intermittent claudication. Br J Prev See Med 1977: 31: 42-48. Cederlof R, Jonsson E, Lundman T. 6n the validity of mailed questionnaires in diagnosing angina pectoris and bronchitis. Arch Environ Health 1966; 13: 738-742. 5. Heyden S, Bartel AG, Tabest E, Cassel JC, Tyroler HA, Cornoni JC, Hames CG. Angina pectoris and the Rose questionnaire. Arch Intern Med 1971; 128: 961-964. 6. Hagman M, Jonsson D, Wilhelmsen L. Prevalence of angina pectoris and myocardial infarction in a general population sample of Swedish men. Acta Med Seand 1977; 201: 571-577. 7. Lundman T, Liljefors I, Cederlof R, Friberg L. The validity of the questionnaire diagnosis. Arch Environ Health 1971; 22: 597-599. T, 8. Feinleib M, Lambert PM, Zeiner-Henriksen Rogot E, Hunt BM, Ingster-Moore L. The BritishNorwegian Migrant Study-analysis of parameters of mortality differentials associated with angina. Biometrics 1982; 38 (Suppl): 55-71. 9. Laoidus L. Benntsson C. Lindauist 0. Sigurdsson JA: Rafnssbn V.-Prognosh for iomen with‘hifferent symptoms and signs suggesting ischaemic heart disease-a 12-year follow-up. J Chron Dis 1985; 38: 741-748. IO. Reunahen A, Aromaa A, Dyorala K, Punsar S, Maatela J, Knekt P. The Social Insurance Institutions Coronary Artery Disease Study. Acta Med Stand 1983; 67 (Suppl): 2-99. 11. Jensen G. Epidemiology of chest pain & angina pectoris with special reference to treatment needs. Acta Med Stand 1982; 682 (Suppl): 11-99. 12. Wilcosky T, Harris R, Weissfeld L. The prevalence and correlates of Rose questionnaire angina among women and men in the Lipid Research Clinics Program Prevalence Study Population. Am J Epidemiol 1987; 1250: 400-404. 13. Lacroix AZ, Haynes SG, Savage DD, Itavlick RJ. Rose questionnaire angina among United States Black, White and Mexican American women and men. Am J Epidemiol 1989; 129: 669686. 14. Friedman LM, Byington RP. Assessment of angina pectoris after myocardial infarction: comparison of Rose questionnaire with physician judgement in the Beta-Blocker Heart Attack Trial. Am J Epidemiol 1985; 121: 555-562. 15. Hlatky MA, Mark DB, Califf RM, Pryor DB. Angina, myocardial ischemia, and coronary artery disease: gold standards, operational definitions and correlations. J Clin Epidemiol 1989; 42: 381-384. 16. Diamond GA, Forrester JS. Analysis of probability as an aid in the clinical diagnosis of coronary artery disease. N Engl J Med 1979; 300: 1350-1358. 17. Bruce RA, Hornstein TR. Exercise stress testing in evaluation of patients with ischemic heart disease. Prog Cardiovasc Dis 1969; 11: 371-381. 18. Okada RD. Boucher CA. Kirshenbaum HK. Kushner FG, Strausi WH, Block’ PC, McKusick Kk, Pohost GM. Improved diagnostic accuracy of thallium-201 stress test using multiple observers and criteria derived from interobserver Analysis of Variance. Am J Cardiol 1980; 46: 619424.
CAROLEWING GARBERet al.
720
19. Parker JA, Campbell CC, Heller GV, Silverman KJ, Markis JE, Royal HD, Paulin S, Kolodny G. Intracoronary thallium-201 assessment of thrombolysis in acute myocardial infarction: Validation of the method for imaging before and after therapy. Invest Radio1 1985; 20: 17-20. 20. Dixon WJ, Ed. BMDP Statistical Software. Berkeley, CA: University of California Press; 1983: 143-182, 437-446. S. Evidence & Reasoning in 21. Wassertheil-Smoller Health. New York: Albert Einstein College of Medicine; 1986. 22. Woolson RF. Statistical Methods for the Analysis of Biomedical Data. New York: John Wiley; 1987: 6&64. 23. Youden WJ. Index for rating diagnostic tests. Cancer 1950; 3: 32-35. 24. Langford HG. Further analysis of the hypertension detection and follow-up_. program. Drugs 1986; 31 (Suppl): 23-28. 25. Lipid Research Clinics. The Lipid Research Clinics Coronary Prevention Trial Results: I. Reduction Incidence of Coronary Heart Disease. JAMA 1984; 251: 351-364. 26. Erikssen J, Forfang K, Storstein 0. Angina pectoris in nresumablv healthv middle-aced men. Eur J Cardiol i977; 6: 285-298. -
27. 28.
29.
30.
31. 32.
33.
RiIkin RD, Hood WB. Bayesian analysis of electrocardiographic exercise stress testing. N Bngl J Med 1977; 297: 681-686. Okada RD, Boucher CA, Strauss HW, Pohost GM. Exercise radionuclide imaging approaches to coronary artery disease. Am J Cardiol 1980; 46: 1188-l 198. Gibson RS, Beller GA. Should exercise electrocardiographic testing be replaced by radioisotope methods. In: Rahimtoola SH, Ed. Controversies in Coronary Artery Disease: Philadelphia, PA: FA Davis Co; 1983: l-31. Bass EB, Follansbee WP, Orchard TJ. Comparison of a supplemented Rose Questionnaire to exercise thallium testing in men and women. J Clin Epidemiol 1989; 42: 385-394. Levine HJ. Difficult problems in the diagnosis of chest pain. Am Heart J 1980; 100: 108-118. Wu SC. Secchi MB. Radice M. Giaunoni G. Sachero A, Oltrana L, Mordsini PL, Foil G. Sex differences in the prevalence of ischemic heart disease and the response to a stress test in a working population. Eur Heart J 1981; 2: 461465. Kannell WB, Feinleib M. Natural history of angina pectoris in the Framingham Study. Am J Cardiol 1972; 29: 154-163.
APPENDIX Modified Rose Questionnaire and Criteria for a Positive Rose Questionnaire Positive Response(s)
Question
1. Have you ever had any pain or discomfort in your chest? 2. Do you get it when you walk uphill or hurry? 3. Do you get it when you walk at an ordinary pace on the level? 4. What do you do if you get it while you are walking? 5. If you stand still, what happens to it? 6. How soon? 7. Will vou show me where it is?
Yes Yes Yes or no
Stop, slow down, or take sublingual nitroglycerin Relieved 10 minutes or less Sternum or left anterior chest, left arm
