Journal of Asthma, 28(2) 129-139 (1991)
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
Evaluation of a New Asthma Questionnaire Michael J. Abramson, M.B.B.S., FRACP, * Michael J. Hensley, M.B.B.S., Ph.D, FRACP,* Nicholas A. Saunders, M.D.B.S., FRACP, FRCP(Can),t and John H. Wlodarczyk, B.Ec., Dip. Med Stats* *Centre for Clinical Epidemiology & Biostatistics and 'Discipline of Medicine University of Newcastle Newcastle, Australia
ABSTRACT The new International Union Against Tuberculosis (IUAT) bronchial symptoms questionnaire was completed by 827 subjects participating in a prospective study of respiratory symptoms and lung function in aluminum smelter workers. A modified Medical Research Council (MRC) questionnaire was also administered. Bronchial reactivity (BR) was measured in 809 subjects by methacholine challenge using a rapid method. Factor analysis demonstrated sensible clustering of responses to items unqiue to the new questionnaire such as nocturnal, spontaneous, and postexertional dyspnea, dust-induced dyspnea and tightness, and breathing difficulty. Responses to IUAT questions concerning past asthma, wheeze, chest tightness, morning cough and sputum, and asthma medication agreed well with corresponding items from the Address reprint requests to: Dr. M.J. Abramson, Department of Social & Preventative Medicine, Monash Medical School, Alfi-ed Hospital, Prahran, Victoria 3181, Australia.
129 Copyright 0 1991 by Marcel Dekker, Inc.
Abramson et al.
130 MRC questionnaire. Questions concerning asthma, medication, dust-induced, nocturnal, and spontaneous dyspnea, chest tightness, wheeze, nocturnal cough, postexertional dyspnea and breathing difficulty also had high validity against the criterion of concurrently measured bronchial reactivity. It is concluded that the IUAT questionnaire is a valid asthma questionnaire.
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
INTRODUCTION The definition of asthma, especially for epidemiological studies remains controversial. There is agreement with the clinical criteria of episodic cough, wheeze, and dyspnea together with variable airflow obstruction, which may require multiple assessments. Since prospective documentation of episodes of clinically important asthma is both contentious and impractical in epidemiological studies, researchers have used questionnaires and more recently measurement of bronchial reactivity (BR)to define asthma. As an objective measure of asthma, BR is relatively sensitive, but has far from good specificity. The first standardized respiratory questionnaire was developed by the British Medical Research Council in the 1950s (1) and has established reproducibility (2). However, most of the symptom items are characteristic of chronic bronchitis and several important questions related to asthma are not included. The American Thoracic Society (ATS) Division of Lung Diseases subsequently released a self-completion questionnaire which has also been shown to be reliable and valid (3). Recently the Respiratory Diseases Committee of the International Union Against Tuberculosis (IUAT)has developed a new bronchial symptoms questionnaire. Issues related to the reproducibility and development of this questionnaire have been reported elsewhere (4,5), but its validity has not been established. For a clinical disorder such as asthma where there is no objective gold standard, the validity of a questionnaire can be assessed by referring it to a number of measures (6):Face validity: does the questionnaire appear sensible? Content validity: are the questions representative and appropriate for the intended population? Concurrent criterion validity:
does the questionnaire produce results consistent with a clinical state concurrently diagnosed by accepted methods? Construct validity: what theoretical constructs does the questionnaire actually measure? The face and content validity of the IUAT questionnaire were established during its development. While the new instrument was not intended to be comprehensive, it does include a number of relevant asthmatic symptoms. Thus meaningful structure is known for the questions and needs to be established for the answers. This is particularly so for IUAT items not covered by a modified Medical Research Council (MMRC) questionnaire such as nocturnal, spontaneous, and postexertional dyspnea, nocturnal cough and dyspnea or chest tightness on exposure to animals, dust, and feathers. Relationships would be expected between responses to these items and established bronchial symptom questions. The correlation between a test and factor common to a group of tests is sometimes called “factorial validity” (7). Construct validity of a new questionnaire is usually established by the correlation between scores from the new instrument and from a questionnaire which is accepted to measure the relevant construct (7). In this case, it is assessed by comparing the prevalence of symptoms elicited by substantially similar items from the MMRC and IUAT questionnaires. Subjects reporting wheeze, chest tightness, morning cough and sputum, past history of asthma, and treatment for asthma on the IUAT questionnaire would be expected to report the same symptoms on the MMRC. Concurrently measured bronchial reactivity has been chosen as the objective criterion against which to judge asthmatic symptom questions. Thus subjects with measurable
Evaluation of a New Asthma Questionnaire
131
bronchial reactivity would be expected to report more symptoms on a valid asthma questionnaire than those with nonreactive airways. In the presence of such symptoms, measured reactivity would be expected to be greater than in their absence.
recorded as positive responses. All symptom items from the IUAT questionnaire were included in the analysis. The IUAT smoking questions were not administered, as this information had already been obtained from the MMRC questionnaire. Matched items were drawn for comparison from the MMRC questionnaire where the content was relevant to asthma. These included symptom questions concerning dyspnea, cough, morning sputum, chest tightness, wheeze, past history of asthma, and use of salbutamol (see Appendix 11).MMRC items excluded from analysis were those concerning cough and sputum for 3 months of a year, past history other than asthma, and other medication. Because not all subjects answered every question on both questionnaires, the total number included in each analysis was less than 827. Factor analysis has been widely used with psychological tests to reduce a large number of observed variables to a few underlying factors. Factor analysis of all IUAT symptom items and the selected MRC items was performed to reduce the data to manageable size and to explore the structure of responses. Responses to IUAT and MMRC were analyzed together to facilitate the interpretation of alternative solutions and to identify items common to both questionnaires. A total of 789 cases had complete responses to all items. Factors were extracted by principal components analysis and then submitted to Varimax rotation (12) using BMDP Program 4M (13). Alternative 4,5,6, and 8 factor solutions were examined as were Equimax and Quartimax rotations of the 6-factor solution. These rotations were performed to simplify interpretation. For item by item comparisons, the data were examined in 2 x 2 contingency tables and chi-square with Yates correction for continuity used as a measure of association (14). Cohen’s Kappa ( x ) was employed as a measure of agreement (15). The null hypothesis of no agreement beyond that expected by chance was tested by comparing x divided by its standard error with the standard normal distribution. McNemar’s test for paired data was used as a measure of
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
METHODS Our subjects were employees of a local aluminum smelter participating in a prospective study of respiratory function. The order of testing was the MMRC questionnaire, methacholine challenge, and finally the IUAT questionnaire. Of the 843 workers on site, 828 first answered a MMRC respiratory symptom questionnaire administered by one of the authors (MJA) or a trained nursing sister. Item response rates were between 99.3%and 100%. All but one subject performed baseline flow volume curves to ATS standards (8) on computerized equipment interfaced with a water sealed spirometer (Gould 2400). After giving informed consent, 809 (97.7%) workers subsequently underwent methacholine challenge up to a dose of 6.14 pmol by a rapid method employing hand-held glass nebulizers (9).Twelve subjects were excluded because of poor baseline lung function (FEV1 c 65% predicted for age and height) and seven declined to participate. The dose of methacholine producing a 20% fall in FEVl (PD20) was interpolated on a logarithmic scale between the last two measurements of FEVl when a drop of this magnitude occurred (10). Bronchial hyperresponsiveness was considered present in the 65 subjects who had a PD20 less than 6.14 pmol, which was the maximum total dose of methacholine administered. Dose-response slope (DRS) was calculated as percent change (%A) in FEVl from baselinehtal dose (11).Prior to expression on a logarithmic scale, 1.5% pmol was added to eliminate negative values. Symptom questions from the new IUAT questionnaire (see Appendix I) were completed by 827 subjects. The item response rates were between 97.3%and 100%. Subjects who responded ‘regular’ or ‘always’ to the question concerning breathing difficulty were
132
Abramson et al. Table 1. Salient Factor Loadings for Items from MMRC and IUAT Questionnaires (Loadings Less than 0.3 are Indicated by -1. FACTOR
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
ITEM Asthma Asthma Current Asthma Salbutamol Medication Morning cough Morning cough Morning phlegm Morning phlegm Wheeze Wheeze Nocturnal cough Tightness Tightness Exertional dyspnea Exertional dyspnea Breathing difficulty Dust tightness Dust dyspnea Spontaneous dyspnea Nocturnal dyspnea
QUESTIONNAIRE
1
2
3
4
5
IUAT MMRC IUAT MMRC IUAT IUAT MRC IUAT MMRC IUAT MMRC IUAT IUAT MMRC MMRC IUAT IUAT IUAT IUAT IUAT IUAT
0.867 0.845 0.784 0.721 0.714
-
-
-
-
0.857 0.830 0.822 0.772
-
Eigenvalue Interpretation
-
-
-
-
-
-
0.81 8 0.784 0.589 0.520 0.31 7
-
-
-
-
-
0.336 0.342
-
-
3.733
2.867
2.323
Asthma
Bronchitis
Wheeze
-
-
marginal homogeneity to test for differences in prevalence (14).
RESULTS The Structure of Responses
The correlation matrix between IUAT and selected MMRC items contained many coefficients > 0.3 suggesting it could be meaningfully factored. The number of cases was adequate as it was greater than 10 times the number of variables. Because this was a n exploratory analysis, no systematic attempt was made to identify outlying cases. As the responses were categorical no formal testing of multivariate normality was appropriate. According to Kaiser’s rule of thumb, only a factor with a n eigenvalue greater than one explained more of the variance than a standardized observed variable (12).Thus the five factors meeting this criterion which explained 62% of the total variance were retained for further analysis. The sixth factor contained
-
0.328 0.534 0.756 0.642 - 0.529
-
0.462
2.1 71 SOB
-
-
-
-
-
0.867 0.825
0.378 1.836
Dust
only two variables: daytime cough and sputum, which were highly correlated, but unrelated to other items. Accordingly these variables were deleted on pragmatic grounds and the 5-factor solution with Varimax rotation selected for conceptual simplicity and ease of description. Factor loadings following Varimax rotation are listed in Table 1. Only loadings greater than 0.3 were interpreted as ‘salient’because a lesser loading accounted for under 9% of the variance overlap between the variable and the factor (12).As is customary, the interpretation of these factors was based on the loadings of questionnaire items. For example, salient loadings on the first factor were from items concerning history of asthma or treatment of asthma. Thus this factor was labelled as a n ‘asthma’ factor. More detailed interpretation is discussed below. Construct Validity
This was assessed by comparing responses to substantially similar items from each
Evaluation of a New Asthma Questionnaire
133
Table 2. Agreement Between IUAT and Corresponding MMRC Items
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
PREVALENCE (%) IUAT
MMRC
MMRC
SE ( x )
P
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
Evaluation of a New Asthma Questionnaire Michael J. Abramson, M.B.B.S., FRACP, * Michael J. Hensley, M.B.B.S., Ph.D, FRACP,* Nicholas A. Saunders, M.D.B.S., FRACP, FRCP(Can),t and John H. Wlodarczyk, B.Ec., Dip. Med Stats* *Centre for Clinical Epidemiology & Biostatistics and 'Discipline of Medicine University of Newcastle Newcastle, Australia
ABSTRACT The new International Union Against Tuberculosis (IUAT) bronchial symptoms questionnaire was completed by 827 subjects participating in a prospective study of respiratory symptoms and lung function in aluminum smelter workers. A modified Medical Research Council (MRC) questionnaire was also administered. Bronchial reactivity (BR) was measured in 809 subjects by methacholine challenge using a rapid method. Factor analysis demonstrated sensible clustering of responses to items unqiue to the new questionnaire such as nocturnal, spontaneous, and postexertional dyspnea, dust-induced dyspnea and tightness, and breathing difficulty. Responses to IUAT questions concerning past asthma, wheeze, chest tightness, morning cough and sputum, and asthma medication agreed well with corresponding items from the Address reprint requests to: Dr. M.J. Abramson, Department of Social & Preventative Medicine, Monash Medical School, Alfi-ed Hospital, Prahran, Victoria 3181, Australia.
129 Copyright 0 1991 by Marcel Dekker, Inc.
Abramson et al.
130 MRC questionnaire. Questions concerning asthma, medication, dust-induced, nocturnal, and spontaneous dyspnea, chest tightness, wheeze, nocturnal cough, postexertional dyspnea and breathing difficulty also had high validity against the criterion of concurrently measured bronchial reactivity. It is concluded that the IUAT questionnaire is a valid asthma questionnaire.
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
INTRODUCTION The definition of asthma, especially for epidemiological studies remains controversial. There is agreement with the clinical criteria of episodic cough, wheeze, and dyspnea together with variable airflow obstruction, which may require multiple assessments. Since prospective documentation of episodes of clinically important asthma is both contentious and impractical in epidemiological studies, researchers have used questionnaires and more recently measurement of bronchial reactivity (BR)to define asthma. As an objective measure of asthma, BR is relatively sensitive, but has far from good specificity. The first standardized respiratory questionnaire was developed by the British Medical Research Council in the 1950s (1) and has established reproducibility (2). However, most of the symptom items are characteristic of chronic bronchitis and several important questions related to asthma are not included. The American Thoracic Society (ATS) Division of Lung Diseases subsequently released a self-completion questionnaire which has also been shown to be reliable and valid (3). Recently the Respiratory Diseases Committee of the International Union Against Tuberculosis (IUAT)has developed a new bronchial symptoms questionnaire. Issues related to the reproducibility and development of this questionnaire have been reported elsewhere (4,5), but its validity has not been established. For a clinical disorder such as asthma where there is no objective gold standard, the validity of a questionnaire can be assessed by referring it to a number of measures (6):Face validity: does the questionnaire appear sensible? Content validity: are the questions representative and appropriate for the intended population? Concurrent criterion validity:
does the questionnaire produce results consistent with a clinical state concurrently diagnosed by accepted methods? Construct validity: what theoretical constructs does the questionnaire actually measure? The face and content validity of the IUAT questionnaire were established during its development. While the new instrument was not intended to be comprehensive, it does include a number of relevant asthmatic symptoms. Thus meaningful structure is known for the questions and needs to be established for the answers. This is particularly so for IUAT items not covered by a modified Medical Research Council (MMRC) questionnaire such as nocturnal, spontaneous, and postexertional dyspnea, nocturnal cough and dyspnea or chest tightness on exposure to animals, dust, and feathers. Relationships would be expected between responses to these items and established bronchial symptom questions. The correlation between a test and factor common to a group of tests is sometimes called “factorial validity” (7). Construct validity of a new questionnaire is usually established by the correlation between scores from the new instrument and from a questionnaire which is accepted to measure the relevant construct (7). In this case, it is assessed by comparing the prevalence of symptoms elicited by substantially similar items from the MMRC and IUAT questionnaires. Subjects reporting wheeze, chest tightness, morning cough and sputum, past history of asthma, and treatment for asthma on the IUAT questionnaire would be expected to report the same symptoms on the MMRC. Concurrently measured bronchial reactivity has been chosen as the objective criterion against which to judge asthmatic symptom questions. Thus subjects with measurable
Evaluation of a New Asthma Questionnaire
131
bronchial reactivity would be expected to report more symptoms on a valid asthma questionnaire than those with nonreactive airways. In the presence of such symptoms, measured reactivity would be expected to be greater than in their absence.
recorded as positive responses. All symptom items from the IUAT questionnaire were included in the analysis. The IUAT smoking questions were not administered, as this information had already been obtained from the MMRC questionnaire. Matched items were drawn for comparison from the MMRC questionnaire where the content was relevant to asthma. These included symptom questions concerning dyspnea, cough, morning sputum, chest tightness, wheeze, past history of asthma, and use of salbutamol (see Appendix 11).MMRC items excluded from analysis were those concerning cough and sputum for 3 months of a year, past history other than asthma, and other medication. Because not all subjects answered every question on both questionnaires, the total number included in each analysis was less than 827. Factor analysis has been widely used with psychological tests to reduce a large number of observed variables to a few underlying factors. Factor analysis of all IUAT symptom items and the selected MRC items was performed to reduce the data to manageable size and to explore the structure of responses. Responses to IUAT and MMRC were analyzed together to facilitate the interpretation of alternative solutions and to identify items common to both questionnaires. A total of 789 cases had complete responses to all items. Factors were extracted by principal components analysis and then submitted to Varimax rotation (12) using BMDP Program 4M (13). Alternative 4,5,6, and 8 factor solutions were examined as were Equimax and Quartimax rotations of the 6-factor solution. These rotations were performed to simplify interpretation. For item by item comparisons, the data were examined in 2 x 2 contingency tables and chi-square with Yates correction for continuity used as a measure of association (14). Cohen’s Kappa ( x ) was employed as a measure of agreement (15). The null hypothesis of no agreement beyond that expected by chance was tested by comparing x divided by its standard error with the standard normal distribution. McNemar’s test for paired data was used as a measure of
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
METHODS Our subjects were employees of a local aluminum smelter participating in a prospective study of respiratory function. The order of testing was the MMRC questionnaire, methacholine challenge, and finally the IUAT questionnaire. Of the 843 workers on site, 828 first answered a MMRC respiratory symptom questionnaire administered by one of the authors (MJA) or a trained nursing sister. Item response rates were between 99.3%and 100%. All but one subject performed baseline flow volume curves to ATS standards (8) on computerized equipment interfaced with a water sealed spirometer (Gould 2400). After giving informed consent, 809 (97.7%) workers subsequently underwent methacholine challenge up to a dose of 6.14 pmol by a rapid method employing hand-held glass nebulizers (9).Twelve subjects were excluded because of poor baseline lung function (FEV1 c 65% predicted for age and height) and seven declined to participate. The dose of methacholine producing a 20% fall in FEVl (PD20) was interpolated on a logarithmic scale between the last two measurements of FEVl when a drop of this magnitude occurred (10). Bronchial hyperresponsiveness was considered present in the 65 subjects who had a PD20 less than 6.14 pmol, which was the maximum total dose of methacholine administered. Dose-response slope (DRS) was calculated as percent change (%A) in FEVl from baselinehtal dose (11).Prior to expression on a logarithmic scale, 1.5% pmol was added to eliminate negative values. Symptom questions from the new IUAT questionnaire (see Appendix I) were completed by 827 subjects. The item response rates were between 97.3%and 100%. Subjects who responded ‘regular’ or ‘always’ to the question concerning breathing difficulty were
132
Abramson et al. Table 1. Salient Factor Loadings for Items from MMRC and IUAT Questionnaires (Loadings Less than 0.3 are Indicated by -1. FACTOR
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
ITEM Asthma Asthma Current Asthma Salbutamol Medication Morning cough Morning cough Morning phlegm Morning phlegm Wheeze Wheeze Nocturnal cough Tightness Tightness Exertional dyspnea Exertional dyspnea Breathing difficulty Dust tightness Dust dyspnea Spontaneous dyspnea Nocturnal dyspnea
QUESTIONNAIRE
1
2
3
4
5
IUAT MMRC IUAT MMRC IUAT IUAT MRC IUAT MMRC IUAT MMRC IUAT IUAT MMRC MMRC IUAT IUAT IUAT IUAT IUAT IUAT
0.867 0.845 0.784 0.721 0.714
-
-
-
-
0.857 0.830 0.822 0.772
-
Eigenvalue Interpretation
-
-
-
-
-
-
0.81 8 0.784 0.589 0.520 0.31 7
-
-
-
-
-
0.336 0.342
-
-
3.733
2.867
2.323
Asthma
Bronchitis
Wheeze
-
-
marginal homogeneity to test for differences in prevalence (14).
RESULTS The Structure of Responses
The correlation matrix between IUAT and selected MMRC items contained many coefficients > 0.3 suggesting it could be meaningfully factored. The number of cases was adequate as it was greater than 10 times the number of variables. Because this was a n exploratory analysis, no systematic attempt was made to identify outlying cases. As the responses were categorical no formal testing of multivariate normality was appropriate. According to Kaiser’s rule of thumb, only a factor with a n eigenvalue greater than one explained more of the variance than a standardized observed variable (12).Thus the five factors meeting this criterion which explained 62% of the total variance were retained for further analysis. The sixth factor contained
-
0.328 0.534 0.756 0.642 - 0.529
-
0.462
2.1 71 SOB
-
-
-
-
-
0.867 0.825
0.378 1.836
Dust
only two variables: daytime cough and sputum, which were highly correlated, but unrelated to other items. Accordingly these variables were deleted on pragmatic grounds and the 5-factor solution with Varimax rotation selected for conceptual simplicity and ease of description. Factor loadings following Varimax rotation are listed in Table 1. Only loadings greater than 0.3 were interpreted as ‘salient’because a lesser loading accounted for under 9% of the variance overlap between the variable and the factor (12).As is customary, the interpretation of these factors was based on the loadings of questionnaire items. For example, salient loadings on the first factor were from items concerning history of asthma or treatment of asthma. Thus this factor was labelled as a n ‘asthma’ factor. More detailed interpretation is discussed below. Construct Validity
This was assessed by comparing responses to substantially similar items from each
Evaluation of a New Asthma Questionnaire
133
Table 2. Agreement Between IUAT and Corresponding MMRC Items
J Asthma Downloaded from informahealthcare.com by UB Giessen on 12/09/14 For personal use only.
PREVALENCE (%) IUAT
MMRC
MMRC
SE ( x )
P
