Sensitivity and specificity of diagnostic tests for impaired nasal respiration Peter S. Vig, BDS, PhD, DOrth, FDSRCS (Eng),* Peter M. Spalding, DDS, MS, MS,** and Ronald R. Lints, DDS, MS*** Pittsburgh, Pa., Lincoln, Neb., and Traverse City, Mich. Diagnostic tests are imperfect and vary in their sensitivity and specificity. The degree of imprecision may be calculated to yield probability estimates of accuracy for both the positive and negative predictions of tests under various conditions. Such information enables clinicians to decide whether to accept or reject test results or the tests themselves. Two pilot studies are reported to establish the diagnostic potential of cephalometric measurements and nasal resistance values for the identification of upper airway impairment. A linear estimate of adenoid size and an area index of adenoid encroachment in the nasopharynx were evaluated as diagnostic tests for increased nasal resistance. The sensitivity of the tests was 31.8% and 18.2%, while specificity was calculated at 83.3% and 66.6%, respectively. In the second study, nasal resistance was evaluated as a test to identify 15ersons whose respiratory mode was equal to or less than 75% nasal airflow. At a NRz value of 5.0 cm H20 per liter per second, the sensitivity of this test was 41.2% and the specificity was 84.0%; with the critical value of NRz at 3.5 cm H20 per liter per second, the sensitivity was 64.7% and the specificity was reduced to 60.0%. The results suggest that these tests are too imprecise for the reliable identification of either those who might benefit from treatment or those for whom treatment is unlikely to yield benefits. (AM J ORTHOD DENTOFACORTHOP 1991;99:354-60.)
T h e purpose of this article is to report two separate, but related, pilot studies that were aimed at testing the sensitivity and specificity of some tests currently used for impaired nasal respiration or, as it is still unfortunately and imprecisely termed, "mouth breathing." A subsidiary, but possibly more important aim, is to demonstrate the application and usefulness of sensitivity analysis for diagnostic procedures as they pertain to orthodontics in general. Although the procedures for evaluating tests are well established in medicine and clinical epidemiologic methodology, they have thus far not been applied to orthodontics.
BACKGROUND All diagnostic tests have some degree of imperfection. Under ideal circumstances, a test should be able to differentiate unequivocally between the presence and absence of a disease, or condition, each time that it is Supported by the United States Public Health Service, National Institutes of Health/National Institutes of Dental Research Grant DE 06881. *Associate Dean of Research and Professor or Orthodontics, School of Dental Medicine, University of Pittsburgh. **Associate Professor. Department of Orthodontics, School of Dentistry, University of Nebraska. ***Private Practice of Orthodontics, Trax'erse City, Michigan. 811121990
354
applied. Unfortunately, however, tests for diagnosing disease may yield incorrect results. The possible outcomes of a diagnostic test are (1) true positive, (2) false positive, (3) true negative, and (4) false negative. The calculated probability that a test will yield false-positive and false-negative results is commonly used to evaluate its precision or predictive power. The ability of a test to distinguish between disease and the absence of disease may be quantified on a 2 × 2 table, as shown in Fig. 1. Four indices are usually calculated. These are sensitivity, specificity, accuracy for positive prediction, and accuracy for negative prediction. Each index addresses a specific attribute of the test's performance. The index that measures the accuracy of positive prediction relates to the question "If a test is positive, how often is the disease present?" The answer is calculated from the ratio a/(a + b) (see Fig. I). The range of possible values for this and the other indices is 0% to 100%. The index that measures accuracy for negative prediction, d/(c + d), addresses the question: "If a test result is negative, how often is the disease absent?" The two more commonly reported indices in the literature are sensitivity and specificity. Sensitivity is computed as a/(a + c) and answers the question "If the disease is present, how often is the test positive?" Specificity is obtained from the ratio
Volume 99 Number 4
d/(b + d), that answers the question "If the disease is absent, how often is the test negative?" Clearly, all the indices are related mathematically, since they are derived from the values a, b, c, and d. The accuracy of positive and negative predictions is related to sensitivity, specificity, and prevalence, which is the number of cases of disease among a group of people at one time and in Fig. 1 is represented as (a + c)/N, where N = (a + b + c + d). More complete discussions of tl~e fundamental principles pertaining to the utility of diagnostic tests are available elsewhere. ~4 If clinical treatment is predicated on the outcome of a test, it is clearly necessary to know how reliable the diagnostic procedure is. Without such information on the accuracy of a diagnosis, it is not possible to estimate the success or failure rate of a treatment, and it follows that, where alternative treatments exist, it becomes impossible to evaluate the efficacy of the available options. From a clinical perspective, both sensitivity and specificity are important attributes of a test. Low sensitivity increases the risk that the test will fail to identify a condition when it exists, while insufficient specificity increases the odds that the test will indicate the presence of a condition that is not, in fact, present. The former imprecision tends to deprive patients of potentially beneficial treatment, while the latter has consequences that may result in unnecessary or inappropriate therapy. Both types of imprecision reduce the utility of the diagnostic process and diminish the chances for appropriate clinical treatment with respect to risk-benefit, cost-benefit, and the probability of obtaining desired outcomes. It is possible to adjust test criteria for optimal sensitivity or specificity. This adjustment is usually made after consideration of the trade-off between the consequences of accepting either a false-positive or a falsenegative result. 56 In life-threatening conditions, where it is preferable to intervene, even if there is some doubt about the validity of the positive test result, it may be reasonable to accept a test with low specificity and high sensitivity. In less critical conditions, in which treatment benefits may be marginal or uncertain, or in which the treatment itself may be elective, it is reasonable to set criteria that yield both high true-positive and truenegative rates diagnostically. If a diagnostic test is incapable of differentiation at an acceptable level of precision between the presence and absence of a condition, then it should be discarded from clinical practice. Such tests are not only a waste of time and money, but they significantly mislead and thus contribute to the incidence of iatrogenic problems. In spite of the long history of interest among ortho-
Diagnostic tests for impaired nasal respiration
355
Disease State Total
+ -!Test Results
Total
a
b
(a+b)
c
d
(c+d)
(a+c)
Prevalence Sensitivity Specificity Acc. Pos. Prediction Acc. Neg. Prediction
(b+d) = = = = =
N
(a+e)/N a/(a+c) d/(b+d) a/(a+b) d/(c+d)
Fig. 1. A 2 x 2 table used for calculations of prevalance, sensitivity, specificity, accuracy of positive prediction, and the accuracy of negative prediction rates. These four indices may be expressed either as percentages or as probabilities. Values can range between 0.0 and 1.0.
dontists for the possible relationship between nasorespiratory function and dentofacial develop/nent, considerable doubt still exists regarding the indications for treatment aimed at modifying breathing or whether, in fact, such treatment significantly improves orthodontic results. As orthodontics, in itself, is elective, and its major and least equivocal benefits may be principally defined as esthetic, any medical or surgical adjunctive treatment should be undertaken with considerable caution and with due concern for the ultimate benefit, risk, and cost. LATERAL CEPHALOMETRIC ESTIMATION OF AIRWAY OBSTRUCTION: STUDY 1 Rationale
Adenoid size and nasopharyngeal airway dimensions, as seen in two-dimensional lateral cephalograms, are still used for the diagnosis of airway obstruction in orthodontic patients. Likewise, some investigators ascribe various dentofacial" morphologic features to "impaired" nasal breathing, which is assumed to exist when airway dimensions are small on cephalograms. The major, and as yet untested, premise is that, below a critical value of a cephalometric linear or area dimension in the nasopharynx, there is an increase in airway resistance that forces a shift from nasal to oral breathing. A further untested assumption implicit in this line of reasoning is that such increases in nasal resistance (NRz) are sufficient to induce postural adaptations that can and do cause significant adverse modifications
356
Am. J. Orthod. Dentofac. Orthop. April 1991
Vig, Spalding, and Lhzts Materials and methods
McNamara L,ne
Fig. 2. Diagram of the nasopharyngeal region, as seen on lateral cephalograms. The linear dimension recommended by McNamara and the "airway space" defined by Shulhofr are shown. Both of these "indices" have been recommended as tests for nasal respiration obstruction caused by the adenoids. The constructed lines in this figure follow: PL = Palatal line; Ba = SpL = basion to sphenoid tangent; PML = perpendicular to PL at the posterior nasal spine; AAL = perpendicular to PL through anterior arch of the atlas. The area within the quadrilateral is divided into Ad (adenoid) and Np (nasopharyngeal airway). McNamara's linear measure of the minimum distance from adenoid to the soft palate is also known.
of facial growth. Such a premise is also implicit in the clinical literature and underlies the tests advocated by Shulhof 7 and McNamara, 8 who are among the more recent advocates of the utility of cephalograms for diagnosing airway obstruction. Those who recommend otolaryngologic or allergy therapy for orthodontic benefit must also believe that treatment to increase nasopharyngeal airway size will reverse these postural adaptations and thereby "improve" subsequent facial growth sufficiently to improve the orthodontic prognosis. Our purpose here is not to discuss the validity of the etiologic assumptions or the efficacy of such treatment. This has already been done elsewhere. 9 Assuming that some benefit may accrue from adopting such a treatment approach, we wish to address a question that is of interest to those who support this view: How accurately can clinicians identify patients who may benefit from such treatment, and how well can they differentiate between such patients and others who are unlikely to derive benefits from treatment? Specifically, the aim of this pilot study was to estimate the diagnostic sensitivity and specificity of the cephalometric methods of McNamara and Shuloff with respect to the nasal resistance values obtained by posterior rhinomanometry.
The sample comprised 40 subjects whose cephalograms showed clearly visible adenoids. The younger patients in this group were deemed by an orthodontist to have radiographic evidence of enlarged adenoids and thus to be possible candidates for ear, nose, and throat evaluation. These patients accounted for 55% of our sample. Members of the total sample g r o u p - - m a l e and female--ranged in age from 6 years to adulthood and were distributed as follows: 6-8 years n = 10 (25.0%) 9-12 years n = 18 (40.0%) 13-16 years n = 12 (30.0%) Over 17 years n = 2 (5.0%) The time that elapsed between cephalometric assessment and nasal resistance testing varied from 0 to 12 months as follows: 0 months, n = 9; 1 month, n = 8; 2 months, n = 6; 3 months, n = 3; 4 months, n = 6, and 5 to 12 months, n = 8. The method for obtaining NRz values was described by Warren.l° Seven values of NRz were obtained for each subject. After the extremes were discarded, the average of the remaining five values was used to characterize each patient's NRz. To evaluate the diagnostic precision of a test, one must establish the critical value that determines the distinction between a positive and a negative test result. Having set the criterion for test results, it is possible to test sensitivity and specificity only if there is a predetermined definition of what constitutes the presence of the condition or disease to which the test is being applied. This process, sometimes referred to as establishing a "gold standard," is frequently more ambiguous than one would like. Whether the "disease" is hypertension or impaired nasal respiration, as revealed by NRz values, some clinical judgment is required to set reasonable parameters for defining the condition. In this study a value of 4.0 cm of HEO/L/sec was selected as representing "high" resistance. Thus subjects with a value equal to or greater than 4.0 were deemed to have the "disease," while lower NRz values placed subjects in the "non-disease" category. It should be noted that setting such a high critical value for increased NRz tends to overestimate the specificity of the cephalometric tests, since low or marginal NRz may be expected to have a weaker association between radiographically determined airway dimensions and respiratory obstruction. In other words, this study was deliberately biased in favor of the finding of high sensitivity of the cephalometric tests. For the cephalometric tests, the published, recom-
Volume 99 Number 4
Diagnostic tests for hnpaired nasal respiration D + = NRz >_ 4.0 cm I t 2 0 / L / s e c . T + = Adenoid - Soft Palate 5. 5.0 ram.
A
D+
D-
T+
7
3
T.
15
15
Prevalence Sensitivity Specificity Acc. Pos. P.rediction Acc. Neg. Prediction
= = = = =
55.0% 31.8% 83.3% 70.0% 50.0%
B
357
D + = N R z > 4 . 0 cm H 2 0 / L / s e c . T + = " A i r w a y Space" ~. 42.0%
D+
D-
T+
4
6
T.
18
12
Prevalence Sensitivity Specificity Acc. Pos. Prediction Acc. Neg. Prediction
= = = = =
55.0% 18.2% 66.6% 40.0% 40.0%
Fig. 3. The two 2 x 2 tables with the test positive and test negative frequencies used for rating the cephalometric tests' ability to predict increased nasal resistance.
mended values of McNamara and Shulhof 7"g were used to establish positive or negative test results. Accordingly, for the minimum linear distance from the adenoid shadow to the dorsal aspect of the soft palate or contiguous nasopharyngeal structures, a measurement of 5 mm or less constituted a positive result, while greater distances were recorded as negative test results. For the Shulhof test, the area of adenoid encroachment in the nasopharynx is the critical measure. 7 A positive test result was recorded if the "airway percent area" was less than 42.0%, with greater values being considered negative (Fig. 2). Results
1. The true positive rate, or sensitivity, of McNamara's test was 31.8%, while the tree negative rate, or specificity, was 83.3%. 2. For the Shuloff"airway size" test, the corresponding values were sensitivity = 18.2% and specificity = 66.6%. 3. The accuracy of positive prediction for the linear dimension defined by McNamara was 70.0%, and for the available airway area defined by Shuloff it was 40.0%. 4. The accuracy of negative prediction was 50.0% and 40.0%, respectively, for these tests. The 2 x 2 tables that yielded these results are shown in Fig. 3. Conclusions
I. The sensitivity of both cephalometric tests is low. If the purpose of these tests is to identify persons
with high nasal resistance from the cephalometric measures, clinicians should be aware that the tests will predict the condition with success rates of only 38% and 26%, respectively. The probability that the tests will identify the condition is substantially worse than would be the case if a coin toss, or random selection, were substituted for the evaluation of cephalometric data. 2. The risk of arriving at incorrect diagnoses, either positive or negative, with these tests is high. 3. If the etiologic premise is valid, a high proportion of patients who might benefit from treatment would not be correctly identified by these tests. 4. The inference of NRz values from such cephalometric estimates of nasopharyngeal patency is unwarranted and argues against the validity of research findings that purport to establish causal relationships between respiratory behavior and cephalometrically determined morphology when such studies fail to include direct measures of respiratory parameters. THE NASAL RESISTANCE TEST: STUDY 2. Rationale
Determination of nasal resistance (NRz) and similar rhinometric or pressure-flow tests are being used increasingly by allergists and otorhinologists to diagnose nasal respiratory obstruction. Arbitrary values of NRz are being used to confirm or rule out diagnoses of impaired nasal breathing. From our previous studies we have shown that it is not uncommon for a patient to have 100% nasal respiration despite a range of NRz values greater than zero. n The diagnostic question here
358
Am. J. Orthod. Dentofac. Orthop. April 1991
Vig, Spalding, and Lints D+ = Less than 75% Nasal Breathing T + = N R z 5.0 em H20/L/see. or a b o v e
A
D+
D-
T+
7
4
T.
10
21
Prevalence = Sensitivity = Specificity = Acc. Pos. Prediction = Acc. Neg. Prediction =
40.5% 41.2% 84.0% 63.6% 67.7%
D + = Less than 75% Nasal Breathing T + = N R z 3.5 crn H 2 0 / L / s e c . or a b o v e
B
D+
D-
T+
11
10
T.
6
15
Prevalence = Sensitivity = Specificity = Acc. Pos. Prediction = Acc. Neg. Prediction =
40.5% 64.7% 60.0% 52.4% 71.0%
Fig. 4. The 2 x 2 tables used for evaluating nasal resistance as a test for identifying persons with less than 75% nasal breathing. Two values for a positive test were established for NRz. Lowering the critical value of NRz makes the test yield a gain in sensitivity but a loss in specificity. This would have the effect of identifying a greater number Of patients as needing treatment, but it would also increase the likelihood of prescribing treatment for those who do not really need it. The cut-off values for such tests are arbitrary, and clinicians must consider the trade-off between the benefits and the risks of intervention versus nonintervention. For some life-threatening conditions, itmay be more acceptable to err in favor of increased sensitivity at the expense of specificity. For purely orthodontic benefits, such a strategy is less rational.
is: How good a predictor of reduced nasal breathing is the estimate of NRz, as recorded by posterior rhinomanometry? Accurately determined respiratory flow rates for both the oral and nasal ports are prerequisites for evaluating the diagnostic precision of NRz. We have previously reported on the SNORT or simultaneous oral and nasal respirometric technique, '2 which has been used for a number of years and for which reproducibility and estimates of error are well known. This technique therefore provides a "gold standard" for determining the extent to which patients breath nasally or orally. Materials and m e t h o d s
The sample consisted of 42 subjects, mixed with respect to sex, age, and the otorhinologist's clinical diagnosis of nasal obstruction. All subjects were tested for nasal resistance and percentage of nasal breathing-i.e., respiratory mode, or percentage of airflow through the nose as a proportion of the total airflow. The respiratory modes of patients in this sample ranged from 38% to 100% nasal. The median value for adults was 98%, and for children it was 75%. In this study < 7 5 % nasal respiration was taken as the arbitrary cut-off point, below which we designated subjects as having nasal impairment (i.e., disease). Two values of NRz were used to establish the diagnostic precision of
the rhinomanometric method. The cut-off values for a positive test were set initially at 5.0 and then at 3.5 cm HzO/L per second. Values equal to or greater than these test criteria were deemed to represent test positives in the two estimates of sensitivity and specificity of NRz. Results
1. At the higher setting of the critical value for the positive NRz test (5.0), we obtained a true positive rate or sensitivity of 41.2% and a true negative rate or specificity of 84%. 2. At the lower threshold (3.5), the values for sensitivity and specificity were 64.7% and 60.0%, respectively. 3. Accuracy for positive prediction at NRz --- 5.0 was 63.6%; at NRz = 3.5 it was 52.4%. 4. Accuracy for negative prediction at NRz = 5.0 was 67.7% at NRz = 3.5 this was 71.0%. The 2 × 2 tables from which these results were obtained are presented in Fig. 4. Conclusions
1. Lowering the threshold for a positive test by reducing the value of the critical NRz from 5.0 to 3.5 increases the sensitivity but decreases the specificity of the test.
Volume 99 Number 4
2. A true positive rate of 64.7% fails to identify 35.3% of cases with the condition. 3. A true negative rate of 60.0% fails to identify 40.0% of cases without the condition. 4. NRz is an imprecise diagnostic test to determine the need for ENT surgery or allergy treatment for nasal respiration below the 75% level. 5. Considering both the risk-benefit and cost-benefit factors, in terms of potential orthodontic improvement, the diagnostic precision of NRz is inadequate for determining appropriate treatment with respect to respiratory mode alteration. Discussion
At present, clinicians who'use cephalograms to diagnose nasal obstruction do not have access to the SNORT technique; at best, they rely on nasal resistance values to confirm or. support a diagnosis of impaired nasal respiration. For these clinicians it is desirable to have currently usable information on the quality of tests that they use in practice. The decision to continue using tests that are known to be imprecise can then be made intelligently, and the consequences of accepting test results can be weighed on an individual basis. In these two pilot studies the prevalence of disease was set by the criteria we selected; it was 55.0% for the cephalometric estimation of airway of obstruction and 40.5% for the nasal resistance test. These figures obviously are higher than would be expected in unselected samples taken from an orthodontic population. One result of such selection bias is to make the sensitivity of the test appear higher than it actually would be. A further point worth considering is the usefulness of a screening test, followed by more refined tests, for patients identified as being at risk. For example, if a test with an accuracy rating of 100% for positive prediction (McNamara's linear measure) is deemed sufficient to identify a patient with possible nasal obstruction, and if this decision is followed by a referral to an otorhinologist who then uses the NRz test, what is the probability of an accurate positive prediction? This is an example of joint probability of the two tests determining the adequacy of the diagnosis. If the respective probabilities of the two tests are 70.0% and 52.4% (i. e., pl = 0.70 and P2 = 0.52), then the joint probability is obtained by their product, which is 0.70 × 0.52, or 0.364. A 36.4% chance of obtaining an accurate diagnosis is certainly unsatisfactory. Our two studies were done at different times and on two separate samples. If the aim is to estimate sensitivity and specificity of the two cephalometric tests as well as NRz with reference to the gold standard of quantified percent of nasal contribution to total respi-
Diagnostic tests f o r impaired nasal respiration
359
ration, it is clearly necessary to use the same sample for all the comparisons and estimates of sensitivity and specificity. The data set available to us precluded such a protocol, primarily because radiation hygiene guidelines do not permit the taking of cephalograms for purely research purposes. As our studies progress, we expect to have a sufficient number of subjects with all the records necessary to refine our present findings. Although one may anticipate some differences in the values to be obtained from different samples, it is unlikely that our conclusions will require substantive revisions. The reason for our confidence in this regard stems from the findings of Warren l° concerning the lumen size required to permit unimpeded nasal respiratory airflow. From model studies, Warren found that, at flow rates consistent With normal respiratory effort and ventilation rates, a nasal airway cross-sectional area of 0.62 cm 2 for adults and 0.43 cm ~ for children was adequate. Since area is a product of depth and width, and the width of the nasopharyngeal space in the vicinity of the adenoids is considerable, it is unlikely that any significant improvement is possible in the diagnostic ability of two-dimensional cephalograms for the determination of nasopharyngeal obstruction to airflow. For example, even if the critical value of the linear distance from adenoid to soft palate were to be reduced to 2.0 mm (from the present 5.0 mm value), a lateral dimension or the width of the nasopharynx of 3.1 cm would yield a cross-sectional area of airway lumen adequate for total nasal respiration. Although the utility of diagnostic tests for individual and population screening programs (e.g., AIDS, hypertension, and breast cancer) has received considerable attention recently, the subject has received relatively scant attention in the orthodontic literature. We do, in fact, use a wide array of tests in orthodontic practice, and treatment decisions are frequently modified because these tests are accepted as valid. This is true not only for routine orthodontic treatment but also for the difficult area of conditions associated with temporomandibular joint pain and dysfunction, as well as for other conditions within the purview of contemporary orthodontic specialties. The secondary aim of this article was to identify the need for a systematic analysis of the reliability of tests that we use and, in particular, to encourage attempts to validate diagnostic procedures in terms of their sensitivity and specificity, as illustrated in our study. The consequences of accepting weak or inadequate diagnostic tests in clinical practice are self-evident. A possibly more damaging result is the application of such tests to clinical research, which is aimed at improving our effectiveness as providers of optimum treatment.
360
Am. J. Orthod. Dentofac. Orthop. April 1991
Vig, Spalding, and Lhlts
If, for example, orthodontists refer a significant number of patients with long lower facial height to otolaryngologists o r to allergists for evaluation o f nasorespiratory obstruction, the selection o f subjects is based on dentofacial morphology. If the medical specialist assumes that orthodontists are knowledgeable about facial growth and that the referred patients have been identified as abnormal growers, the bias is introduced in favor of finding some etiologic factor which, in this case, is nasal obstruction. In the absence o f tests better than cephalometric radiography or NRz, the likelihood of finding a high prevelance of mouth breathing exists. If the medical practitioner then reviews his or her records, a statistical association beyond chance may be expected between the presence o f long-faced morphology and nasal obstruction. This statistical finding would, in fact, be clinically s p u r i o u s - - t h e result of a combination of selection bias and lack of diagnostic precision. Nevertheless, such findings could be, and in fact have been in the past, regarded as compelling clhzical evidence derived from two independent clinical specialties. Such methodologic pitfalls are not uncommon in clinical research, and their effects on practice tend to be particularly resistant to eradication from the clinical disciplines concerned. Research workers who challenge such concepts face the reluctance of clinicians to give up familiar procedures on which they have relied, even if these procedures can be shown to have questionable utility. Only by increasing general awareness o f the need for evaluating diagnostic tests can progress in this important area o f orthodontics be made. General conclusions
1. The applicability o f sensitivity and specificity testing to orthodontic diagnostic procedures has been demonstrated. 2. Both o f the two diagnostic applications o f cepha-
lometrics that were examined are too imprecise for establishing nasopharyngeal obstruction with adequate sensitivity or specificity. 3. The use o f nasal resistance determination to corroborate less than 75% nasal breathing is a test that also has limited precision and, therefore, utility. REFERENCES 1. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 1978;299:926-30. 2. Weinstein MC, Feinberg HV. Clinical decision analysis. Philadelphia: WB Saunders, 1980. 3. Feinstein AR. Clinical biostatisties. St. Louis: CV Mosby 1977. 4. Oriner PF, Maycwski RJ, Mushlin At, Greenland P. Selection and interpretation of diagnostic tests and procedures. Ann Intern Med 1981;94:453-600. 5. Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1981;8:283-98. 6. Pauker SO, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med 1980;302:1109-17. 7. Shulhof RJ. Considerations of airway in orthodontics. J Clin Orthod 1978;12:440--4. 8. McNamara JA Jr. A method of cephalometric evaluation. AM J OR'IttOD 1984;86:449-69. 9. Vig PS. Respiration, nasal airway and orthodontics: a review of current clinical concepts and research, in: Johnston LE Jr ed. New Vistas in Orthodontics. Philadelphia: Lea & Febiger, 1985. 10. Warren DW. A quantitative technique for assessing nasal airway •impairment. A,~tJ OR'rItoD 1984;86:306-14. I I. Keall HJ. The relationship between nasal resistance and respiratory mode [Master's Thesis]. Ann Arbor: The University of Michigan, 1986. 12. Keall CL, Vig PS. An improved technique for the simultaneous measurement of nasal and oral respiration. A,~ J ORrHoI~ 1987;91:207-12.
Reprint requests to: Dr. Peter S. Vig Salk Hall School of Dental Medicine University of Pittsburgh Pittsburgh, PA 15261
T h e purpose of this article is to report two separate, but related, pilot studies that were aimed at testing the sensitivity and specificity of some tests currently used for impaired nasal respiration or, as it is still unfortunately and imprecisely termed, "mouth breathing." A subsidiary, but possibly more important aim, is to demonstrate the application and usefulness of sensitivity analysis for diagnostic procedures as they pertain to orthodontics in general. Although the procedures for evaluating tests are well established in medicine and clinical epidemiologic methodology, they have thus far not been applied to orthodontics.
BACKGROUND All diagnostic tests have some degree of imperfection. Under ideal circumstances, a test should be able to differentiate unequivocally between the presence and absence of a disease, or condition, each time that it is Supported by the United States Public Health Service, National Institutes of Health/National Institutes of Dental Research Grant DE 06881. *Associate Dean of Research and Professor or Orthodontics, School of Dental Medicine, University of Pittsburgh. **Associate Professor. Department of Orthodontics, School of Dentistry, University of Nebraska. ***Private Practice of Orthodontics, Trax'erse City, Michigan. 811121990
354
applied. Unfortunately, however, tests for diagnosing disease may yield incorrect results. The possible outcomes of a diagnostic test are (1) true positive, (2) false positive, (3) true negative, and (4) false negative. The calculated probability that a test will yield false-positive and false-negative results is commonly used to evaluate its precision or predictive power. The ability of a test to distinguish between disease and the absence of disease may be quantified on a 2 × 2 table, as shown in Fig. 1. Four indices are usually calculated. These are sensitivity, specificity, accuracy for positive prediction, and accuracy for negative prediction. Each index addresses a specific attribute of the test's performance. The index that measures the accuracy of positive prediction relates to the question "If a test is positive, how often is the disease present?" The answer is calculated from the ratio a/(a + b) (see Fig. I). The range of possible values for this and the other indices is 0% to 100%. The index that measures accuracy for negative prediction, d/(c + d), addresses the question: "If a test result is negative, how often is the disease absent?" The two more commonly reported indices in the literature are sensitivity and specificity. Sensitivity is computed as a/(a + c) and answers the question "If the disease is present, how often is the test positive?" Specificity is obtained from the ratio
Volume 99 Number 4
d/(b + d), that answers the question "If the disease is absent, how often is the test negative?" Clearly, all the indices are related mathematically, since they are derived from the values a, b, c, and d. The accuracy of positive and negative predictions is related to sensitivity, specificity, and prevalence, which is the number of cases of disease among a group of people at one time and in Fig. 1 is represented as (a + c)/N, where N = (a + b + c + d). More complete discussions of tl~e fundamental principles pertaining to the utility of diagnostic tests are available elsewhere. ~4 If clinical treatment is predicated on the outcome of a test, it is clearly necessary to know how reliable the diagnostic procedure is. Without such information on the accuracy of a diagnosis, it is not possible to estimate the success or failure rate of a treatment, and it follows that, where alternative treatments exist, it becomes impossible to evaluate the efficacy of the available options. From a clinical perspective, both sensitivity and specificity are important attributes of a test. Low sensitivity increases the risk that the test will fail to identify a condition when it exists, while insufficient specificity increases the odds that the test will indicate the presence of a condition that is not, in fact, present. The former imprecision tends to deprive patients of potentially beneficial treatment, while the latter has consequences that may result in unnecessary or inappropriate therapy. Both types of imprecision reduce the utility of the diagnostic process and diminish the chances for appropriate clinical treatment with respect to risk-benefit, cost-benefit, and the probability of obtaining desired outcomes. It is possible to adjust test criteria for optimal sensitivity or specificity. This adjustment is usually made after consideration of the trade-off between the consequences of accepting either a false-positive or a falsenegative result. 56 In life-threatening conditions, where it is preferable to intervene, even if there is some doubt about the validity of the positive test result, it may be reasonable to accept a test with low specificity and high sensitivity. In less critical conditions, in which treatment benefits may be marginal or uncertain, or in which the treatment itself may be elective, it is reasonable to set criteria that yield both high true-positive and truenegative rates diagnostically. If a diagnostic test is incapable of differentiation at an acceptable level of precision between the presence and absence of a condition, then it should be discarded from clinical practice. Such tests are not only a waste of time and money, but they significantly mislead and thus contribute to the incidence of iatrogenic problems. In spite of the long history of interest among ortho-
Diagnostic tests for impaired nasal respiration
355
Disease State Total
+ -!Test Results
Total
a
b
(a+b)
c
d
(c+d)
(a+c)
Prevalence Sensitivity Specificity Acc. Pos. Prediction Acc. Neg. Prediction
(b+d) = = = = =
N
(a+e)/N a/(a+c) d/(b+d) a/(a+b) d/(c+d)
Fig. 1. A 2 x 2 table used for calculations of prevalance, sensitivity, specificity, accuracy of positive prediction, and the accuracy of negative prediction rates. These four indices may be expressed either as percentages or as probabilities. Values can range between 0.0 and 1.0.
dontists for the possible relationship between nasorespiratory function and dentofacial develop/nent, considerable doubt still exists regarding the indications for treatment aimed at modifying breathing or whether, in fact, such treatment significantly improves orthodontic results. As orthodontics, in itself, is elective, and its major and least equivocal benefits may be principally defined as esthetic, any medical or surgical adjunctive treatment should be undertaken with considerable caution and with due concern for the ultimate benefit, risk, and cost. LATERAL CEPHALOMETRIC ESTIMATION OF AIRWAY OBSTRUCTION: STUDY 1 Rationale
Adenoid size and nasopharyngeal airway dimensions, as seen in two-dimensional lateral cephalograms, are still used for the diagnosis of airway obstruction in orthodontic patients. Likewise, some investigators ascribe various dentofacial" morphologic features to "impaired" nasal breathing, which is assumed to exist when airway dimensions are small on cephalograms. The major, and as yet untested, premise is that, below a critical value of a cephalometric linear or area dimension in the nasopharynx, there is an increase in airway resistance that forces a shift from nasal to oral breathing. A further untested assumption implicit in this line of reasoning is that such increases in nasal resistance (NRz) are sufficient to induce postural adaptations that can and do cause significant adverse modifications
356
Am. J. Orthod. Dentofac. Orthop. April 1991
Vig, Spalding, and Lhzts Materials and methods
McNamara L,ne
Fig. 2. Diagram of the nasopharyngeal region, as seen on lateral cephalograms. The linear dimension recommended by McNamara and the "airway space" defined by Shulhofr are shown. Both of these "indices" have been recommended as tests for nasal respiration obstruction caused by the adenoids. The constructed lines in this figure follow: PL = Palatal line; Ba = SpL = basion to sphenoid tangent; PML = perpendicular to PL at the posterior nasal spine; AAL = perpendicular to PL through anterior arch of the atlas. The area within the quadrilateral is divided into Ad (adenoid) and Np (nasopharyngeal airway). McNamara's linear measure of the minimum distance from adenoid to the soft palate is also known.
of facial growth. Such a premise is also implicit in the clinical literature and underlies the tests advocated by Shulhof 7 and McNamara, 8 who are among the more recent advocates of the utility of cephalograms for diagnosing airway obstruction. Those who recommend otolaryngologic or allergy therapy for orthodontic benefit must also believe that treatment to increase nasopharyngeal airway size will reverse these postural adaptations and thereby "improve" subsequent facial growth sufficiently to improve the orthodontic prognosis. Our purpose here is not to discuss the validity of the etiologic assumptions or the efficacy of such treatment. This has already been done elsewhere. 9 Assuming that some benefit may accrue from adopting such a treatment approach, we wish to address a question that is of interest to those who support this view: How accurately can clinicians identify patients who may benefit from such treatment, and how well can they differentiate between such patients and others who are unlikely to derive benefits from treatment? Specifically, the aim of this pilot study was to estimate the diagnostic sensitivity and specificity of the cephalometric methods of McNamara and Shuloff with respect to the nasal resistance values obtained by posterior rhinomanometry.
The sample comprised 40 subjects whose cephalograms showed clearly visible adenoids. The younger patients in this group were deemed by an orthodontist to have radiographic evidence of enlarged adenoids and thus to be possible candidates for ear, nose, and throat evaluation. These patients accounted for 55% of our sample. Members of the total sample g r o u p - - m a l e and female--ranged in age from 6 years to adulthood and were distributed as follows: 6-8 years n = 10 (25.0%) 9-12 years n = 18 (40.0%) 13-16 years n = 12 (30.0%) Over 17 years n = 2 (5.0%) The time that elapsed between cephalometric assessment and nasal resistance testing varied from 0 to 12 months as follows: 0 months, n = 9; 1 month, n = 8; 2 months, n = 6; 3 months, n = 3; 4 months, n = 6, and 5 to 12 months, n = 8. The method for obtaining NRz values was described by Warren.l° Seven values of NRz were obtained for each subject. After the extremes were discarded, the average of the remaining five values was used to characterize each patient's NRz. To evaluate the diagnostic precision of a test, one must establish the critical value that determines the distinction between a positive and a negative test result. Having set the criterion for test results, it is possible to test sensitivity and specificity only if there is a predetermined definition of what constitutes the presence of the condition or disease to which the test is being applied. This process, sometimes referred to as establishing a "gold standard," is frequently more ambiguous than one would like. Whether the "disease" is hypertension or impaired nasal respiration, as revealed by NRz values, some clinical judgment is required to set reasonable parameters for defining the condition. In this study a value of 4.0 cm of HEO/L/sec was selected as representing "high" resistance. Thus subjects with a value equal to or greater than 4.0 were deemed to have the "disease," while lower NRz values placed subjects in the "non-disease" category. It should be noted that setting such a high critical value for increased NRz tends to overestimate the specificity of the cephalometric tests, since low or marginal NRz may be expected to have a weaker association between radiographically determined airway dimensions and respiratory obstruction. In other words, this study was deliberately biased in favor of the finding of high sensitivity of the cephalometric tests. For the cephalometric tests, the published, recom-
Volume 99 Number 4
Diagnostic tests for hnpaired nasal respiration D + = NRz >_ 4.0 cm I t 2 0 / L / s e c . T + = Adenoid - Soft Palate 5. 5.0 ram.
A
D+
D-
T+
7
3
T.
15
15
Prevalence Sensitivity Specificity Acc. Pos. P.rediction Acc. Neg. Prediction
= = = = =
55.0% 31.8% 83.3% 70.0% 50.0%
B
357
D + = N R z > 4 . 0 cm H 2 0 / L / s e c . T + = " A i r w a y Space" ~. 42.0%
D+
D-
T+
4
6
T.
18
12
Prevalence Sensitivity Specificity Acc. Pos. Prediction Acc. Neg. Prediction
= = = = =
55.0% 18.2% 66.6% 40.0% 40.0%
Fig. 3. The two 2 x 2 tables with the test positive and test negative frequencies used for rating the cephalometric tests' ability to predict increased nasal resistance.
mended values of McNamara and Shulhof 7"g were used to establish positive or negative test results. Accordingly, for the minimum linear distance from the adenoid shadow to the dorsal aspect of the soft palate or contiguous nasopharyngeal structures, a measurement of 5 mm or less constituted a positive result, while greater distances were recorded as negative test results. For the Shulhof test, the area of adenoid encroachment in the nasopharynx is the critical measure. 7 A positive test result was recorded if the "airway percent area" was less than 42.0%, with greater values being considered negative (Fig. 2). Results
1. The true positive rate, or sensitivity, of McNamara's test was 31.8%, while the tree negative rate, or specificity, was 83.3%. 2. For the Shuloff"airway size" test, the corresponding values were sensitivity = 18.2% and specificity = 66.6%. 3. The accuracy of positive prediction for the linear dimension defined by McNamara was 70.0%, and for the available airway area defined by Shuloff it was 40.0%. 4. The accuracy of negative prediction was 50.0% and 40.0%, respectively, for these tests. The 2 x 2 tables that yielded these results are shown in Fig. 3. Conclusions
I. The sensitivity of both cephalometric tests is low. If the purpose of these tests is to identify persons
with high nasal resistance from the cephalometric measures, clinicians should be aware that the tests will predict the condition with success rates of only 38% and 26%, respectively. The probability that the tests will identify the condition is substantially worse than would be the case if a coin toss, or random selection, were substituted for the evaluation of cephalometric data. 2. The risk of arriving at incorrect diagnoses, either positive or negative, with these tests is high. 3. If the etiologic premise is valid, a high proportion of patients who might benefit from treatment would not be correctly identified by these tests. 4. The inference of NRz values from such cephalometric estimates of nasopharyngeal patency is unwarranted and argues against the validity of research findings that purport to establish causal relationships between respiratory behavior and cephalometrically determined morphology when such studies fail to include direct measures of respiratory parameters. THE NASAL RESISTANCE TEST: STUDY 2. Rationale
Determination of nasal resistance (NRz) and similar rhinometric or pressure-flow tests are being used increasingly by allergists and otorhinologists to diagnose nasal respiratory obstruction. Arbitrary values of NRz are being used to confirm or rule out diagnoses of impaired nasal breathing. From our previous studies we have shown that it is not uncommon for a patient to have 100% nasal respiration despite a range of NRz values greater than zero. n The diagnostic question here
358
Am. J. Orthod. Dentofac. Orthop. April 1991
Vig, Spalding, and Lints D+ = Less than 75% Nasal Breathing T + = N R z 5.0 em H20/L/see. or a b o v e
A
D+
D-
T+
7
4
T.
10
21
Prevalence = Sensitivity = Specificity = Acc. Pos. Prediction = Acc. Neg. Prediction =
40.5% 41.2% 84.0% 63.6% 67.7%
D + = Less than 75% Nasal Breathing T + = N R z 3.5 crn H 2 0 / L / s e c . or a b o v e
B
D+
D-
T+
11
10
T.
6
15
Prevalence = Sensitivity = Specificity = Acc. Pos. Prediction = Acc. Neg. Prediction =
40.5% 64.7% 60.0% 52.4% 71.0%
Fig. 4. The 2 x 2 tables used for evaluating nasal resistance as a test for identifying persons with less than 75% nasal breathing. Two values for a positive test were established for NRz. Lowering the critical value of NRz makes the test yield a gain in sensitivity but a loss in specificity. This would have the effect of identifying a greater number Of patients as needing treatment, but it would also increase the likelihood of prescribing treatment for those who do not really need it. The cut-off values for such tests are arbitrary, and clinicians must consider the trade-off between the benefits and the risks of intervention versus nonintervention. For some life-threatening conditions, itmay be more acceptable to err in favor of increased sensitivity at the expense of specificity. For purely orthodontic benefits, such a strategy is less rational.
is: How good a predictor of reduced nasal breathing is the estimate of NRz, as recorded by posterior rhinomanometry? Accurately determined respiratory flow rates for both the oral and nasal ports are prerequisites for evaluating the diagnostic precision of NRz. We have previously reported on the SNORT or simultaneous oral and nasal respirometric technique, '2 which has been used for a number of years and for which reproducibility and estimates of error are well known. This technique therefore provides a "gold standard" for determining the extent to which patients breath nasally or orally. Materials and m e t h o d s
The sample consisted of 42 subjects, mixed with respect to sex, age, and the otorhinologist's clinical diagnosis of nasal obstruction. All subjects were tested for nasal resistance and percentage of nasal breathing-i.e., respiratory mode, or percentage of airflow through the nose as a proportion of the total airflow. The respiratory modes of patients in this sample ranged from 38% to 100% nasal. The median value for adults was 98%, and for children it was 75%. In this study < 7 5 % nasal respiration was taken as the arbitrary cut-off point, below which we designated subjects as having nasal impairment (i.e., disease). Two values of NRz were used to establish the diagnostic precision of
the rhinomanometric method. The cut-off values for a positive test were set initially at 5.0 and then at 3.5 cm HzO/L per second. Values equal to or greater than these test criteria were deemed to represent test positives in the two estimates of sensitivity and specificity of NRz. Results
1. At the higher setting of the critical value for the positive NRz test (5.0), we obtained a true positive rate or sensitivity of 41.2% and a true negative rate or specificity of 84%. 2. At the lower threshold (3.5), the values for sensitivity and specificity were 64.7% and 60.0%, respectively. 3. Accuracy for positive prediction at NRz --- 5.0 was 63.6%; at NRz = 3.5 it was 52.4%. 4. Accuracy for negative prediction at NRz = 5.0 was 67.7% at NRz = 3.5 this was 71.0%. The 2 × 2 tables from which these results were obtained are presented in Fig. 4. Conclusions
1. Lowering the threshold for a positive test by reducing the value of the critical NRz from 5.0 to 3.5 increases the sensitivity but decreases the specificity of the test.
Volume 99 Number 4
2. A true positive rate of 64.7% fails to identify 35.3% of cases with the condition. 3. A true negative rate of 60.0% fails to identify 40.0% of cases without the condition. 4. NRz is an imprecise diagnostic test to determine the need for ENT surgery or allergy treatment for nasal respiration below the 75% level. 5. Considering both the risk-benefit and cost-benefit factors, in terms of potential orthodontic improvement, the diagnostic precision of NRz is inadequate for determining appropriate treatment with respect to respiratory mode alteration. Discussion
At present, clinicians who'use cephalograms to diagnose nasal obstruction do not have access to the SNORT technique; at best, they rely on nasal resistance values to confirm or. support a diagnosis of impaired nasal respiration. For these clinicians it is desirable to have currently usable information on the quality of tests that they use in practice. The decision to continue using tests that are known to be imprecise can then be made intelligently, and the consequences of accepting test results can be weighed on an individual basis. In these two pilot studies the prevalence of disease was set by the criteria we selected; it was 55.0% for the cephalometric estimation of airway of obstruction and 40.5% for the nasal resistance test. These figures obviously are higher than would be expected in unselected samples taken from an orthodontic population. One result of such selection bias is to make the sensitivity of the test appear higher than it actually would be. A further point worth considering is the usefulness of a screening test, followed by more refined tests, for patients identified as being at risk. For example, if a test with an accuracy rating of 100% for positive prediction (McNamara's linear measure) is deemed sufficient to identify a patient with possible nasal obstruction, and if this decision is followed by a referral to an otorhinologist who then uses the NRz test, what is the probability of an accurate positive prediction? This is an example of joint probability of the two tests determining the adequacy of the diagnosis. If the respective probabilities of the two tests are 70.0% and 52.4% (i. e., pl = 0.70 and P2 = 0.52), then the joint probability is obtained by their product, which is 0.70 × 0.52, or 0.364. A 36.4% chance of obtaining an accurate diagnosis is certainly unsatisfactory. Our two studies were done at different times and on two separate samples. If the aim is to estimate sensitivity and specificity of the two cephalometric tests as well as NRz with reference to the gold standard of quantified percent of nasal contribution to total respi-
Diagnostic tests f o r impaired nasal respiration
359
ration, it is clearly necessary to use the same sample for all the comparisons and estimates of sensitivity and specificity. The data set available to us precluded such a protocol, primarily because radiation hygiene guidelines do not permit the taking of cephalograms for purely research purposes. As our studies progress, we expect to have a sufficient number of subjects with all the records necessary to refine our present findings. Although one may anticipate some differences in the values to be obtained from different samples, it is unlikely that our conclusions will require substantive revisions. The reason for our confidence in this regard stems from the findings of Warren l° concerning the lumen size required to permit unimpeded nasal respiratory airflow. From model studies, Warren found that, at flow rates consistent With normal respiratory effort and ventilation rates, a nasal airway cross-sectional area of 0.62 cm 2 for adults and 0.43 cm ~ for children was adequate. Since area is a product of depth and width, and the width of the nasopharyngeal space in the vicinity of the adenoids is considerable, it is unlikely that any significant improvement is possible in the diagnostic ability of two-dimensional cephalograms for the determination of nasopharyngeal obstruction to airflow. For example, even if the critical value of the linear distance from adenoid to soft palate were to be reduced to 2.0 mm (from the present 5.0 mm value), a lateral dimension or the width of the nasopharynx of 3.1 cm would yield a cross-sectional area of airway lumen adequate for total nasal respiration. Although the utility of diagnostic tests for individual and population screening programs (e.g., AIDS, hypertension, and breast cancer) has received considerable attention recently, the subject has received relatively scant attention in the orthodontic literature. We do, in fact, use a wide array of tests in orthodontic practice, and treatment decisions are frequently modified because these tests are accepted as valid. This is true not only for routine orthodontic treatment but also for the difficult area of conditions associated with temporomandibular joint pain and dysfunction, as well as for other conditions within the purview of contemporary orthodontic specialties. The secondary aim of this article was to identify the need for a systematic analysis of the reliability of tests that we use and, in particular, to encourage attempts to validate diagnostic procedures in terms of their sensitivity and specificity, as illustrated in our study. The consequences of accepting weak or inadequate diagnostic tests in clinical practice are self-evident. A possibly more damaging result is the application of such tests to clinical research, which is aimed at improving our effectiveness as providers of optimum treatment.
360
Am. J. Orthod. Dentofac. Orthop. April 1991
Vig, Spalding, and Lhlts
If, for example, orthodontists refer a significant number of patients with long lower facial height to otolaryngologists o r to allergists for evaluation o f nasorespiratory obstruction, the selection o f subjects is based on dentofacial morphology. If the medical specialist assumes that orthodontists are knowledgeable about facial growth and that the referred patients have been identified as abnormal growers, the bias is introduced in favor of finding some etiologic factor which, in this case, is nasal obstruction. In the absence o f tests better than cephalometric radiography or NRz, the likelihood of finding a high prevelance of mouth breathing exists. If the medical practitioner then reviews his or her records, a statistical association beyond chance may be expected between the presence o f long-faced morphology and nasal obstruction. This statistical finding would, in fact, be clinically s p u r i o u s - - t h e result of a combination of selection bias and lack of diagnostic precision. Nevertheless, such findings could be, and in fact have been in the past, regarded as compelling clhzical evidence derived from two independent clinical specialties. Such methodologic pitfalls are not uncommon in clinical research, and their effects on practice tend to be particularly resistant to eradication from the clinical disciplines concerned. Research workers who challenge such concepts face the reluctance of clinicians to give up familiar procedures on which they have relied, even if these procedures can be shown to have questionable utility. Only by increasing general awareness o f the need for evaluating diagnostic tests can progress in this important area o f orthodontics be made. General conclusions
1. The applicability o f sensitivity and specificity testing to orthodontic diagnostic procedures has been demonstrated. 2. Both o f the two diagnostic applications o f cepha-
lometrics that were examined are too imprecise for establishing nasopharyngeal obstruction with adequate sensitivity or specificity. 3. The use o f nasal resistance determination to corroborate less than 75% nasal breathing is a test that also has limited precision and, therefore, utility. REFERENCES 1. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 1978;299:926-30. 2. Weinstein MC, Feinberg HV. Clinical decision analysis. Philadelphia: WB Saunders, 1980. 3. Feinstein AR. Clinical biostatisties. St. Louis: CV Mosby 1977. 4. Oriner PF, Maycwski RJ, Mushlin At, Greenland P. Selection and interpretation of diagnostic tests and procedures. Ann Intern Med 1981;94:453-600. 5. Metz CE. Basic principles of ROC analysis. Semin Nucl Med 1981;8:283-98. 6. Pauker SO, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med 1980;302:1109-17. 7. Shulhof RJ. Considerations of airway in orthodontics. J Clin Orthod 1978;12:440--4. 8. McNamara JA Jr. A method of cephalometric evaluation. AM J OR'IttOD 1984;86:449-69. 9. Vig PS. Respiration, nasal airway and orthodontics: a review of current clinical concepts and research, in: Johnston LE Jr ed. New Vistas in Orthodontics. Philadelphia: Lea & Febiger, 1985. 10. Warren DW. A quantitative technique for assessing nasal airway •impairment. A,~tJ OR'rItoD 1984;86:306-14. I I. Keall HJ. The relationship between nasal resistance and respiratory mode [Master's Thesis]. Ann Arbor: The University of Michigan, 1986. 12. Keall CL, Vig PS. An improved technique for the simultaneous measurement of nasal and oral respiration. A,~ J ORrHoI~ 1987;91:207-12.
Reprint requests to: Dr. Peter S. Vig Salk Hall School of Dental Medicine University of Pittsburgh Pittsburgh, PA 15261
