Journal of Public Health Dentistry . ISSN 0022-4006
Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2009-2010 Bruce A. Dye, DDS, MPH1; Xianfen Li, MS2; Brenda G. Lewis, MPH1; Tim Iafolla, DDS, MPH3; Eugenio D. Beltran-Aguilar, DMD, MPH, MS, DrPH4; Paul I. Eke, PhD4 1 National Center for Health Statistics, Centers for Disease Control and Prevention, Hyattsville, MD, USA 2 Harris Corporation, Fairfax, VA, USA 3 National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD, USA 4 National Center for Chronic Disease Prevention and Health Promotion, Division of Oral Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
Keywords NHANES; oral health; data reliability; epidemiology; surveillance; periodontal disease; quality assurance; dental public health. Correspondence Dr. Bruce A. Dye, CDC/NCHS/NHANES Program, 3311 Toledo Road, RM 4416, Hyattsville, MD 20782. Tel.: 301-458-4199; Fax: 301-458-4029; e-mail: [email protected]. Bruce A. Dye and Brenda G. Lewis are with the National Center for Health Statistics, Centers for Disease Control and Prevention. Xianfen Li is with the Harris Corporation. Tim Iafolla is with the National Institutes of Health, National Institute of Dental and Craniofacial Research. Eugenio D. Beltran-Aguilar and Paul I. Eke are with the National Center for Chronic Disease Prevention and Health Promotion, Division of Oral Health, Centers for Disease Control and Prevention. Ethics Statement: All study participants gave informed consent in accordance with the Ethics Review Board and study ethic guidelines at the Centers for Disease Control and Prevention. The authors do not have any financial or other competing interests to declare.
Abstract Objective: In 2009-2010, the oral health component for the National Health and Nutrition Examination Survey (NHANES) focused on adult periodontal health and included a full mouth periodontal examination as well as a series of questions adminis during the home interview. During this period, intraoral assessments were conducted by dental hygienists. Methods: This report provides oral health content information and results of dental examiner reliability for data collected during NHANES 2009-2010 on 7,189 persons aged 3-19 years and 30 years and older representing the US civilian, noninstitutionalized population in these age groups. Results: For caries and dental sealant assessments, Kappa statistics ranged from 0.71 to 1.00. Kappa scores for moderate and severe periodontitis using the Centers for Disease Control and Prevention/American Academy of Periodontology case definition guidelines was 0.70, but were lower for other periodontal status definitions. When defining moderate or severe periodontitis based on the NHANES 2003-2004 study, protocols using data from only three facial periodontal sites, the Kappa scores were 0.64 and 0.55. Interclass correlation coefficients (ICCs) for mean attachment loss were 0.80 or higher for both examiners. Site-specific mean attachment loss ICCs were generally higher for interproximal measurements compared with mid-facial and mid-lingual measurements. Conclusion: Overall, the data reliability analyses conducted for 2009-2010 indicate an acceptable level of data quality and that examiner (dental hygienist) performance in this data collection cycle is similar to prior survey periods since the NHANES continuous survey began in 1999.
Received: 8/28/2012; accepted: 4/5/2014. doi: 10.1111/jphd.12056 Journal of Public Health Dentistry 74 (2014) 248–256
Introduction Public health surveillance is the ongoing collection, evaluation, and interpretation of systematically collected health data (1). However, there are some factors that affect the sur248
veillance of oral health diseases and related conditions that lead to the periodic model rather than the ongoing model for surveillance that is often desired (2). Some of these factors are related to the chronic nature of many oral diseases and the high levels of resources needed to have an ongoing
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
surveillance process. It is generally believed within the professional dental community that dental caries and periodontal diseases generally progress slowly and can have latent periods of low or no activity. Although both diseases require the presence of putative bacteria, other conditions must be present to facilitate disease development and progression. Consequently, disease status does not change as frequently in populations as a more typical infectious disease would. More importantly, the mechanism we rely on at the national level to assess for the prevalence of these two common oral diseases has been based on a physical evaluation, which has remained essentially unchanged since the 1960s. The Centers for Disease Control and Prevention (CDC) has been exploring alternate methodologies to ascertain oral health status that is clinically meaningful and scientifically based that can be used for public health surveillance when resources are limited, but information collection remains important for monitoring the public’s health. From 2005 to 2008, the CDC supported a basic screening examination (BSE) in the National Health and Nutrition Examination Survey (NHANES), designed to maintain a minimal level of dental caries surveillance in the United States that could support the monitoring of five Healthy People 2010 objectives (3). The BSE was modeled on a similar screening tool developed by the American Association of State and Territorial Dental Directors to promote dental caries and sealant surveillance at the state and local levels (4). The two key elements of the BSE that facilitates its use in a limited resource, epidemiological setting, are protocols devised for subjectlevel screening instead of the typical tooth-level assessments and the employment of nondental professionals to conduct the screenings. Historically, meaningful and systematic periodontal disease surveillance has been difficult to maintain in in the United States (5). Developing and implementing a screening protocol for periodontal disease based on similar basic screening principles used for dental caries and sealants have been stymied with numerous challenges as well as concerns that the data would not be relevant to dental professionals or policymakers. The CDC began a “Periodontal Disease Surveillance Project” in collaboration with the American Academy of Periodontology (AAP) in 2003 to seek alternative population-based surveillance measures for periodontal diseases that could be integrated into existing surveillance mechanisms (6). The main objective of the project was to identify, test, and validate self-reported measures to assess the prevalence of periodontal diseases. Several notable outcomes that have emerged from this work include: a) the development of standard case definitions for surveillance of periodontitis (7); b) the development and evaluation selfreport measures for predicting prevalence of periodontitis (8); c) assessments of the accuracy of NHANES partial mouth periodontal examination protocols for determining
Oral health overview: NHANES 2009-2010
prevalence of periodontitis (9); and d) field testing of the full-mouth periodontal examination protocols for use in NHANES. Findings from the latter two outcomes provided the evidence to advocate for a full mouth periodontal examination to be conducted on NHANES to provide more valid estimates of periodontitis in the United States. The first look at periodontal disease prevalence based on the full-mouth periodontal examination and the performance of the periodontal self-reported measures recently are published elsewhere (10,11). Since 1999, NHANES has been operating as a continuous, annual survey that provides nationally representative data in 2-year cycles. Like previous national health examination surveys, participants are interviewed in their homes and then complete a health examination at a Mobile Examination Center (MEC). From 1999 to 2004, NHANES conducted a partial mouth periodontal examination that was similar to the protocols used in NHANES III (1988-1994), but with a slight enhancement made for 2001-2004 (i.e., the addition of a third facial measurement site) (12,13). Although NHANES was the data source for the Healthy People 2010 objective monitoring periodontal disease status, NHANES data collection for periodontal disease surveillance was discontinued after 2004. In NHANES 2009-2010, periodontal data collection was resumed using a full-mouth periodontal examination protocol, which concurred with the introduction of self-reported periodontal measures in the home interview. The aim of the study presented here is to describe the 20092010 NHANES oral health component, the actions taken to ensure data quality, and the results of analyses assessing data reliability. Information in this paper will be: a) critical for evaluating the validity of the full-mouth periodontal protocols used for the first time in a national health examination survey; b) useful for researchers by providing guidance in properly using the NHANES 2009-2010 data; and c) significant for policy discussions among the communities of interest.
Methods NHANES 2009-2010 overview NHANES is conducted by the CDC’s National Center for Health Statistics (NCHS) and uses a stratified, multistage probability sampling design to select participants. The sampling frame represents the civilian, noninstutionalized population living in the 50 states and the District of Columbia. Oversampling is performed to improve estimation and test for differences between selected subgroups. Changes to the oversampling protocol have periodically been made since 1999, but for NHANES 2009-2010, the survey oversampled all Hispanics, and non-Hispanic Blacks, persons aged 60 years and older, and low-income Whites. Informed consent is
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
249
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
Table 1 Sampling Design Characteristics for National Health and Nutrition Examination Survey (NHANES), 2009-2010 Characteristic
NHANES 2009-2010
Age of the target population Dental exam age Number of survey locations Eligible geographical area For sample Groups target for oversampling
From birth 3 years and older 30
All ages examined in the MEC All persons 3 years and older MEC examined All persons 3 years and older Oral health examined
50 states + DC Persons 60 years and older; all Hispanics; non-Hispanic Blacks; low-income Whites 10,253 9,211 7,189
DC, District of Columbia; MEC, mobile examination center.
obtained for all participants, and all data collection protocols are approved by the CDC/NCHS Ethics Review Board. Sample design characteristics for NHANES 2009-2010 are presented in Table 1.
NHANES 2009-10 oral health component The 2009-2010 oral health component included eight questions administered in a home interview and a physical evaluation conducted in a Mobile Examination Center (MEC). These eight questions were collectively designed to assess periodontal health (Appendix A) and were previously tested for internal validity and performance in estimating periodontitis (8). The eight oral health questions were administered only to adults aged 30 years and older by trained interviewers. Following completion of the home interview, participants received a MEC appointment to complete a number of clinical and laboratory assessments. For 2009-2010, the MEC oral health assessment was performed by trained dental hygienists possessing a state registration in at least one US jurisdiction. Other MEC personnel were trained as dental recorders. NHANES has two full-time MEC teams collecting data at the same time, and each team was assigned a dental hygienist as the primary dental examiner. Both primary dental examiners performed nearly 92 percent of all dental examinations undertaken in 2009-2010. All primary and back-up dental examiners were trained and calibrated by the survey reference examiner. These examinations took place in a designated room inside the MEC that includes a portable dental chair, light, and compressed air. The composition of NHANES oral health examinations includes multiple age-specific assessments. In 2009-2010, there was limited overlap of assessments for individuals in the broad age groups, with a basic tooth count being the only common assessment for both children and adults. Further250
more, participants aged 20-29 years received no oral health assessments. In 2009-2010, children and adolescents aged 3-19 years were eligible for the following assessments: a) a tooth count to identify the presence or absence of permanent and/or primary teeth, including retained dental root tips and dental implants; b) the BSE, which ascertained the presence of at least one tooth affected by untreated dental decay, a dental restoration, or a dental sealant; and c) a dental fluorosis screening of the upper six anterior teeth. The tooth count and BSE administered in 2009-2010 utilized the same examination protocols that were used in 20052008 and have been previously described (3). In brief, a dental lesion was considered to be “untreated” only if the carious lesion was considered to be cavitated, and a sealant was considered present even if part of the sealant was not visible. Third molars were excluded from the BSE. The dental fluorosis screening was based on criteria described by Dean in 1942 and used on NHANES 1999-2004 (12,13). Each maxillary anterior tooth was examined and assigned to one of six categories according to its degree of dental fluorosis, and only the six maxillary anterior teeth were classified. Adults aged 30 years and older were eligible for a tooth count and periodontal examination at the MEC. The tooth count assessment was identical to what was conducted on the children. In 2009-2010, a full-mouth periodontal examination was administered on adult participants who had at least one permanent tooth present. Gingival recession and pocket depth (PD) measures were made at six sites per tooth [the disto-facial (DF), mid-facial (BF), mesio-facial (MF), distolingual (DL), mid-lingual (BL), and mesio-lingual (ML) sites] using an HU-Friedy periodontal probe (Hu-Friedy Mfg. Co., Inc., Chicago, IL, USA) color coded and graduated at 2, 4, 6, 8, 10, and 12 mm. An algorithm in the data entry program calculated loss of attachment from the information on gingival recession and PD. Unlike previous NHANES periodontal assessments, all four quadrants were examined. The positioning of the periodontal probe for the facial sites (DF, BF, and MF) followed the 2003-2004 NHANES periodontal examination protocols (13). These same guidelines were used for the lingual measurements, and all measurements were rounded to the lowest whole millimeter.
Quality assurance Data collected in NHANES are directly entered into an electronic data management system by dental recorders, and automated data management utilities are also used that check entered data for out-of-range values to reduce errors. Additional measures used to ensure quality data include extensive training of staff and periodic evaluation of data collection. Dental examiners participated in a comprehensive training and calibration period followed by periodic monitoring and recalibration. During the training session, dental examiners
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
received instruction from the survey reference examiner that included a slide presentation on the study protocols, data entry, and assessment criteria. Demonstration examinations were conducted by the reference examiner, followed by standardization sessions where the reference examiner and examiner evaluated the same volunteers. Examiners were encouraged to ask questions regarding the assessment criteria, and feedback was provided by the reference examiner after each examination round to minimize differences in the application of criteria and coding. A preliminary calibration session followed standardization. Dental examiners conducted independent replicate examinations without discussion, and the data were analyzed to assess consistency between each examiner and the reference examiner. The training and preliminary calibration sessions were conducted over a 5-day period in the Washington, DC, metropolitan area. Follow-up replicate examinations were conducted at the MEC during normal field operations over a 3 to 4-day period. The collection of quality oral health data also was facilitated by the reference examiner visiting each dental examiner two to three times each year to observe data collection and to randomly replicate approximately 25-30 oral health examinations during each MEC visit. Data from these replicate exams were used to produce interrater reliability statistics to objectively evaluate examiner performance. Although dental examiners were aware of the interrater evaluations being conducted, the primary dental examiner and reference examiner were blinded to each other’s observations. Interexaminer statistics produced for this report include percent agreement, Kappa statistics, and interclass correlation coefficients (ICCs) using SAS software (version 9.2, SAS Institute Inc., Cary, NC, USA). All reliability statistics calculations followed similar procedures used for the NHANES 2003-2004 or the 2005-2008 oral health quality assurance reports to facilitate comparisons (3,13). Thus, Kappa statistics were calculated at the indicator level – as opposed to the tooth, surface, or site level – and are shown only if the number of observations was >1 for the concordant cells and observations with values “cannot assess” were excluded from the calculations. Many of these indicators represent the cornerstone of oral health surveillance in the United States and are incorporated into Healthy People objectives. Kappa statistics calculated for tooth count assessed four indicators: complete tooth loss (i.e., edentulism), retention of all third molars, having at least one retained root tip, and tooth retention, which was the total number of primary and permanent teeth present. Kappa statistics calculated for the BSE included having at least one tooth with dental caries, one tooth with a dental restoration, and one tooth with a dental sealant. For the evaluation of periodontal status, a number of different definitions and categories were used, including the CDC Periodontitis Workgroup (CDC/AAP) cases definitions, which have been recommended for periodontitis
Oral health overview: NHANES 2009-2010
surveillance (6). The CDC/AAP case definitions require information from two interproximal sites (DF, MF, ML, and/or DL) and are not dependent upon the presence of an adjacent tooth. In addition to calculating Kappa statistics for categorical periodontal status variables, interrater reliability for continuous variables was assessed by comparing ICCs generated from subject-level means (mm) for attachment loss (AL) and PD using measurements obtained from all six periodontal sites.
Results Table 2 shows the number of individuals participating in the home interview, MEC examination, and the oral health examination by various characteristics in NHANES Table 2 Number of Sampled Persons Aged 3-19 years and 30 years and Older with Interview, MEC, and Oral Health Exams by Selected Demographic Characteristics, National Health and Nutrition Examination Survey (NHANES) 2009-2010 NHANES 2009-2010 Characteristic Age (years) 3-11 12-19 30-44 45-54 55-64 65-74 75+ Sex Male Female Race and ethnicity Mexican American Other Hispanic Non-Hispanic White Non-Hispanic Black Other race Poverty status (FPL) Less than 100% 100%–199% 200% or higher Education Less than high school High school More than high school Smoking history Current smoker Former smoker Never smoker Total
HIQ
MEC
OHX
1,906 1,339 1,602 1,074 978 816 707
1,842 1,310 1,562 1,052 963 792 668
1,795 1,266 1,368 910 788 596 466
4,199 4,223
4,088 4,101
3,647 3,542
1,807 903 3,630 1,586 496
1,745 874 3,547 1,539 484
1,553 791 3,042 1,366 437
1,906 2,216 3,569
1,868 2,069 3,479
1,643 1,840 3,042
1,545 1,160 2,458
1,504 1,127 2,393
1,211 913 1,994
1,043 1,424 2,710 8,422
1,023 1,384 2,630 8,189
837 1,109 2,182 7,189
Notes: Education and smoking are only for sampled persons aged 30 years and older. FPL, federal poverty level; HIQ, home interview; MEC, mobile examination center; OHX, oral health examination.
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
251
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
2009-2010. Among the 8,189 sampled persons aged 3-19 years and 30 years and older who had a MEC exam, 7,189 persons had an oral health examination. Although there were a number of reasons why a person may have missed an oral health exam including refusals, the predominant reason for adults not completing the oral health exam was because of a self-report of a preexisting medical condition that required antibiotic prophylaxis (n = 295). The interrater reliability statistics for each dental examiner (A and B) for tooth count, BSE, and periodontal status are shown in Table 3. The Kappa statistics for the tooth count assessment ranged from 0.79 to 1.00, with a percent agreement ranging from 98 percent to 100 percent. For untreated caries, Kappa scores were 1.00 and 0.86, and percent agreement was 100 percent and 95 percent. Both examiners performed equally when identifying dental restorations (Kappa 0.82 and 0.83, percent agreement 86 percent–87 percent) and dental sealants (Kappa 0.71 both, percent agreement 86 percent–87 percent). For periodontal status assessments, incorporating fullmouth probing information from all six periodontal sites, both examiners performed equally (Kappa 0.70) when assessments were made using the CDC/AAP case definition for combined moderate–severe periodontitis. Although Kappa scores were lower when only mild periodontitis case definition was evaluated (0.41 and 0.36), percent agreement scores were relatively high (89.8 percent and 92.7 percent). When applying the case definition for periodontal disease used in the published Series 11 Report (14), interrater reliability was consistent for both examiners (Kappa 0.56 and 0.58, percent agreement 77.1 percent and 77.6 percent). Kappa scores for various thresholds of AL and PD ranged from 0.51 to 0.71. When evaluating interexaminer agreement using information collected from three periodontal sites (DF, BF, and MF) alone for comparability with the NHANES 2003-2004 periodontal examination protocols, Kappa scores for combined moderate–severe periodontitis ranged from 0.55 to 0.64. When assessing for periodontal disease using the criteria of one or more sites with >3 mm AL and >4 mm PD, the Kappa scores were 0.51 and 0.56 for Examiners A and B. Table 4 shows the mean values and intraclass correlation coefficients when continuous measures of periodontal status were used. For overall AL and PD across all periodontal sites, the interclass coefficients ranged from 0.80 to 0.88 and from 0.60 to 0.71 respectively. For measures of overall gingival recession (CJ mean – six sites), ICCs ranged from 0.91 to 0.96. For AL, ICC for facial measures ranged from 0.71 to 0.87, and for the lingual measures from 0.69 to 0.90. For PD, the ICC values were slightly lower than for AL. The ICCs for interproximal facial and lingual PD measures ranged from 0.63 to 0.81 and from 0.62 to 0.72, respectively. However, the ICCs for the mid-site PD measures ranged from 0.15 to 0.59 for both examiners, with Examiner B having lower ICC values for both mid-facial and mid-lingual PD measurements. When 252
using information collected from only the three facial periodontal sites for comparability with the NHANES 2003-2004 protocols to calculate the interclass coefficients, the ICCs for mean AL was 0.87 and 0.82, and mean PD was 0.72 and 0.61. Table 3 Interrater Reliability Statistics for Selected Oral Health Conditions, National Health and Nutrition Examination Survey (NHANES) 2009-2010 NHANES 2009-2010 #
A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B
Assessments Tooth count Edentulism Edentulism Tooth retention Tooth retention 1 or more retained root tips 1 or more retained root tips Have all 4 third molars Have all 4 third molars BSE assessment Has untreated caries Has untreated caries Has restorations Has restorations Has dental sealants Has dental sealants Periodontal status Moderate + Severe periodontitis Moderate + Severe periodontitis Mild periodontitis Mild periodontitis Periodontal disease 6 sites Periodontal disease 6 sites 1 site with > = 4 mm AL 1 site with > = 4 mm AL 1 site with > = 6 mm AL 1 site with > = 6 mm AL 1 site with > = 5 mm PD 1 site with > = 5 mm PD 1 site with > = 7 mm PD 1 site with > = 7 mm PD Moderate + Severe periodontitis 3 sites† Moderate + Severe periodontitis 3 sites† Periodontal disease 3 sites† Periodontal disease 3 sites†
n
R%
O%
Kappa
ASE
90 100 90 100 90 100 90 100
6.67 13.00 31.11 24.00 3.33 12.00 6.67 4.00
100.00 100.00 100.00 100.00 100.00 99.00 97.78 100.00
1.00 1.00 1.00 1.00 1.00 0.95 0.79 1.00
0.00 0.00 0.00 0.00 0.00 0.05 0.14 0.00
36 38 36 38 36 38
8.33 26.32 16.67 34.21 38.88 39.47
100.00 94.74 94.44 92.11 86.11 86.84
1.00 0.86 0.82 0.83 0.71 0.71
0.00 0.25 0.12 0.10 0.12 0.12
48
22.92
87.50
0.70
0.11
49
36.73
85.70
0.71
0.10
48 49 48 49 48 49 48 49 48 49 48 49 48
4.17 6.12 37.50 36.73 43.75 46.94 20.83 20.41 25.00 10.20 6.12 4.08 14.58
89.77 92.70 77.08 77.55 85.42 77.55 87.50 77.55 85.42 89.80 97.96 97.96 83.33
0.41 0.36 0.56 0.54 0.71 0.56 0.67 0.51 0.64 0.61 0.66 0.66 0.55
0.20 0.29 0.11 0.12 0.10 0.10 0.12 0.11 0.12 0.15 0.32 0.32 0.13
49
30.61
83.67
0.64
0.11
48 49
20.83 28.57
77.08 81.63
0.51 0.56
0.11 0.13
Notes: † For the combined mesio-, mid-, and disto-facial periodontal sites (NHANES 2003-2004 examined sites). n: Number of replicate exams. R%: Reference examiner percent of cases. O%: Observed percent agreement. #: Dental examiner identification. AL, attachment loss; ASE, asymptotic standard error (Kappa); BSE, basic screening examination; PD, pocket depth.
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
Oral health overview: NHANES 2009-2010
Table 4 Dental Examiner Interclass Correlation Coefficients (ICCs) for Selected Periodontal Measures, National Health and Nutrition Examination Survey (NHANES) 2009-2010 NHANES 2009-2010 Reference Examiner mean mean Bias SE ICC
#
Periodontal measures
n
A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B
AL mean – 6 sites* AL mean – 6 sites* PD mean – 6 sites* PD mean – 6 sites* CJ mean – 6 sites* CJ mean – 6 sites* AL mean – disto facial AL mean – disto facial AL mean – mid facial AL mean – mid facial AL mean – mesio facial AL mean – mesio facial AL mean – disto lingual AL mean – disto lingual AL mean – mid lingual AL Mean – mid lingual AL mean – mesio lingual AL mean – mesio lingual PD mean – disto facial PD mean – disto facial PD mean – mid facial PD mean – mid facial PD mean – mesio facial PD mean – mesio facial PD mean – disto lingual PD mean – disto lingual PD mean – mid lingual PD mean – mid lingual PD mean – mesio lingual PD mean – mesio lingual AL mean – 3 sites† AL mean – 3 sites† PD mean – 3 sites† PD mean – 3 sites†
48 1.03 49 1.12 48 1.19 49 1.13 48 0.17 49 −0.04 48 1.12 49 1.19 48 0.81 49 1.16 48 1.10 49 1.23 48 1.14 49 1.20 48 0.88 49 1.08 48 1.10 49 1.18 48 1.43 49 1.42 48 0.50 49 0.44 48 1.54 49 1.43 48 1.51 49 1.39 48 0.67 49 0.67 48 1.51 49 1.41 48 1.01 49 1.19 48 1.16 49 1.10
1.45 1.48 1.66 1.49 0.22 0.03 1.32 1.29 1.34 1.70 1.38 1.37 1.52 1.41 1.59 1.69 1.55 1.42 1.90 1.58 0.78 1.05 2.05 1.67 2.00 1.70 1.07 1.28 2.15 1.71 1.34 1.45 1.58 1.43
0.05 0.05 0.04 0.03 0.03 0.03 0.08 0.07 0.05 0.05 0.08 0.05 0.07 0.07 0.08 0.06 0.12 0.07 0.07 0.05 0.06 0.04 0.07 0.04 0.06 0.05 0.05 0.04 0.09 0.04 0.06 0.05 0.05 0.03
0.88 0.80 0.71 0.60 0.96 0.91 0.84 0.75 0.71 0.76 0.87 0.82 0.90 0.80 0.69 0.72 0.79 0.76 0.63 0.81 0.54 0.15 0.75 0.74 0.68 0.67 0.59 0.36 0.62 0.72 0.87 0.82 0.72 0.61
Notes: * Overall mean is for the combined mesio-, mid-, and disto-facial plus the mesio-, mid-, and disto-lingual periodontal sites. † Overall mean is for the combined mesio-, mid-, and disto-facial periodontal sites (NHANES 2003-2004 examined sites). Bias SE: Standard error of the difference between the dental examiner and the reference examiner. n: Number of replicate exams. #: Dental examiner identification. AL, attachment loss; PD, pocket depth; CJ, gingival recession.
Discussion In the United States, NHANES is the primary source of data for surveillance of dental diseases and oral health-related issues at the national level. Because the 2009-2010 NHANES
oral health component included a full-mouth periodontal examination, this paper reports on a broad range of examiner reliability statistics for periodontal measurements. This likely represents one of the most comprehensive reporting of interexaminer reliability findings pertaining to a periodontal assessment conducted as part of any large epidemiologic study to date. Additionally, this paper provides additional information on the reliability of the BSE which was used in NHANES 2005-2008 (3). The main difference between the 2005-2008 BSE and the 2009-2010 BSE was that the examiners were health technologists in the earlier survey period, whereas the examiners were dental hygienists in 2009-2010. This paper is part four of a series of methodology reports that covers 12 years of oral health data collection since the introduction of the continuous NHANES in 1999 (3,12,13). The content of this report is presented in a format designed to facilitate comparisons between the various survey periods to show examiner performance and ongoing efforts to ensure a high level of data quality across multiple years of oral health data collection in NHANES. For example, although a fullmouth periodontal examination was conducted in 20092010, we calculated partial mouth periodontal examination statistics to permit examiner performance comparisons to the NHANES 2003-2004, which was the last survey cycle to include a periodontal assessment. For this report, we focus on two important measures of examiner reliability: interclass correlation coefficients for continuous data and Kappa statistics for categorical data. It is well known that Kappa statistics corrects the agreement due to what would be expected by chance alone; thus, their values are lower compared with percent agreement calculations. Although researchers may use different standards in evaluating examiner agreement, we used guidelines proposed by Landis and Koch for interpreting Kappa scores to maintain consistency in interpretation with previous NHANES oral health data quality reports (15). To summarize: a Kappa statistic ≤0 is reflective of having “poor agreement,” >0 but ≤0.20 is “slight agreement,” 0.21-0.40 is “fair agreement,” 0.41-0.60 is “moderate agreement,” 0.610.80 is “substantial agreement,” and >0.80 is “almost perfect agreement.” In 2009-2010, examiner performance for the Tooth Count assessment was considered to be excellent. Interexaminer agreement was almost perfect for edentulism, tooth retention, and assessing for retained dental root tips. There was a difference in the measure of agreement between examiner A with the reference examiner and examiner B with the reference examiner regarding third molar assessments (substantial versus almost perfect). Agreement was considered almost perfect for two of the three BSE assessments during 20092010 for both examiners, given that the Kappa statistics ranged from 0.86 to 1.00 for untreated caries, and from 0.82 to 0.83 for dental restorations. For the remaining BSE
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
253
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
assessment, examiner agreement was considered substantial for dental sealants for both examiners with a Kappa score of 0.71. Both examiners had substantial agreement with the reference examiners in measuring moderate and severe periodontitis (CDC/AAP case definition) using six periodontal sites (Kappa scores were 0.70 to 0.71). Although both examiners had substantial agreement for PD measures with thresholds at 5 mm and 7 mm given that the Kappa scores ranged from 0.61 to 0.66, agreement was not as consistent for both examiners regarding AL thresholds at 4 mm and 6 mm. Agreement was considered to be substantial for Examiner A (Kappa scores of 0.71 and 0.67) and moderate for Examiner B (Kappa scores of 0.56 and 0.51). Overall, examiner consistency increased with severity of disease for both examiners and reliability was comparable for both examiners. Another set of reliability statistics we have presented in this report are ICCs. Although it has been suggested that a threshold of 0.75 or greater would represent excellent reliability (16), examiner bias (the mean difference between reference examiner and survey examiners) also is an important consideration. For the NHANES 2009-2010 periodontal data, the mean AL ICC statistic as measured for all six sites around all teeth was 0.80 and 0.88 for each examiner, indicating excellent overall reliability for AL. When evaluating the ICCs calculated for each of the six sites, the range was 0.69-0.90. Overall recession measure reliability data also were excellent, but overall PD reliability was lower. When evaluating the individual sites, examiner reliability for both dental hygienists was lower than desired, with the mid-facial and midlingual sites having the lowest ICC statistics ranging from 0.15 to 0.59. It is not clear why examiner reliability was lower for the mid-tooth sites given that intuitively probe placement should have been more challenging for interproximal sites, especially lingual interproximal sites. Likewise, examiner bias was higher for posterior teeth compared with anterior teeth. When considering the morphology of posterior teeth and supporting bone structure, it may be that probes were placed more anteriorly from the mid line and this placement was able to identify more diseased sites or the probe was angled more which allowed it to slip into a furcation. Common factors that usually contribute to measurement bias include inconsistent angulation, probe pressure, and measurement rounding. The periodontal probes are marked in 2 mm increments, and examiners are trained to round down to the nearest whole millimeter. It is clear, however, that measurement error was restricted to PD measurements and not recession measurements in the mid-tooth area. Although data reliability is less than optimal from these two mid-tooth sites, only information obtained from the interproximal sites (mesial and distal tooth sites) are used to derive the CDC/ AAP case definitions for periodontitis. 254
An important consideration when evaluating periodontal data reliability is the direction of the observed bias between the dental examiners and the reference examiner with regard to the periodontal measurements. Since NHANES began direct periodontal measurements with a periodontal probe in 1988, periodontal measurement data have suggested that dental examiners consistently overmeasures compared with the reference examiners. This measurement error has been systematic across reference examiners and survey periods. Unlike conducting other replicate oral health measures, for instance dental caries, repeating periodontal measurements are more difficult on study participants because of immediate changes in the soft tissue and subsequent bleeding from earlier probing as well as participant anxiety due to tenderness. When comparing the NHANES 2009-2010 examiner reliability results based on the NHANES 2003-2004 periodontal examination protocols, the calculated examiner agreement was lower for some periodontal measures. When periodontal disease was defined as having at least 3 mm of AL and 4 mm of PD at the same site, the Kappa was 0.64 with percent agreement between 92 percent and 93 percent in 2003-2004 (13) compared with 2009-2010 where the Kappa was 0.51 to 0.56 and the percent agreement was 77 percent and 82 percent. When evaluating continuous measures of periodontal disease, the ICCs for PD was 0.61 and 0.86 for 2003-2004 and 0.61 and 0.72 for 2009-2010. However, the ICCs for mean AL measures indicate greater comparability between 2003-2004 and 2009-2010. In 2003-2004, mean AL ICCs were 0.86 and 0.93 (12) compared with 0.82 and 0.87 in 2009-2010. Given that the differences in examiner reliability statistics are not large and the reference examiner remained the same, it appears that overall examiner performance could be considered comparable between 2003-2004 and 2009-2010 for the periodontal examination. Although NHANES 2-year data cycles are considered nationally representative and the overall assessments of oral health data reliability for NHANES 2009-2010 are considered very good, analysts should be cautious regarding interpretation of 2-year estimates from the continuous NHANES. In addition to sampling variations as a result of the survey design or methodology changes affecting data collection, the use of 2-year data cycles alone could be affected by insufficient sample size to calculate oral health estimates, which could produce statistically unreliable estimates. To ensure reliability of estimates, analysts should routinely evaluate whether the denominator count is >30 and the relative standard error is greater than 30 percent. For most oral health calculations, using 4 years of data for analyses will reduce the effect of sampling variation between the two 2-year periods and produce more accurate estimates. Given the continuous data collection environment of the current NHANES, numerous oral health assessments have
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
been modified or replaced since 1999. Beginning in NHANES 2011-2012, a comprehensive dental caries and sealant assessment, comparable with the methodology used during 1999-2004, has replaced the BSE data collection. Moreover, beginning with the 2011-2012 data collection cycle, NHANES has returned to using dentists as dental examiners. The oral health data collected by dental hygienists during NHANES 2009-2010 were the first oral health examination data collected in a national US examination survey to include a full-mouth periodontal examination. Given the overall rarity of a full-mouth periodontal examination conducted at any national level, these interexaminer findings provide unique insight into the strengths and limitations of monitoring data reliability in large-scale epidemiologic studies.
Acknowledgments The 2009-2010 NHANES oral health component was a funding and content collaborative effort between the NIH / National Institute of Dental and Craniofacial Research, the CDC / National Center for Health Promotion and Disease Prevention Division of Oral Health, and the CDC / National Center for Health Statistics. References 1. Lee LM, Teutsch SM, Thacker SB, St. Louis ME. Principles and practice of public health surveillance. 3rd ed. New York: Oxford University Press; 2010. 2. Beltran-Aguilar ED, Malvitz DM, Lockwood S, Rozier GR, Tomar SL. Oral health surveillance: past, present, and future challenges. J Public Health Dent. 2003;63:141-9. 3. Dye BA, Barker LK, Li X, Lewis BG, Beltran-Aguilar ED. Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2005-08. J Public Health Dent. 2011;71:54-61. 4. Association of State and Territorial Dental Directors. Basic screening surveys: an approach to monitoring community oral health. Columbus, OH: Association of State and Territorial Dental Directors; 1999.
Oral health overview: NHANES 2009-2010
5. Dye BA, Thornton-Evans G. A brief history of national surveillance efforts for periodontal disease in the US. J Periodontol. 2007;78:1373-9. 6. Eke PI, Page RC, Wei L, Thornton-Evans G, Genco RJ. Update of the case definitions for population-based surveillance of periodontitis. J Periodontol. 2012;83:1449-54. 7. Page RC, Eke PI. Case definitions for use in population-based surveillance of periodontitis. J Periodontol. 2007;78: 1387-99. 8. Eke PI, Dye BA. Assessment of self-reported measures for predicting population prevalence of periodontitis. J Periodontol. 2009;80:1371-9. 9. Eke PI, Thornton-Evans GO, Wei L, Borgnakke WS, Dye BA. Accuracy of NHANES periodontal examination protocols. J Dent Res. 2010;89:1208-13. 10. Eke PI, Dye BA, Wei L, Thornton-Evans GO, Genco RJ. Prevalence of periodontitis in adults in the United States: 2009 and 2010. J Dent Res. 2012;91:914-20. 11. Eke PI, Dye BA, Wei L, Slade GD, Thornton-Evans GO, Beck JD, Taylor GW, Borgnakke WS, Page RC, Genco RJ. Self-reported measures for surveillance of periodontitis. J Dent Res. 2013;92:1041-7. 12. Dye BA, Barker LK, Selwitz RH, Lewis BG, Wu T, Fryar CD, Ostchega Y, Beltran ED, Ley E. Overview and quality assurance for the National Health and Nutrition Examination Survey (NHANES) oral health component, 1999-2002. Community Dent Oral Epidemiol. 2007;35:140-51. 13. Dye BA, Nowjack-Raymer R, Barker LK, Nunn JH, Steele JG, Tan S, Lewis BG, Beltran ED. Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2003-2004. J Public Health Dent. 2008;68:218-26. 14. Dye BA, Tan S, Smith V, Lewis BG, Barker LK, Thornton-Evans G, Eke P, Beltran-Aguilar ED, Horowitz AM, Li CH. Trends in oral health Status – United States, 1988-1994 and 1999-2004. Vital Health Stat. 2007. 15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33: 159-74. 16. Kingman A, Albandar JM. Methodological aspects of epidemiological studies of periodontal diseases. Periodontol 2000. 2002;29:11-30.
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
255
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
Appendix A Periodontal health questions and valid responses Gum disease is a common problem with the mouth. People with gum disease might have swollen gums, receding gums, sore or infected gums, or loose teeth. Q1: {Do you/Does SP} think {you/s/he} might have gum disease? Yes/No/Refused/Don’t Know Q2: Overall, how would {you/SP} rate the health of {your/his/her} teeth and gums? Excellent/Very Good/Good/Fair/Poor/Refused/Don’t Know Q3: {Have you/Has SP} ever had treatment for gum disease such as scaling and root planing, sometimes called deep cleaning? Yes/No/Refused/Don’t Know Q4: {Have you/Has SP} ever had any teeth become loose on their own, without an injury? Yes/No/Refused/Don’t Know Q5: {Have you/Has SP} ever been told by a dental professional that {you/he/she} lost bone around {your/his/her} teeth? Yes/No/Refused/Don’t Know Q6: During the past 3 months, {have you/has SP} noticed a tooth that doesn’t look right? Yes/No/Refused/Don’t Know Q7: Aside from brushing {your/his/her} teeth with a toothbrush, in the last 7 days, how many days did {you/SP} use dental floss or any other device to clean between{your/his/her} teeth? Number of Days/Refused/Don’t Know Q8: Aside from brushing {your/his/her} teeth with a toothbrush, in the last 7 days, how many days did {you/SP} use mouthwash or other dental rinse product that {you use/he/she uses} to treat dental disease or dental problems? Number of Days/Refused/Don’t Know
256
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2009-2010 Bruce A. Dye, DDS, MPH1; Xianfen Li, MS2; Brenda G. Lewis, MPH1; Tim Iafolla, DDS, MPH3; Eugenio D. Beltran-Aguilar, DMD, MPH, MS, DrPH4; Paul I. Eke, PhD4 1 National Center for Health Statistics, Centers for Disease Control and Prevention, Hyattsville, MD, USA 2 Harris Corporation, Fairfax, VA, USA 3 National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD, USA 4 National Center for Chronic Disease Prevention and Health Promotion, Division of Oral Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
Keywords NHANES; oral health; data reliability; epidemiology; surveillance; periodontal disease; quality assurance; dental public health. Correspondence Dr. Bruce A. Dye, CDC/NCHS/NHANES Program, 3311 Toledo Road, RM 4416, Hyattsville, MD 20782. Tel.: 301-458-4199; Fax: 301-458-4029; e-mail: [email protected]. Bruce A. Dye and Brenda G. Lewis are with the National Center for Health Statistics, Centers for Disease Control and Prevention. Xianfen Li is with the Harris Corporation. Tim Iafolla is with the National Institutes of Health, National Institute of Dental and Craniofacial Research. Eugenio D. Beltran-Aguilar and Paul I. Eke are with the National Center for Chronic Disease Prevention and Health Promotion, Division of Oral Health, Centers for Disease Control and Prevention. Ethics Statement: All study participants gave informed consent in accordance with the Ethics Review Board and study ethic guidelines at the Centers for Disease Control and Prevention. The authors do not have any financial or other competing interests to declare.
Abstract Objective: In 2009-2010, the oral health component for the National Health and Nutrition Examination Survey (NHANES) focused on adult periodontal health and included a full mouth periodontal examination as well as a series of questions adminis during the home interview. During this period, intraoral assessments were conducted by dental hygienists. Methods: This report provides oral health content information and results of dental examiner reliability for data collected during NHANES 2009-2010 on 7,189 persons aged 3-19 years and 30 years and older representing the US civilian, noninstitutionalized population in these age groups. Results: For caries and dental sealant assessments, Kappa statistics ranged from 0.71 to 1.00. Kappa scores for moderate and severe periodontitis using the Centers for Disease Control and Prevention/American Academy of Periodontology case definition guidelines was 0.70, but were lower for other periodontal status definitions. When defining moderate or severe periodontitis based on the NHANES 2003-2004 study, protocols using data from only three facial periodontal sites, the Kappa scores were 0.64 and 0.55. Interclass correlation coefficients (ICCs) for mean attachment loss were 0.80 or higher for both examiners. Site-specific mean attachment loss ICCs were generally higher for interproximal measurements compared with mid-facial and mid-lingual measurements. Conclusion: Overall, the data reliability analyses conducted for 2009-2010 indicate an acceptable level of data quality and that examiner (dental hygienist) performance in this data collection cycle is similar to prior survey periods since the NHANES continuous survey began in 1999.
Received: 8/28/2012; accepted: 4/5/2014. doi: 10.1111/jphd.12056 Journal of Public Health Dentistry 74 (2014) 248–256
Introduction Public health surveillance is the ongoing collection, evaluation, and interpretation of systematically collected health data (1). However, there are some factors that affect the sur248
veillance of oral health diseases and related conditions that lead to the periodic model rather than the ongoing model for surveillance that is often desired (2). Some of these factors are related to the chronic nature of many oral diseases and the high levels of resources needed to have an ongoing
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
surveillance process. It is generally believed within the professional dental community that dental caries and periodontal diseases generally progress slowly and can have latent periods of low or no activity. Although both diseases require the presence of putative bacteria, other conditions must be present to facilitate disease development and progression. Consequently, disease status does not change as frequently in populations as a more typical infectious disease would. More importantly, the mechanism we rely on at the national level to assess for the prevalence of these two common oral diseases has been based on a physical evaluation, which has remained essentially unchanged since the 1960s. The Centers for Disease Control and Prevention (CDC) has been exploring alternate methodologies to ascertain oral health status that is clinically meaningful and scientifically based that can be used for public health surveillance when resources are limited, but information collection remains important for monitoring the public’s health. From 2005 to 2008, the CDC supported a basic screening examination (BSE) in the National Health and Nutrition Examination Survey (NHANES), designed to maintain a minimal level of dental caries surveillance in the United States that could support the monitoring of five Healthy People 2010 objectives (3). The BSE was modeled on a similar screening tool developed by the American Association of State and Territorial Dental Directors to promote dental caries and sealant surveillance at the state and local levels (4). The two key elements of the BSE that facilitates its use in a limited resource, epidemiological setting, are protocols devised for subjectlevel screening instead of the typical tooth-level assessments and the employment of nondental professionals to conduct the screenings. Historically, meaningful and systematic periodontal disease surveillance has been difficult to maintain in in the United States (5). Developing and implementing a screening protocol for periodontal disease based on similar basic screening principles used for dental caries and sealants have been stymied with numerous challenges as well as concerns that the data would not be relevant to dental professionals or policymakers. The CDC began a “Periodontal Disease Surveillance Project” in collaboration with the American Academy of Periodontology (AAP) in 2003 to seek alternative population-based surveillance measures for periodontal diseases that could be integrated into existing surveillance mechanisms (6). The main objective of the project was to identify, test, and validate self-reported measures to assess the prevalence of periodontal diseases. Several notable outcomes that have emerged from this work include: a) the development of standard case definitions for surveillance of periodontitis (7); b) the development and evaluation selfreport measures for predicting prevalence of periodontitis (8); c) assessments of the accuracy of NHANES partial mouth periodontal examination protocols for determining
Oral health overview: NHANES 2009-2010
prevalence of periodontitis (9); and d) field testing of the full-mouth periodontal examination protocols for use in NHANES. Findings from the latter two outcomes provided the evidence to advocate for a full mouth periodontal examination to be conducted on NHANES to provide more valid estimates of periodontitis in the United States. The first look at periodontal disease prevalence based on the full-mouth periodontal examination and the performance of the periodontal self-reported measures recently are published elsewhere (10,11). Since 1999, NHANES has been operating as a continuous, annual survey that provides nationally representative data in 2-year cycles. Like previous national health examination surveys, participants are interviewed in their homes and then complete a health examination at a Mobile Examination Center (MEC). From 1999 to 2004, NHANES conducted a partial mouth periodontal examination that was similar to the protocols used in NHANES III (1988-1994), but with a slight enhancement made for 2001-2004 (i.e., the addition of a third facial measurement site) (12,13). Although NHANES was the data source for the Healthy People 2010 objective monitoring periodontal disease status, NHANES data collection for periodontal disease surveillance was discontinued after 2004. In NHANES 2009-2010, periodontal data collection was resumed using a full-mouth periodontal examination protocol, which concurred with the introduction of self-reported periodontal measures in the home interview. The aim of the study presented here is to describe the 20092010 NHANES oral health component, the actions taken to ensure data quality, and the results of analyses assessing data reliability. Information in this paper will be: a) critical for evaluating the validity of the full-mouth periodontal protocols used for the first time in a national health examination survey; b) useful for researchers by providing guidance in properly using the NHANES 2009-2010 data; and c) significant for policy discussions among the communities of interest.
Methods NHANES 2009-2010 overview NHANES is conducted by the CDC’s National Center for Health Statistics (NCHS) and uses a stratified, multistage probability sampling design to select participants. The sampling frame represents the civilian, noninstutionalized population living in the 50 states and the District of Columbia. Oversampling is performed to improve estimation and test for differences between selected subgroups. Changes to the oversampling protocol have periodically been made since 1999, but for NHANES 2009-2010, the survey oversampled all Hispanics, and non-Hispanic Blacks, persons aged 60 years and older, and low-income Whites. Informed consent is
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
249
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
Table 1 Sampling Design Characteristics for National Health and Nutrition Examination Survey (NHANES), 2009-2010 Characteristic
NHANES 2009-2010
Age of the target population Dental exam age Number of survey locations Eligible geographical area For sample Groups target for oversampling
From birth 3 years and older 30
All ages examined in the MEC All persons 3 years and older MEC examined All persons 3 years and older Oral health examined
50 states + DC Persons 60 years and older; all Hispanics; non-Hispanic Blacks; low-income Whites 10,253 9,211 7,189
DC, District of Columbia; MEC, mobile examination center.
obtained for all participants, and all data collection protocols are approved by the CDC/NCHS Ethics Review Board. Sample design characteristics for NHANES 2009-2010 are presented in Table 1.
NHANES 2009-10 oral health component The 2009-2010 oral health component included eight questions administered in a home interview and a physical evaluation conducted in a Mobile Examination Center (MEC). These eight questions were collectively designed to assess periodontal health (Appendix A) and were previously tested for internal validity and performance in estimating periodontitis (8). The eight oral health questions were administered only to adults aged 30 years and older by trained interviewers. Following completion of the home interview, participants received a MEC appointment to complete a number of clinical and laboratory assessments. For 2009-2010, the MEC oral health assessment was performed by trained dental hygienists possessing a state registration in at least one US jurisdiction. Other MEC personnel were trained as dental recorders. NHANES has two full-time MEC teams collecting data at the same time, and each team was assigned a dental hygienist as the primary dental examiner. Both primary dental examiners performed nearly 92 percent of all dental examinations undertaken in 2009-2010. All primary and back-up dental examiners were trained and calibrated by the survey reference examiner. These examinations took place in a designated room inside the MEC that includes a portable dental chair, light, and compressed air. The composition of NHANES oral health examinations includes multiple age-specific assessments. In 2009-2010, there was limited overlap of assessments for individuals in the broad age groups, with a basic tooth count being the only common assessment for both children and adults. Further250
more, participants aged 20-29 years received no oral health assessments. In 2009-2010, children and adolescents aged 3-19 years were eligible for the following assessments: a) a tooth count to identify the presence or absence of permanent and/or primary teeth, including retained dental root tips and dental implants; b) the BSE, which ascertained the presence of at least one tooth affected by untreated dental decay, a dental restoration, or a dental sealant; and c) a dental fluorosis screening of the upper six anterior teeth. The tooth count and BSE administered in 2009-2010 utilized the same examination protocols that were used in 20052008 and have been previously described (3). In brief, a dental lesion was considered to be “untreated” only if the carious lesion was considered to be cavitated, and a sealant was considered present even if part of the sealant was not visible. Third molars were excluded from the BSE. The dental fluorosis screening was based on criteria described by Dean in 1942 and used on NHANES 1999-2004 (12,13). Each maxillary anterior tooth was examined and assigned to one of six categories according to its degree of dental fluorosis, and only the six maxillary anterior teeth were classified. Adults aged 30 years and older were eligible for a tooth count and periodontal examination at the MEC. The tooth count assessment was identical to what was conducted on the children. In 2009-2010, a full-mouth periodontal examination was administered on adult participants who had at least one permanent tooth present. Gingival recession and pocket depth (PD) measures were made at six sites per tooth [the disto-facial (DF), mid-facial (BF), mesio-facial (MF), distolingual (DL), mid-lingual (BL), and mesio-lingual (ML) sites] using an HU-Friedy periodontal probe (Hu-Friedy Mfg. Co., Inc., Chicago, IL, USA) color coded and graduated at 2, 4, 6, 8, 10, and 12 mm. An algorithm in the data entry program calculated loss of attachment from the information on gingival recession and PD. Unlike previous NHANES periodontal assessments, all four quadrants were examined. The positioning of the periodontal probe for the facial sites (DF, BF, and MF) followed the 2003-2004 NHANES periodontal examination protocols (13). These same guidelines were used for the lingual measurements, and all measurements were rounded to the lowest whole millimeter.
Quality assurance Data collected in NHANES are directly entered into an electronic data management system by dental recorders, and automated data management utilities are also used that check entered data for out-of-range values to reduce errors. Additional measures used to ensure quality data include extensive training of staff and periodic evaluation of data collection. Dental examiners participated in a comprehensive training and calibration period followed by periodic monitoring and recalibration. During the training session, dental examiners
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
received instruction from the survey reference examiner that included a slide presentation on the study protocols, data entry, and assessment criteria. Demonstration examinations were conducted by the reference examiner, followed by standardization sessions where the reference examiner and examiner evaluated the same volunteers. Examiners were encouraged to ask questions regarding the assessment criteria, and feedback was provided by the reference examiner after each examination round to minimize differences in the application of criteria and coding. A preliminary calibration session followed standardization. Dental examiners conducted independent replicate examinations without discussion, and the data were analyzed to assess consistency between each examiner and the reference examiner. The training and preliminary calibration sessions were conducted over a 5-day period in the Washington, DC, metropolitan area. Follow-up replicate examinations were conducted at the MEC during normal field operations over a 3 to 4-day period. The collection of quality oral health data also was facilitated by the reference examiner visiting each dental examiner two to three times each year to observe data collection and to randomly replicate approximately 25-30 oral health examinations during each MEC visit. Data from these replicate exams were used to produce interrater reliability statistics to objectively evaluate examiner performance. Although dental examiners were aware of the interrater evaluations being conducted, the primary dental examiner and reference examiner were blinded to each other’s observations. Interexaminer statistics produced for this report include percent agreement, Kappa statistics, and interclass correlation coefficients (ICCs) using SAS software (version 9.2, SAS Institute Inc., Cary, NC, USA). All reliability statistics calculations followed similar procedures used for the NHANES 2003-2004 or the 2005-2008 oral health quality assurance reports to facilitate comparisons (3,13). Thus, Kappa statistics were calculated at the indicator level – as opposed to the tooth, surface, or site level – and are shown only if the number of observations was >1 for the concordant cells and observations with values “cannot assess” were excluded from the calculations. Many of these indicators represent the cornerstone of oral health surveillance in the United States and are incorporated into Healthy People objectives. Kappa statistics calculated for tooth count assessed four indicators: complete tooth loss (i.e., edentulism), retention of all third molars, having at least one retained root tip, and tooth retention, which was the total number of primary and permanent teeth present. Kappa statistics calculated for the BSE included having at least one tooth with dental caries, one tooth with a dental restoration, and one tooth with a dental sealant. For the evaluation of periodontal status, a number of different definitions and categories were used, including the CDC Periodontitis Workgroup (CDC/AAP) cases definitions, which have been recommended for periodontitis
Oral health overview: NHANES 2009-2010
surveillance (6). The CDC/AAP case definitions require information from two interproximal sites (DF, MF, ML, and/or DL) and are not dependent upon the presence of an adjacent tooth. In addition to calculating Kappa statistics for categorical periodontal status variables, interrater reliability for continuous variables was assessed by comparing ICCs generated from subject-level means (mm) for attachment loss (AL) and PD using measurements obtained from all six periodontal sites.
Results Table 2 shows the number of individuals participating in the home interview, MEC examination, and the oral health examination by various characteristics in NHANES Table 2 Number of Sampled Persons Aged 3-19 years and 30 years and Older with Interview, MEC, and Oral Health Exams by Selected Demographic Characteristics, National Health and Nutrition Examination Survey (NHANES) 2009-2010 NHANES 2009-2010 Characteristic Age (years) 3-11 12-19 30-44 45-54 55-64 65-74 75+ Sex Male Female Race and ethnicity Mexican American Other Hispanic Non-Hispanic White Non-Hispanic Black Other race Poverty status (FPL) Less than 100% 100%–199% 200% or higher Education Less than high school High school More than high school Smoking history Current smoker Former smoker Never smoker Total
HIQ
MEC
OHX
1,906 1,339 1,602 1,074 978 816 707
1,842 1,310 1,562 1,052 963 792 668
1,795 1,266 1,368 910 788 596 466
4,199 4,223
4,088 4,101
3,647 3,542
1,807 903 3,630 1,586 496
1,745 874 3,547 1,539 484
1,553 791 3,042 1,366 437
1,906 2,216 3,569
1,868 2,069 3,479
1,643 1,840 3,042
1,545 1,160 2,458
1,504 1,127 2,393
1,211 913 1,994
1,043 1,424 2,710 8,422
1,023 1,384 2,630 8,189
837 1,109 2,182 7,189
Notes: Education and smoking are only for sampled persons aged 30 years and older. FPL, federal poverty level; HIQ, home interview; MEC, mobile examination center; OHX, oral health examination.
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
251
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
2009-2010. Among the 8,189 sampled persons aged 3-19 years and 30 years and older who had a MEC exam, 7,189 persons had an oral health examination. Although there were a number of reasons why a person may have missed an oral health exam including refusals, the predominant reason for adults not completing the oral health exam was because of a self-report of a preexisting medical condition that required antibiotic prophylaxis (n = 295). The interrater reliability statistics for each dental examiner (A and B) for tooth count, BSE, and periodontal status are shown in Table 3. The Kappa statistics for the tooth count assessment ranged from 0.79 to 1.00, with a percent agreement ranging from 98 percent to 100 percent. For untreated caries, Kappa scores were 1.00 and 0.86, and percent agreement was 100 percent and 95 percent. Both examiners performed equally when identifying dental restorations (Kappa 0.82 and 0.83, percent agreement 86 percent–87 percent) and dental sealants (Kappa 0.71 both, percent agreement 86 percent–87 percent). For periodontal status assessments, incorporating fullmouth probing information from all six periodontal sites, both examiners performed equally (Kappa 0.70) when assessments were made using the CDC/AAP case definition for combined moderate–severe periodontitis. Although Kappa scores were lower when only mild periodontitis case definition was evaluated (0.41 and 0.36), percent agreement scores were relatively high (89.8 percent and 92.7 percent). When applying the case definition for periodontal disease used in the published Series 11 Report (14), interrater reliability was consistent for both examiners (Kappa 0.56 and 0.58, percent agreement 77.1 percent and 77.6 percent). Kappa scores for various thresholds of AL and PD ranged from 0.51 to 0.71. When evaluating interexaminer agreement using information collected from three periodontal sites (DF, BF, and MF) alone for comparability with the NHANES 2003-2004 periodontal examination protocols, Kappa scores for combined moderate–severe periodontitis ranged from 0.55 to 0.64. When assessing for periodontal disease using the criteria of one or more sites with >3 mm AL and >4 mm PD, the Kappa scores were 0.51 and 0.56 for Examiners A and B. Table 4 shows the mean values and intraclass correlation coefficients when continuous measures of periodontal status were used. For overall AL and PD across all periodontal sites, the interclass coefficients ranged from 0.80 to 0.88 and from 0.60 to 0.71 respectively. For measures of overall gingival recession (CJ mean – six sites), ICCs ranged from 0.91 to 0.96. For AL, ICC for facial measures ranged from 0.71 to 0.87, and for the lingual measures from 0.69 to 0.90. For PD, the ICC values were slightly lower than for AL. The ICCs for interproximal facial and lingual PD measures ranged from 0.63 to 0.81 and from 0.62 to 0.72, respectively. However, the ICCs for the mid-site PD measures ranged from 0.15 to 0.59 for both examiners, with Examiner B having lower ICC values for both mid-facial and mid-lingual PD measurements. When 252
using information collected from only the three facial periodontal sites for comparability with the NHANES 2003-2004 protocols to calculate the interclass coefficients, the ICCs for mean AL was 0.87 and 0.82, and mean PD was 0.72 and 0.61. Table 3 Interrater Reliability Statistics for Selected Oral Health Conditions, National Health and Nutrition Examination Survey (NHANES) 2009-2010 NHANES 2009-2010 #
A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B
Assessments Tooth count Edentulism Edentulism Tooth retention Tooth retention 1 or more retained root tips 1 or more retained root tips Have all 4 third molars Have all 4 third molars BSE assessment Has untreated caries Has untreated caries Has restorations Has restorations Has dental sealants Has dental sealants Periodontal status Moderate + Severe periodontitis Moderate + Severe periodontitis Mild periodontitis Mild periodontitis Periodontal disease 6 sites Periodontal disease 6 sites 1 site with > = 4 mm AL 1 site with > = 4 mm AL 1 site with > = 6 mm AL 1 site with > = 6 mm AL 1 site with > = 5 mm PD 1 site with > = 5 mm PD 1 site with > = 7 mm PD 1 site with > = 7 mm PD Moderate + Severe periodontitis 3 sites† Moderate + Severe periodontitis 3 sites† Periodontal disease 3 sites† Periodontal disease 3 sites†
n
R%
O%
Kappa
ASE
90 100 90 100 90 100 90 100
6.67 13.00 31.11 24.00 3.33 12.00 6.67 4.00
100.00 100.00 100.00 100.00 100.00 99.00 97.78 100.00
1.00 1.00 1.00 1.00 1.00 0.95 0.79 1.00
0.00 0.00 0.00 0.00 0.00 0.05 0.14 0.00
36 38 36 38 36 38
8.33 26.32 16.67 34.21 38.88 39.47
100.00 94.74 94.44 92.11 86.11 86.84
1.00 0.86 0.82 0.83 0.71 0.71
0.00 0.25 0.12 0.10 0.12 0.12
48
22.92
87.50
0.70
0.11
49
36.73
85.70
0.71
0.10
48 49 48 49 48 49 48 49 48 49 48 49 48
4.17 6.12 37.50 36.73 43.75 46.94 20.83 20.41 25.00 10.20 6.12 4.08 14.58
89.77 92.70 77.08 77.55 85.42 77.55 87.50 77.55 85.42 89.80 97.96 97.96 83.33
0.41 0.36 0.56 0.54 0.71 0.56 0.67 0.51 0.64 0.61 0.66 0.66 0.55
0.20 0.29 0.11 0.12 0.10 0.10 0.12 0.11 0.12 0.15 0.32 0.32 0.13
49
30.61
83.67
0.64
0.11
48 49
20.83 28.57
77.08 81.63
0.51 0.56
0.11 0.13
Notes: † For the combined mesio-, mid-, and disto-facial periodontal sites (NHANES 2003-2004 examined sites). n: Number of replicate exams. R%: Reference examiner percent of cases. O%: Observed percent agreement. #: Dental examiner identification. AL, attachment loss; ASE, asymptotic standard error (Kappa); BSE, basic screening examination; PD, pocket depth.
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
Oral health overview: NHANES 2009-2010
Table 4 Dental Examiner Interclass Correlation Coefficients (ICCs) for Selected Periodontal Measures, National Health and Nutrition Examination Survey (NHANES) 2009-2010 NHANES 2009-2010 Reference Examiner mean mean Bias SE ICC
#
Periodontal measures
n
A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B A B
AL mean – 6 sites* AL mean – 6 sites* PD mean – 6 sites* PD mean – 6 sites* CJ mean – 6 sites* CJ mean – 6 sites* AL mean – disto facial AL mean – disto facial AL mean – mid facial AL mean – mid facial AL mean – mesio facial AL mean – mesio facial AL mean – disto lingual AL mean – disto lingual AL mean – mid lingual AL Mean – mid lingual AL mean – mesio lingual AL mean – mesio lingual PD mean – disto facial PD mean – disto facial PD mean – mid facial PD mean – mid facial PD mean – mesio facial PD mean – mesio facial PD mean – disto lingual PD mean – disto lingual PD mean – mid lingual PD mean – mid lingual PD mean – mesio lingual PD mean – mesio lingual AL mean – 3 sites† AL mean – 3 sites† PD mean – 3 sites† PD mean – 3 sites†
48 1.03 49 1.12 48 1.19 49 1.13 48 0.17 49 −0.04 48 1.12 49 1.19 48 0.81 49 1.16 48 1.10 49 1.23 48 1.14 49 1.20 48 0.88 49 1.08 48 1.10 49 1.18 48 1.43 49 1.42 48 0.50 49 0.44 48 1.54 49 1.43 48 1.51 49 1.39 48 0.67 49 0.67 48 1.51 49 1.41 48 1.01 49 1.19 48 1.16 49 1.10
1.45 1.48 1.66 1.49 0.22 0.03 1.32 1.29 1.34 1.70 1.38 1.37 1.52 1.41 1.59 1.69 1.55 1.42 1.90 1.58 0.78 1.05 2.05 1.67 2.00 1.70 1.07 1.28 2.15 1.71 1.34 1.45 1.58 1.43
0.05 0.05 0.04 0.03 0.03 0.03 0.08 0.07 0.05 0.05 0.08 0.05 0.07 0.07 0.08 0.06 0.12 0.07 0.07 0.05 0.06 0.04 0.07 0.04 0.06 0.05 0.05 0.04 0.09 0.04 0.06 0.05 0.05 0.03
0.88 0.80 0.71 0.60 0.96 0.91 0.84 0.75 0.71 0.76 0.87 0.82 0.90 0.80 0.69 0.72 0.79 0.76 0.63 0.81 0.54 0.15 0.75 0.74 0.68 0.67 0.59 0.36 0.62 0.72 0.87 0.82 0.72 0.61
Notes: * Overall mean is for the combined mesio-, mid-, and disto-facial plus the mesio-, mid-, and disto-lingual periodontal sites. † Overall mean is for the combined mesio-, mid-, and disto-facial periodontal sites (NHANES 2003-2004 examined sites). Bias SE: Standard error of the difference between the dental examiner and the reference examiner. n: Number of replicate exams. #: Dental examiner identification. AL, attachment loss; PD, pocket depth; CJ, gingival recession.
Discussion In the United States, NHANES is the primary source of data for surveillance of dental diseases and oral health-related issues at the national level. Because the 2009-2010 NHANES
oral health component included a full-mouth periodontal examination, this paper reports on a broad range of examiner reliability statistics for periodontal measurements. This likely represents one of the most comprehensive reporting of interexaminer reliability findings pertaining to a periodontal assessment conducted as part of any large epidemiologic study to date. Additionally, this paper provides additional information on the reliability of the BSE which was used in NHANES 2005-2008 (3). The main difference between the 2005-2008 BSE and the 2009-2010 BSE was that the examiners were health technologists in the earlier survey period, whereas the examiners were dental hygienists in 2009-2010. This paper is part four of a series of methodology reports that covers 12 years of oral health data collection since the introduction of the continuous NHANES in 1999 (3,12,13). The content of this report is presented in a format designed to facilitate comparisons between the various survey periods to show examiner performance and ongoing efforts to ensure a high level of data quality across multiple years of oral health data collection in NHANES. For example, although a fullmouth periodontal examination was conducted in 20092010, we calculated partial mouth periodontal examination statistics to permit examiner performance comparisons to the NHANES 2003-2004, which was the last survey cycle to include a periodontal assessment. For this report, we focus on two important measures of examiner reliability: interclass correlation coefficients for continuous data and Kappa statistics for categorical data. It is well known that Kappa statistics corrects the agreement due to what would be expected by chance alone; thus, their values are lower compared with percent agreement calculations. Although researchers may use different standards in evaluating examiner agreement, we used guidelines proposed by Landis and Koch for interpreting Kappa scores to maintain consistency in interpretation with previous NHANES oral health data quality reports (15). To summarize: a Kappa statistic ≤0 is reflective of having “poor agreement,” >0 but ≤0.20 is “slight agreement,” 0.21-0.40 is “fair agreement,” 0.41-0.60 is “moderate agreement,” 0.610.80 is “substantial agreement,” and >0.80 is “almost perfect agreement.” In 2009-2010, examiner performance for the Tooth Count assessment was considered to be excellent. Interexaminer agreement was almost perfect for edentulism, tooth retention, and assessing for retained dental root tips. There was a difference in the measure of agreement between examiner A with the reference examiner and examiner B with the reference examiner regarding third molar assessments (substantial versus almost perfect). Agreement was considered almost perfect for two of the three BSE assessments during 20092010 for both examiners, given that the Kappa statistics ranged from 0.86 to 1.00 for untreated caries, and from 0.82 to 0.83 for dental restorations. For the remaining BSE
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
253
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
assessment, examiner agreement was considered substantial for dental sealants for both examiners with a Kappa score of 0.71. Both examiners had substantial agreement with the reference examiners in measuring moderate and severe periodontitis (CDC/AAP case definition) using six periodontal sites (Kappa scores were 0.70 to 0.71). Although both examiners had substantial agreement for PD measures with thresholds at 5 mm and 7 mm given that the Kappa scores ranged from 0.61 to 0.66, agreement was not as consistent for both examiners regarding AL thresholds at 4 mm and 6 mm. Agreement was considered to be substantial for Examiner A (Kappa scores of 0.71 and 0.67) and moderate for Examiner B (Kappa scores of 0.56 and 0.51). Overall, examiner consistency increased with severity of disease for both examiners and reliability was comparable for both examiners. Another set of reliability statistics we have presented in this report are ICCs. Although it has been suggested that a threshold of 0.75 or greater would represent excellent reliability (16), examiner bias (the mean difference between reference examiner and survey examiners) also is an important consideration. For the NHANES 2009-2010 periodontal data, the mean AL ICC statistic as measured for all six sites around all teeth was 0.80 and 0.88 for each examiner, indicating excellent overall reliability for AL. When evaluating the ICCs calculated for each of the six sites, the range was 0.69-0.90. Overall recession measure reliability data also were excellent, but overall PD reliability was lower. When evaluating the individual sites, examiner reliability for both dental hygienists was lower than desired, with the mid-facial and midlingual sites having the lowest ICC statistics ranging from 0.15 to 0.59. It is not clear why examiner reliability was lower for the mid-tooth sites given that intuitively probe placement should have been more challenging for interproximal sites, especially lingual interproximal sites. Likewise, examiner bias was higher for posterior teeth compared with anterior teeth. When considering the morphology of posterior teeth and supporting bone structure, it may be that probes were placed more anteriorly from the mid line and this placement was able to identify more diseased sites or the probe was angled more which allowed it to slip into a furcation. Common factors that usually contribute to measurement bias include inconsistent angulation, probe pressure, and measurement rounding. The periodontal probes are marked in 2 mm increments, and examiners are trained to round down to the nearest whole millimeter. It is clear, however, that measurement error was restricted to PD measurements and not recession measurements in the mid-tooth area. Although data reliability is less than optimal from these two mid-tooth sites, only information obtained from the interproximal sites (mesial and distal tooth sites) are used to derive the CDC/ AAP case definitions for periodontitis. 254
An important consideration when evaluating periodontal data reliability is the direction of the observed bias between the dental examiners and the reference examiner with regard to the periodontal measurements. Since NHANES began direct periodontal measurements with a periodontal probe in 1988, periodontal measurement data have suggested that dental examiners consistently overmeasures compared with the reference examiners. This measurement error has been systematic across reference examiners and survey periods. Unlike conducting other replicate oral health measures, for instance dental caries, repeating periodontal measurements are more difficult on study participants because of immediate changes in the soft tissue and subsequent bleeding from earlier probing as well as participant anxiety due to tenderness. When comparing the NHANES 2009-2010 examiner reliability results based on the NHANES 2003-2004 periodontal examination protocols, the calculated examiner agreement was lower for some periodontal measures. When periodontal disease was defined as having at least 3 mm of AL and 4 mm of PD at the same site, the Kappa was 0.64 with percent agreement between 92 percent and 93 percent in 2003-2004 (13) compared with 2009-2010 where the Kappa was 0.51 to 0.56 and the percent agreement was 77 percent and 82 percent. When evaluating continuous measures of periodontal disease, the ICCs for PD was 0.61 and 0.86 for 2003-2004 and 0.61 and 0.72 for 2009-2010. However, the ICCs for mean AL measures indicate greater comparability between 2003-2004 and 2009-2010. In 2003-2004, mean AL ICCs were 0.86 and 0.93 (12) compared with 0.82 and 0.87 in 2009-2010. Given that the differences in examiner reliability statistics are not large and the reference examiner remained the same, it appears that overall examiner performance could be considered comparable between 2003-2004 and 2009-2010 for the periodontal examination. Although NHANES 2-year data cycles are considered nationally representative and the overall assessments of oral health data reliability for NHANES 2009-2010 are considered very good, analysts should be cautious regarding interpretation of 2-year estimates from the continuous NHANES. In addition to sampling variations as a result of the survey design or methodology changes affecting data collection, the use of 2-year data cycles alone could be affected by insufficient sample size to calculate oral health estimates, which could produce statistically unreliable estimates. To ensure reliability of estimates, analysts should routinely evaluate whether the denominator count is >30 and the relative standard error is greater than 30 percent. For most oral health calculations, using 4 years of data for analyses will reduce the effect of sampling variation between the two 2-year periods and produce more accurate estimates. Given the continuous data collection environment of the current NHANES, numerous oral health assessments have
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
B.A. Dye et al.
been modified or replaced since 1999. Beginning in NHANES 2011-2012, a comprehensive dental caries and sealant assessment, comparable with the methodology used during 1999-2004, has replaced the BSE data collection. Moreover, beginning with the 2011-2012 data collection cycle, NHANES has returned to using dentists as dental examiners. The oral health data collected by dental hygienists during NHANES 2009-2010 were the first oral health examination data collected in a national US examination survey to include a full-mouth periodontal examination. Given the overall rarity of a full-mouth periodontal examination conducted at any national level, these interexaminer findings provide unique insight into the strengths and limitations of monitoring data reliability in large-scale epidemiologic studies.
Acknowledgments The 2009-2010 NHANES oral health component was a funding and content collaborative effort between the NIH / National Institute of Dental and Craniofacial Research, the CDC / National Center for Health Promotion and Disease Prevention Division of Oral Health, and the CDC / National Center for Health Statistics. References 1. Lee LM, Teutsch SM, Thacker SB, St. Louis ME. Principles and practice of public health surveillance. 3rd ed. New York: Oxford University Press; 2010. 2. Beltran-Aguilar ED, Malvitz DM, Lockwood S, Rozier GR, Tomar SL. Oral health surveillance: past, present, and future challenges. J Public Health Dent. 2003;63:141-9. 3. Dye BA, Barker LK, Li X, Lewis BG, Beltran-Aguilar ED. Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2005-08. J Public Health Dent. 2011;71:54-61. 4. Association of State and Territorial Dental Directors. Basic screening surveys: an approach to monitoring community oral health. Columbus, OH: Association of State and Territorial Dental Directors; 1999.
Oral health overview: NHANES 2009-2010
5. Dye BA, Thornton-Evans G. A brief history of national surveillance efforts for periodontal disease in the US. J Periodontol. 2007;78:1373-9. 6. Eke PI, Page RC, Wei L, Thornton-Evans G, Genco RJ. Update of the case definitions for population-based surveillance of periodontitis. J Periodontol. 2012;83:1449-54. 7. Page RC, Eke PI. Case definitions for use in population-based surveillance of periodontitis. J Periodontol. 2007;78: 1387-99. 8. Eke PI, Dye BA. Assessment of self-reported measures for predicting population prevalence of periodontitis. J Periodontol. 2009;80:1371-9. 9. Eke PI, Thornton-Evans GO, Wei L, Borgnakke WS, Dye BA. Accuracy of NHANES periodontal examination protocols. J Dent Res. 2010;89:1208-13. 10. Eke PI, Dye BA, Wei L, Thornton-Evans GO, Genco RJ. Prevalence of periodontitis in adults in the United States: 2009 and 2010. J Dent Res. 2012;91:914-20. 11. Eke PI, Dye BA, Wei L, Slade GD, Thornton-Evans GO, Beck JD, Taylor GW, Borgnakke WS, Page RC, Genco RJ. Self-reported measures for surveillance of periodontitis. J Dent Res. 2013;92:1041-7. 12. Dye BA, Barker LK, Selwitz RH, Lewis BG, Wu T, Fryar CD, Ostchega Y, Beltran ED, Ley E. Overview and quality assurance for the National Health and Nutrition Examination Survey (NHANES) oral health component, 1999-2002. Community Dent Oral Epidemiol. 2007;35:140-51. 13. Dye BA, Nowjack-Raymer R, Barker LK, Nunn JH, Steele JG, Tan S, Lewis BG, Beltran ED. Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2003-2004. J Public Health Dent. 2008;68:218-26. 14. Dye BA, Tan S, Smith V, Lewis BG, Barker LK, Thornton-Evans G, Eke P, Beltran-Aguilar ED, Horowitz AM, Li CH. Trends in oral health Status – United States, 1988-1994 and 1999-2004. Vital Health Stat. 2007. 15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33: 159-74. 16. Kingman A, Albandar JM. Methodological aspects of epidemiological studies of periodontal diseases. Periodontol 2000. 2002;29:11-30.
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
255
Oral health overview: NHANES 2009-2010
B.A. Dye et al.
Appendix A Periodontal health questions and valid responses Gum disease is a common problem with the mouth. People with gum disease might have swollen gums, receding gums, sore or infected gums, or loose teeth. Q1: {Do you/Does SP} think {you/s/he} might have gum disease? Yes/No/Refused/Don’t Know Q2: Overall, how would {you/SP} rate the health of {your/his/her} teeth and gums? Excellent/Very Good/Good/Fair/Poor/Refused/Don’t Know Q3: {Have you/Has SP} ever had treatment for gum disease such as scaling and root planing, sometimes called deep cleaning? Yes/No/Refused/Don’t Know Q4: {Have you/Has SP} ever had any teeth become loose on their own, without an injury? Yes/No/Refused/Don’t Know Q5: {Have you/Has SP} ever been told by a dental professional that {you/he/she} lost bone around {your/his/her} teeth? Yes/No/Refused/Don’t Know Q6: During the past 3 months, {have you/has SP} noticed a tooth that doesn’t look right? Yes/No/Refused/Don’t Know Q7: Aside from brushing {your/his/her} teeth with a toothbrush, in the last 7 days, how many days did {you/SP} use dental floss or any other device to clean between{your/his/her} teeth? Number of Days/Refused/Don’t Know Q8: Aside from brushing {your/his/her} teeth with a toothbrush, in the last 7 days, how many days did {you/SP} use mouthwash or other dental rinse product that {you use/he/she uses} to treat dental disease or dental problems? Number of Days/Refused/Don’t Know
256
Published 2014. This article is a U.S. Government work and is in the public domain in the USA.
