The Completeness of AIDS Case Reporting, 1988: A Multisite Collaborative Surveillance Project
U*.-UU
A
Lisa Rosenblum, MD, MPH, James W. Buehler, MD, Meade W. Morgan, PhD, Samuel Costa, AL, Julia Hidalgo, ScD, Richard Holmes, MPH, Loren Lieb, MPH, Anne Shields, RN, MHA, and Bruce M. Whyte, MD
Introduction
Methods
Acquired immunodeficiency syndrome (AIDS) surveillance data are used to monitor trends, assess the future impact of the epidemic, detect new patterns of disease, facilitate the development and evaluation of prevention measures for human immunodeficiency virus (HIV) infection or AIDS (HIV/AIDS), and guide policy decisions related to allocation of resources. Given the important uses of these data, an estimate of the completeness of AIDS case reporting is needed. Completeness of reporting and, in turn, the accuracy of the estimates of the future scope of the epidemic have been the subject of public health, academic, and congressional inquiry.' Initial studies estimated reporting levels to be 85% or higher.2'3 Subsequent studies based on reviews of hospital records or death certificates in areas with active surveillance (e.g., routine contact of reporting facilities to obtain case reports) also estimated high levels of reporting.4However, several studies in areas without active surveillance or in outpatient settings found reporting levels to be lower (59% to 73%).5711-13 In addition, a sociologic survey suggested that persons with AIDS who were White or from the Mid-
Study Population and Design
west were underreported."4 However, these studies used diverse methods and varied in criteria used to select patient records for review, criteria for counting cases as residents of the reporting site, length of time to allow for routine reporting to occur, and procedures for case investigation. The purpose of this study was to estimate the completeness of AIDS reporting in multiple sites through the use of a standardized protocol.
The study was conducted by six state or county health departments (Alabama, Georgia, Los Angeles, Maryland, New Jersey, and Washington State). These sites have cumulatively reported 20% of persons with AIDS in the United States. 15 Sites were selected on the basis of objective review of applications received by the Centers for Disease Control. In each site, computerized medical care databases were used to identify records to link with AIDS surveillance (AIDS Reporting System). These databases included statewide hospital discharge records (Maryland, New Jersey, and Washington) and statewide Medicaid claims (Maryland and Washington); in areas where such state or countywide data were unavailable, discharge records were identified from selected hospitals (Alabama, Georgia, and Lisa Rosenblum, James W. Buehler, and Meade W. Morgan are with the Division of HIV/AIDS, National Center for Infectious Diseases, Centers for Disease Control, Atlanta, Ga. Samuel Costa is with the New Jersey Department of Health, Trenton. Julia Hidalgo is with the State Department of Health and Mental Hygiene, Baltimore, Md. Richard Holmes is with the Alabama Department of Health, Montgomery. Loren Lieb is with the Department of Health Services, Los Angeles, Calif. Anne Shields is with the State of Washington, Department of Health, Seattle. Bruce M. Whyte is with the Department of Human Resources, Atlanta, Ga. Requests for reprints should be sent to Lisa Rosenblum, MD, MPH, Division of HIV/ AIDS, Mail Stop E 47, Centers for Disease Control, Atlanta, GA 30333. This paper was submitted to the Journal October 29, 1991, and accepted with revisions April 7, 1992.
American Journal of Public Health 1495
Rosenbim et aL
and information gathered as part of routine AIDS surveillance on the standard case report fonn, such as date of diagnosis of AIDS, demographic characteristics, exposures to HIV, diseases indicative of AIDS, and laboratory data.15 For persons who were previously reported with AIDS, these AIDS Reporting System data items were also included in the study database.
Intemal Validation To further assess the usefulness of medical charts in providing adequate documentation of HIV/AIDS status, a sample of previously reported AIDS cases was selected for chart review. HIV/AIDS status, as indicated in the chart, was recorded.
Analysis
Los Angeles). Among the latter areas, health departments targeted large medical centers that were potentially major sources for AIDS case reporting. Persons eligible for inclusion in the study were adults and children (less than 13 years of age) who were identified by the above secondary data sources as being discharged from hospitals or seen as outpatients during 1988 and as having diagnoses suggesting or indicating HIV/AIDS. Such diagnoses included HIV/AIDS (International Classification ofDiseases, 9th version, clinical modification codes 042 through 044 and 795.8), immunodeficiency (codes 279.1 through 279.9, including code 279.19, which was designated for a diagnosis of AIDS during 1983 through 1986), and illnesses in the 1987 AIDS case definition (Table 1).16-19 Secondary data source records of eligible patients were linked with the AIDS Reporting System by local health departments. Patients reported to the AIDS Reporting System by September 1989 were considered as previously reported. For patients not reported to the AIDS Reporting System, secondary data source records were stratified into two groups based on diagnoses: (1) a diagnosis of HIV/AIDS and (2) a diagnosis of immunodeficiency or illness in the AIDS case definition without mention of HIV/AIDS. In each site, investigations to determine HIV/AIDS status 1496 American Journal of Public Health
were performed for all or a random sample
(n = 150)ofpersonsineachdiagnosisgroup in each secondary data source. A sample of records was reviewed in Maryland, New Jersey, and Washington; all records were reviewed in the remaining sites. From all secondary data sources and sites combined, a sample of 2150 records was selected for chart review (corresponding to a weighted estimate of 6600 persons). For each hospitalization in 1988, abstractors trained by each health department reviewed the face sheet, admission history and physical, discharge summary, and laboratory information. Reviews were extended to additional years and other sections of the chart, as needed, to ascertain HIV/AIDS status. If a patient had no hospitalization record (e.g., outpatient Medicaid enrollees), the outpatient record was reviewed. If HIV/AIDS status could not be determined from the chart alone, health care providers were consulted. On the basis of the investigation, HV/AIDS status was classified as AIDS (based on the 1987 case definition), HIV infection without AIDS, or no HIV infection.
Data Collection For persons not reported to the AIDS Reporting System who were determined to have AIDS or HIVwithout AIDS, data collected included diagnoses in each hospital discharge or outpatient record in 1988
Completeness of reporting was estimated for persons diagnosed with AIDS through 1988 and residing within the jurisdiction of the reporting health department. For sites using statewide hospital secondary data sources, estimates of completeness of reporting were provided for each state. For sites using both statewide hospital and Medicaid secondary data sources, completeness of reporting was estimated by using each database independently. In addition, the two secondary data sources were merged to estimate completeness of reporting for patients identified by one secondary data source but not by the other. For example, Medicaid inpatients were defined as Medicaid enrollees who were also identified by the hospital secondary data source. Medicaid outpatients were Medicaid enrollees who were not identified by the hospital secondary data source. For sites using selected hospitals, completeness of reporting was estimated collectively for persons diagnosedwith AIDS at the selected hospitals. Results represent completeness of reporting estimates for adults and children combined, except where stated otherwise. Completeness of reporting was calculated as the proportion of persons with AIDS detected by the secondary data source who were reported to the AIDS Reporting System [A/(A + B)] (Table 2).20To estimate completeness of reporting in each site, each person's record was weighted inversely proportional to the probability of selection into the stratified random sample descnrbed above. Likelihood based confidence bounds2l based on the binomial distribution were calculated for the completeness of reporting estimates. An estimate of completeness of reporting for the six sites collectivelywas calculated as the weighted average of the completeness of reporting in November 1992, Vol. 82, No. 1 1
Completeness of AIDS Reporig
each site; the weights were the estimated number of persons with AIDS in each site. Confidence bounds were calculated using weighted least squares.22 Two methods were used to adjust the estimate of completeness of reporting to account for the potential for underascertainment of cases due to inadequate documentation of HIV/AIDS status in the medical record or incomplete investigation during the study period. First, we determined the proportion of previously reported cases who were misclassified by study investigators as not having AIDS based on the chart reviews. We then adjusted the estimate of completeness of reporting based on the assumption that the same degree of misclassification observed for reported cases had occurred in persons not reported to the AIDS Reporting System. Second, previously unreported persons who had a case definition illness listed on the medical record without documentation of a definitive diagnosis or of HIV serology were considered to have "presumptive" AIDS. Because both methods had minimal effect on the level of reporting in each site (the reporting level decreased by 0% to 1% across sites), the unadjusted results are presented.
Results Overall Findings Of 11190 persons with hospital or Medicaid 1988 secondary data source records listing diagnoses that indicated or suggested HIV/AIDS in the six sites, 6600 were not listed in the AIDS Reporting System. Of these persons not previously reported as having AIDS, investigations by health departments detennined that 12% had AIDS (as defined by the 1987 case definition), 42% had HIV infection withNovember 1992, Vol. 82, No. 11
out AIDS, and 46% had no evidence of HIV infection (Table 3). Among persons having 1988 hospital records in the six sites, an estimated 7550 were identified as having HIV/AIDS (including persons identified from the above investigation and previously reported persons with AIDS). Of these, 5130 (68%) had AIDS, as defined by the 1987 case definition. Of the 5130 hospitalized persons with AIDS, an estimated 4500 were residents of the six sites and diagnosed through 1988.
Completeness ofAIDS Reponing Hospital secondary data source. Of the estimated 4500 hospitalized persons who were diagnosed with illnesses meeting the 1987 AIDS case definition through 1988 and who resided in the six sites, completeness of reporting was 92% overall (95% CI = 89%, 96%) and ranged from 89% to 97% across sites (Table 4). In all sites, collectively, reporting levels were 90% or higher among persons in each major demographic group and HIV exposure category and in persons diagnosed prior to 1988 and during 1988 (Table 5). Among previously reported cases, 67% were reported to local health departments within 2 months of the date of diagnosis and 82% within 5 months of diagnosis.
In several sites, differences among groups in reporting levels were observed. In Maryland, observed reporting levels were lower in children diagnosed at 1 to 12 years of age (4 of 7 [61%]; 95% CI = 24%, 80%) than in adults (503 of 521 [97%]; 95% CI = 93%, 98%). In other sites, there were even fewer pediatric records in the samples, and reporting levels were not estimated separately in children. Among adults in Maryland, reportingwas lower in those with blood or blood product exposure to HIV (19 of 24 [79%]; 95% CI = 53%, 88%) than in those with other modes of exposure (484 of 500 [97%]; 95% CI = 94%, 98%). In Los Angeles, reporting was lower in persons with no identified mode of HIV exposure (6 of 12 [50%o]; 95% CI = 30%, 64%) than in those with identified modes of exposure (443 of 475 [93%]; 95% CI = 91%, 95%). In Georgia, reporting was also lower in those without identified modes of HIV exposure (13 of 17 [77%]; 95% CI = 53%, 86%) than in those with identified modes of exposure (500 of 558 [90%]; 95% CI = 87%, 91%). In each site, no differences in reporting levels were observed between Whites and Blacks except in Washington, where reporting levels were lower in Blacks (25 of 28 [89%]; 95% CI = 68%, 90%) than in American Joumal of Public Health 1497
Rosenbiuna et al.
Whites (405 of 413 [98%]; 95% CI = 96%, 98%). Reporting levels were lower in persons diagnosed with AIDS during 1988 (90%) than in persons diagnosed with AIDS before 1988 (96%) (Table 5). However, the year of diagnosis did not affect any of the site-specific analyses of reporting levels by demographic characteristic or HIV exposure category. Medicaid secondary data source. In the two sites reviewing Medicaid records, reporting levels were high overall and higher in inpatients (414 of 417 [99%]; 95% CI = 95%, 99%) than outpatients (192 of 215 [90%o]; 95% CI = 79%, 90%o). Hospital and Medicaid secondary data sources. Persons with AIDS who had HIV/AIDS diagnoses listed on the computerized medical record were no more likely to be reported than those who had other listed diagnoses (e.g., illnesses in the AIDS case definition) without mention of HIV/ AIDS. Of persons with AIDS having hospital secondary data source records, completeness of reporting was 92% (4046 of 4386) among those whose records listed HIV/AIDS diagnoses compared with 93% (111 of 119) amongthosewhose records did not list HIV/AIDS diagnoses. Similarly, of persons with AIDS having Medicaid sec1498 American Journal of Public Health
ondary data source records, these proportions were 95% (123 of 129) and 96% (483 of 503).
wisuion Completeness of AIDS reporting was high overall (92%); range = 89% to 97%) in hospitalized persons diagnosed with AIDS (as defined by the 1987 case definition) through 1988 in six sites. Reporting levels were 90% or higher in adults, men, women, Whites, and Blacks; in men reporting sexual contact with men, in persons exposed to HIV through heterosexual contact, and in persons reporting injecting-rug use; and in persons diagnosed before 1988 and persons diagnosed during 1988. Among Medicaid enrollees, reporting was high in both inpatients and outpatients. The method used to estimate the completeness of reporting is based on the assumption that the AIDS Reporting System and the secondary data source are independent. Although it is unlikely that the data sources are independent, the magnitude and the direction of the interdependence of the databases are unknown. Most likely, the probability of identifying a case on the secondary data source is greater if the person had been
reported to the AIDS Reporting System than if the person had not been reported. If this is true, the method used will overestimate the completeness of reporting. Underascertainment of cases due to inadequate documentation of HIV/AIDS status in the medical record or incomplete investigation during the study may also result in overestimation of the level of reporting. However, the two analyses performed to assess the degree of misclassification of HIV/AIDS status as a result of incomplete investigation suggested that this was not a major problem. The "net" of diagnostic codes that was used to select records from the secondary data source to link with the AIDS Reporting System included diagnoses of HIV/AIDS and those suggesting HIV/ AIDS (e.g., immunodeficiency and illnesses in the AIDS case definition). The use of the net may result in overestimating reporting levels if unreported cases are less likely to have diagnoses in the net than are reported cases. However, our data suggest that this is not likely to be a major factor. There was no difference in reporting levels between persons with HIV/ AIDS diagnoses listed and persons with a case definition illness listed without mention of HIV/AIDS. To further evaluate the net ofcodes, 1989 hospital charts are being reviewed for persons 25 to 44 years of age (not reported to the AIDS Reporting System) with diagnoses not a part of our net (e.g., pneumonia with the organism unspecified and sepsis) and without mention of HIV/AIDS. On the basis of preliminary results from a sample of 100 records in two sites, no persons with HIV infection have been identified (A. Shields and B. Whyte, written communication, September 1991). Three reporting sites in our study used statewide population-based study designs; therefore, these results are generalizable to the state level. Although results from the other three sites using selected hospital facilities cannot be generalized to the reporting state or county, our findings are consistent across all sites. High levels of completeness of reporting among hospitalized persons were found in these sites despite differences in AIDS incidence, geographic location, and active surveillance procedures. Our findings are consistent with previous reports of high levels of completeness of reporting in sites with active suiveillance.4-1 Our results are also consistent with previous studies that found minimal differences in reporting levels by demographic characteristics or HIV exNovember 1992, Vol. 82, No. 11
Completeness of AIIDS Reporting
posure category in sites with active surveillance.2,5.10 Although approaches to surveillance vaxy among areas, all states and territories receive funds and technical assistance from the Centers for Disease Control for active AIDS surveillance. Our study focused on evaluation of completeness of AIDS reporting in hospitalized adults and adult Medicaid enrollees. Further efforts to evaluate AIDS surveillance should focus on children and outpatients in a variety of clinical settings. In Maryland, reporting levels were lower in children and persons with blood or blood product exposure to HIV than in others, although there were too few cases sampled in these groups to precisely estimate completeness of reporting. These findings are consistent with a study of reported AIDS cases in the United States showing that reporting was more delayed in children and persons with blood exposure to HIV (including persons with hemophilia) than in others.23 The lower levels of reporting in persons with no identified HIV exposure in two sites may reflect the recency of case ascertainment and the small number of persons with no identified exposure (6 of 12 and 4 of 17 cases were not reported in each site, respectively). If further investigation were to identify a mode of exposure in only a few of the newly identified cases, the level of reporting would rise markedly in persons with no known exposure. In summary, completeness of AIDS reporting among hospitalized persons was high, overall and in each demographic group and the leading HIV exposure groups; reporting was also timely. In most areas, hospitals have been the focus of active surveillance to identify persons with AIDS, which includes persons with late stage HIV infection and opportunistic infections or neoplasms (1987 case definition). However, a proposal has been made to expand the AIDS surveillance definition to include all persons with HIV infection with laboratory evidence of CD4+ T-lymphocyte counts less than 200/mm3, in addition to those meeting the current case definition.24 The proposed expansion would result in a substantial increase in those considered to meet surveillance criteria for AIDS.25 In addition to increasing the number of hospitalized persons meeting the definition, this change is likely to have an even greater impact on outpatients. The challenge will be for surveillance to achieve high levels of reporting among outpatients as well as inpatients. Knowledge of the completeness of AIDS case reporting is necessary for inNovember 1992, Vol. 82, No. 11
terpreting surveillance data, which are vital for providing information to assess the impact of the epidemic, evaluate prevention measures, and formulate public health policy. Review of secondary data sources of medical records may provide a valuable means of assessing completeness of reporting. Our findings from these six sites demonstrate that given the programmatic focus on a problem of grave public health importance, surveillance can provide accurate and timely information for public health planners in the setting of an evolving epidemic. []
Acknowledgments We thank Dr. Bernard Nahlen (Centers for Disease Control, Los Angeles, Calif); Dr. William Lafferty (Washington Health Department, Seattle, Wash); Joseph Bareta (Maryland Health Department); John Baldwin (New Jersey Department of Health); and the AIDS surveillance staff and chart abstractors from each project site for their contributions to the project. We also thank Thomas Skinner (Centers for Disease Control, Atlanta, Ga) for computer programmning support.
References 1. US General Accounting Office. AIDS Forecasting: Undercount of Cases and Lack of Key Data Weaken Exsting Estimates. Gaithersburg, Md: US General Accounting Office; 1989. GAO/PEMD-89-13. 2. Hardy AM, Starcher ET II, Morgan WM, et al. Review of death certificates to assess
completeness of AIDS reporting. Public
Healh Rep. 1987;102:386-391. 3. Chamberland ME, Castro KG, Haverkos HW, et al. Acquired immunodeficiency syndrome, New York City: evaluation of an active surveillance system. JAA4A. 1985;253:383-387. 4. Lafferty WE, Hopkins SG, Hoeny J, Harwell JD, Shoemaker PC, Kobayashi JM. Hospital charges for people with AIDS in Washington State: utilization of statewide hospital discharge data base. Am J Public Health. 1989;78:949-952. 5. Modessit SK, Hulman S, Fleming D. Evaluation of active versus passive AIDS surveillance in Oregon. Am J Public Health.
1990;80:463-464. 6. O'Keith G, Froelicher E, Garza A, Lieb L, Eastman C, Williams A. Using International Classification of Disease-9th revision (ICD-9) coded hospital discharge data to augment AIDS surveillance. In: Abstracts of the VI International Conference on AIDS; June 1990; San Francisco, Calif. Abstract Th.C.735. 7. Jones JL, Rion P, Hermann P, Kettinger L, Gamble WB. Improvement of AIDS case reporting, South Carolina. JAMA4. 1991; 265:356. Letter. 8. Mokotoff ED, Galbraith MA. Underreporting of AIDS. JAMA4. 1990;263:24492550. Letter. 9. Oddleifson S, Heinze-Lacey B, Makarewicz L, Day J, Seage G. Evaluation of
AIDS case reporting in Massachusetts. In: Abstracts of the International Conference on AIDS; June 1989; Montreal, Canada. June 1989. Abstract T.A.P.74. 10. Hindon RH, Thomas P, Nicholas A, Miller N, Schultz S. Evaluating completeness of New York City's case registxy. In: Abstracts of the V International Conference on AIDS; June 1989; Montreal, Canada. Abstract T.A.P.70. 11. Buehler JW, Berkelman RL, Stehr-Green JK. Completeness of AIDS surveillance. J AIDS. In press. 12. Conway GA, Colley-Niemeyer BD, Pursley C, et al. Underreporting of AIDS cases in South Carolina, 1986 and 1987. JAMA 1990;262:2859-2863. 13. Greenberg AE, Evans R, Bryan E, Lopez E, Thomas P. Completeness of AIDS case reporting and the spectrum of HIV-related disease among outpatients in New York City. In: Abstracts of the VI International Conference on AIDS; June 1990; San Francisco, Calif. Abstract Th.C.708. 14. Laumann EO, Gagnon JH, Michaels S, Michaels RT, Coleman JS. Monitoring the AIDS epidemic in the United States: a network approach. Science. 1989;244:11861189. 15. US Department of Health and Human Services. AIDS Public Information Data Set (documentation). Atlanta, Ga: Public Health Service, Centers for Disease Control; 1989. 16. World Health Organization. Intemational Classification of Diseases, Manual of the International Statistical Classiication of Diseases, Injuwes, and Causes of Death. 9th revision. Geneva Switzerland: World Health Organization; 1977. 17. Centers for Disease Control. Human immunodeficiency virus (HIV) infection codes: official authorized addendum 1CD9-CM (revision no. 1). MMWR 1987;36:124. 18. Converse M. ICD-9-CM notes: update on AIDS, 279.19. JAm Rec Assoc. 1983;54: 31. 19. Centers for Disease Control. Revision of the CDC surveillance case definition of ac-
quired immunodeficiency syndrome.
MMWR 1987;36:3-15.
20. Sekar CC, Deming WE. On a method of estimating birth and death rates and the extent of registration.Am StatAssoc J. 1949; 44:101-115. 21. Cox DR, Hinldey DV. 7heoretical Statistics. Cambridge, England: University Press; 1974:217. 22. Draper N, Smith H. Applied Regession Analysis. 2nd ed. New York: Wiley; 1980: 108-109. 23. Brookmeyer R, [Lao Jiangang. The analysis of delays in disease reporting: methods and results for the acquired immunodeficiency syndrome. Am J EpidemioL 1990; 132:355-365. 24. Centers for Disease Control. 1991 classification system for H1V infection and expanded case definition in adolescents and adults. MMWR In press. 25. Farizo K, BuehlerJ, Chamberland M, et al. Spectrum of disease in persons with human immunodeficiency virus infection in the United States. JAAL4. 1992;267:17981805.
American Journal of Public Health 1499
U*.-UU
A
Lisa Rosenblum, MD, MPH, James W. Buehler, MD, Meade W. Morgan, PhD, Samuel Costa, AL, Julia Hidalgo, ScD, Richard Holmes, MPH, Loren Lieb, MPH, Anne Shields, RN, MHA, and Bruce M. Whyte, MD
Introduction
Methods
Acquired immunodeficiency syndrome (AIDS) surveillance data are used to monitor trends, assess the future impact of the epidemic, detect new patterns of disease, facilitate the development and evaluation of prevention measures for human immunodeficiency virus (HIV) infection or AIDS (HIV/AIDS), and guide policy decisions related to allocation of resources. Given the important uses of these data, an estimate of the completeness of AIDS case reporting is needed. Completeness of reporting and, in turn, the accuracy of the estimates of the future scope of the epidemic have been the subject of public health, academic, and congressional inquiry.' Initial studies estimated reporting levels to be 85% or higher.2'3 Subsequent studies based on reviews of hospital records or death certificates in areas with active surveillance (e.g., routine contact of reporting facilities to obtain case reports) also estimated high levels of reporting.4However, several studies in areas without active surveillance or in outpatient settings found reporting levels to be lower (59% to 73%).5711-13 In addition, a sociologic survey suggested that persons with AIDS who were White or from the Mid-
Study Population and Design
west were underreported."4 However, these studies used diverse methods and varied in criteria used to select patient records for review, criteria for counting cases as residents of the reporting site, length of time to allow for routine reporting to occur, and procedures for case investigation. The purpose of this study was to estimate the completeness of AIDS reporting in multiple sites through the use of a standardized protocol.
The study was conducted by six state or county health departments (Alabama, Georgia, Los Angeles, Maryland, New Jersey, and Washington State). These sites have cumulatively reported 20% of persons with AIDS in the United States. 15 Sites were selected on the basis of objective review of applications received by the Centers for Disease Control. In each site, computerized medical care databases were used to identify records to link with AIDS surveillance (AIDS Reporting System). These databases included statewide hospital discharge records (Maryland, New Jersey, and Washington) and statewide Medicaid claims (Maryland and Washington); in areas where such state or countywide data were unavailable, discharge records were identified from selected hospitals (Alabama, Georgia, and Lisa Rosenblum, James W. Buehler, and Meade W. Morgan are with the Division of HIV/AIDS, National Center for Infectious Diseases, Centers for Disease Control, Atlanta, Ga. Samuel Costa is with the New Jersey Department of Health, Trenton. Julia Hidalgo is with the State Department of Health and Mental Hygiene, Baltimore, Md. Richard Holmes is with the Alabama Department of Health, Montgomery. Loren Lieb is with the Department of Health Services, Los Angeles, Calif. Anne Shields is with the State of Washington, Department of Health, Seattle. Bruce M. Whyte is with the Department of Human Resources, Atlanta, Ga. Requests for reprints should be sent to Lisa Rosenblum, MD, MPH, Division of HIV/ AIDS, Mail Stop E 47, Centers for Disease Control, Atlanta, GA 30333. This paper was submitted to the Journal October 29, 1991, and accepted with revisions April 7, 1992.
American Journal of Public Health 1495
Rosenbim et aL
and information gathered as part of routine AIDS surveillance on the standard case report fonn, such as date of diagnosis of AIDS, demographic characteristics, exposures to HIV, diseases indicative of AIDS, and laboratory data.15 For persons who were previously reported with AIDS, these AIDS Reporting System data items were also included in the study database.
Intemal Validation To further assess the usefulness of medical charts in providing adequate documentation of HIV/AIDS status, a sample of previously reported AIDS cases was selected for chart review. HIV/AIDS status, as indicated in the chart, was recorded.
Analysis
Los Angeles). Among the latter areas, health departments targeted large medical centers that were potentially major sources for AIDS case reporting. Persons eligible for inclusion in the study were adults and children (less than 13 years of age) who were identified by the above secondary data sources as being discharged from hospitals or seen as outpatients during 1988 and as having diagnoses suggesting or indicating HIV/AIDS. Such diagnoses included HIV/AIDS (International Classification ofDiseases, 9th version, clinical modification codes 042 through 044 and 795.8), immunodeficiency (codes 279.1 through 279.9, including code 279.19, which was designated for a diagnosis of AIDS during 1983 through 1986), and illnesses in the 1987 AIDS case definition (Table 1).16-19 Secondary data source records of eligible patients were linked with the AIDS Reporting System by local health departments. Patients reported to the AIDS Reporting System by September 1989 were considered as previously reported. For patients not reported to the AIDS Reporting System, secondary data source records were stratified into two groups based on diagnoses: (1) a diagnosis of HIV/AIDS and (2) a diagnosis of immunodeficiency or illness in the AIDS case definition without mention of HIV/AIDS. In each site, investigations to determine HIV/AIDS status 1496 American Journal of Public Health
were performed for all or a random sample
(n = 150)ofpersonsineachdiagnosisgroup in each secondary data source. A sample of records was reviewed in Maryland, New Jersey, and Washington; all records were reviewed in the remaining sites. From all secondary data sources and sites combined, a sample of 2150 records was selected for chart review (corresponding to a weighted estimate of 6600 persons). For each hospitalization in 1988, abstractors trained by each health department reviewed the face sheet, admission history and physical, discharge summary, and laboratory information. Reviews were extended to additional years and other sections of the chart, as needed, to ascertain HIV/AIDS status. If a patient had no hospitalization record (e.g., outpatient Medicaid enrollees), the outpatient record was reviewed. If HIV/AIDS status could not be determined from the chart alone, health care providers were consulted. On the basis of the investigation, HV/AIDS status was classified as AIDS (based on the 1987 case definition), HIV infection without AIDS, or no HIV infection.
Data Collection For persons not reported to the AIDS Reporting System who were determined to have AIDS or HIVwithout AIDS, data collected included diagnoses in each hospital discharge or outpatient record in 1988
Completeness of reporting was estimated for persons diagnosed with AIDS through 1988 and residing within the jurisdiction of the reporting health department. For sites using statewide hospital secondary data sources, estimates of completeness of reporting were provided for each state. For sites using both statewide hospital and Medicaid secondary data sources, completeness of reporting was estimated by using each database independently. In addition, the two secondary data sources were merged to estimate completeness of reporting for patients identified by one secondary data source but not by the other. For example, Medicaid inpatients were defined as Medicaid enrollees who were also identified by the hospital secondary data source. Medicaid outpatients were Medicaid enrollees who were not identified by the hospital secondary data source. For sites using selected hospitals, completeness of reporting was estimated collectively for persons diagnosedwith AIDS at the selected hospitals. Results represent completeness of reporting estimates for adults and children combined, except where stated otherwise. Completeness of reporting was calculated as the proportion of persons with AIDS detected by the secondary data source who were reported to the AIDS Reporting System [A/(A + B)] (Table 2).20To estimate completeness of reporting in each site, each person's record was weighted inversely proportional to the probability of selection into the stratified random sample descnrbed above. Likelihood based confidence bounds2l based on the binomial distribution were calculated for the completeness of reporting estimates. An estimate of completeness of reporting for the six sites collectivelywas calculated as the weighted average of the completeness of reporting in November 1992, Vol. 82, No. 1 1
Completeness of AIDS Reporig
each site; the weights were the estimated number of persons with AIDS in each site. Confidence bounds were calculated using weighted least squares.22 Two methods were used to adjust the estimate of completeness of reporting to account for the potential for underascertainment of cases due to inadequate documentation of HIV/AIDS status in the medical record or incomplete investigation during the study period. First, we determined the proportion of previously reported cases who were misclassified by study investigators as not having AIDS based on the chart reviews. We then adjusted the estimate of completeness of reporting based on the assumption that the same degree of misclassification observed for reported cases had occurred in persons not reported to the AIDS Reporting System. Second, previously unreported persons who had a case definition illness listed on the medical record without documentation of a definitive diagnosis or of HIV serology were considered to have "presumptive" AIDS. Because both methods had minimal effect on the level of reporting in each site (the reporting level decreased by 0% to 1% across sites), the unadjusted results are presented.
Results Overall Findings Of 11190 persons with hospital or Medicaid 1988 secondary data source records listing diagnoses that indicated or suggested HIV/AIDS in the six sites, 6600 were not listed in the AIDS Reporting System. Of these persons not previously reported as having AIDS, investigations by health departments detennined that 12% had AIDS (as defined by the 1987 case definition), 42% had HIV infection withNovember 1992, Vol. 82, No. 11
out AIDS, and 46% had no evidence of HIV infection (Table 3). Among persons having 1988 hospital records in the six sites, an estimated 7550 were identified as having HIV/AIDS (including persons identified from the above investigation and previously reported persons with AIDS). Of these, 5130 (68%) had AIDS, as defined by the 1987 case definition. Of the 5130 hospitalized persons with AIDS, an estimated 4500 were residents of the six sites and diagnosed through 1988.
Completeness ofAIDS Reponing Hospital secondary data source. Of the estimated 4500 hospitalized persons who were diagnosed with illnesses meeting the 1987 AIDS case definition through 1988 and who resided in the six sites, completeness of reporting was 92% overall (95% CI = 89%, 96%) and ranged from 89% to 97% across sites (Table 4). In all sites, collectively, reporting levels were 90% or higher among persons in each major demographic group and HIV exposure category and in persons diagnosed prior to 1988 and during 1988 (Table 5). Among previously reported cases, 67% were reported to local health departments within 2 months of the date of diagnosis and 82% within 5 months of diagnosis.
In several sites, differences among groups in reporting levels were observed. In Maryland, observed reporting levels were lower in children diagnosed at 1 to 12 years of age (4 of 7 [61%]; 95% CI = 24%, 80%) than in adults (503 of 521 [97%]; 95% CI = 93%, 98%). In other sites, there were even fewer pediatric records in the samples, and reporting levels were not estimated separately in children. Among adults in Maryland, reportingwas lower in those with blood or blood product exposure to HIV (19 of 24 [79%]; 95% CI = 53%, 88%) than in those with other modes of exposure (484 of 500 [97%]; 95% CI = 94%, 98%). In Los Angeles, reporting was lower in persons with no identified mode of HIV exposure (6 of 12 [50%o]; 95% CI = 30%, 64%) than in those with identified modes of exposure (443 of 475 [93%]; 95% CI = 91%, 95%). In Georgia, reporting was also lower in those without identified modes of HIV exposure (13 of 17 [77%]; 95% CI = 53%, 86%) than in those with identified modes of exposure (500 of 558 [90%]; 95% CI = 87%, 91%). In each site, no differences in reporting levels were observed between Whites and Blacks except in Washington, where reporting levels were lower in Blacks (25 of 28 [89%]; 95% CI = 68%, 90%) than in American Joumal of Public Health 1497
Rosenbiuna et al.
Whites (405 of 413 [98%]; 95% CI = 96%, 98%). Reporting levels were lower in persons diagnosed with AIDS during 1988 (90%) than in persons diagnosed with AIDS before 1988 (96%) (Table 5). However, the year of diagnosis did not affect any of the site-specific analyses of reporting levels by demographic characteristic or HIV exposure category. Medicaid secondary data source. In the two sites reviewing Medicaid records, reporting levels were high overall and higher in inpatients (414 of 417 [99%]; 95% CI = 95%, 99%) than outpatients (192 of 215 [90%o]; 95% CI = 79%, 90%o). Hospital and Medicaid secondary data sources. Persons with AIDS who had HIV/AIDS diagnoses listed on the computerized medical record were no more likely to be reported than those who had other listed diagnoses (e.g., illnesses in the AIDS case definition) without mention of HIV/ AIDS. Of persons with AIDS having hospital secondary data source records, completeness of reporting was 92% (4046 of 4386) among those whose records listed HIV/AIDS diagnoses compared with 93% (111 of 119) amongthosewhose records did not list HIV/AIDS diagnoses. Similarly, of persons with AIDS having Medicaid sec1498 American Journal of Public Health
ondary data source records, these proportions were 95% (123 of 129) and 96% (483 of 503).
wisuion Completeness of AIDS reporting was high overall (92%); range = 89% to 97%) in hospitalized persons diagnosed with AIDS (as defined by the 1987 case definition) through 1988 in six sites. Reporting levels were 90% or higher in adults, men, women, Whites, and Blacks; in men reporting sexual contact with men, in persons exposed to HIV through heterosexual contact, and in persons reporting injecting-rug use; and in persons diagnosed before 1988 and persons diagnosed during 1988. Among Medicaid enrollees, reporting was high in both inpatients and outpatients. The method used to estimate the completeness of reporting is based on the assumption that the AIDS Reporting System and the secondary data source are independent. Although it is unlikely that the data sources are independent, the magnitude and the direction of the interdependence of the databases are unknown. Most likely, the probability of identifying a case on the secondary data source is greater if the person had been
reported to the AIDS Reporting System than if the person had not been reported. If this is true, the method used will overestimate the completeness of reporting. Underascertainment of cases due to inadequate documentation of HIV/AIDS status in the medical record or incomplete investigation during the study may also result in overestimation of the level of reporting. However, the two analyses performed to assess the degree of misclassification of HIV/AIDS status as a result of incomplete investigation suggested that this was not a major problem. The "net" of diagnostic codes that was used to select records from the secondary data source to link with the AIDS Reporting System included diagnoses of HIV/AIDS and those suggesting HIV/ AIDS (e.g., immunodeficiency and illnesses in the AIDS case definition). The use of the net may result in overestimating reporting levels if unreported cases are less likely to have diagnoses in the net than are reported cases. However, our data suggest that this is not likely to be a major factor. There was no difference in reporting levels between persons with HIV/ AIDS diagnoses listed and persons with a case definition illness listed without mention of HIV/AIDS. To further evaluate the net ofcodes, 1989 hospital charts are being reviewed for persons 25 to 44 years of age (not reported to the AIDS Reporting System) with diagnoses not a part of our net (e.g., pneumonia with the organism unspecified and sepsis) and without mention of HIV/AIDS. On the basis of preliminary results from a sample of 100 records in two sites, no persons with HIV infection have been identified (A. Shields and B. Whyte, written communication, September 1991). Three reporting sites in our study used statewide population-based study designs; therefore, these results are generalizable to the state level. Although results from the other three sites using selected hospital facilities cannot be generalized to the reporting state or county, our findings are consistent across all sites. High levels of completeness of reporting among hospitalized persons were found in these sites despite differences in AIDS incidence, geographic location, and active surveillance procedures. Our findings are consistent with previous reports of high levels of completeness of reporting in sites with active suiveillance.4-1 Our results are also consistent with previous studies that found minimal differences in reporting levels by demographic characteristics or HIV exNovember 1992, Vol. 82, No. 11
Completeness of AIIDS Reporting
posure category in sites with active surveillance.2,5.10 Although approaches to surveillance vaxy among areas, all states and territories receive funds and technical assistance from the Centers for Disease Control for active AIDS surveillance. Our study focused on evaluation of completeness of AIDS reporting in hospitalized adults and adult Medicaid enrollees. Further efforts to evaluate AIDS surveillance should focus on children and outpatients in a variety of clinical settings. In Maryland, reporting levels were lower in children and persons with blood or blood product exposure to HIV than in others, although there were too few cases sampled in these groups to precisely estimate completeness of reporting. These findings are consistent with a study of reported AIDS cases in the United States showing that reporting was more delayed in children and persons with blood exposure to HIV (including persons with hemophilia) than in others.23 The lower levels of reporting in persons with no identified HIV exposure in two sites may reflect the recency of case ascertainment and the small number of persons with no identified exposure (6 of 12 and 4 of 17 cases were not reported in each site, respectively). If further investigation were to identify a mode of exposure in only a few of the newly identified cases, the level of reporting would rise markedly in persons with no known exposure. In summary, completeness of AIDS reporting among hospitalized persons was high, overall and in each demographic group and the leading HIV exposure groups; reporting was also timely. In most areas, hospitals have been the focus of active surveillance to identify persons with AIDS, which includes persons with late stage HIV infection and opportunistic infections or neoplasms (1987 case definition). However, a proposal has been made to expand the AIDS surveillance definition to include all persons with HIV infection with laboratory evidence of CD4+ T-lymphocyte counts less than 200/mm3, in addition to those meeting the current case definition.24 The proposed expansion would result in a substantial increase in those considered to meet surveillance criteria for AIDS.25 In addition to increasing the number of hospitalized persons meeting the definition, this change is likely to have an even greater impact on outpatients. The challenge will be for surveillance to achieve high levels of reporting among outpatients as well as inpatients. Knowledge of the completeness of AIDS case reporting is necessary for inNovember 1992, Vol. 82, No. 11
terpreting surveillance data, which are vital for providing information to assess the impact of the epidemic, evaluate prevention measures, and formulate public health policy. Review of secondary data sources of medical records may provide a valuable means of assessing completeness of reporting. Our findings from these six sites demonstrate that given the programmatic focus on a problem of grave public health importance, surveillance can provide accurate and timely information for public health planners in the setting of an evolving epidemic. []
Acknowledgments We thank Dr. Bernard Nahlen (Centers for Disease Control, Los Angeles, Calif); Dr. William Lafferty (Washington Health Department, Seattle, Wash); Joseph Bareta (Maryland Health Department); John Baldwin (New Jersey Department of Health); and the AIDS surveillance staff and chart abstractors from each project site for their contributions to the project. We also thank Thomas Skinner (Centers for Disease Control, Atlanta, Ga) for computer programmning support.
References 1. US General Accounting Office. AIDS Forecasting: Undercount of Cases and Lack of Key Data Weaken Exsting Estimates. Gaithersburg, Md: US General Accounting Office; 1989. GAO/PEMD-89-13. 2. Hardy AM, Starcher ET II, Morgan WM, et al. Review of death certificates to assess
completeness of AIDS reporting. Public
Healh Rep. 1987;102:386-391. 3. Chamberland ME, Castro KG, Haverkos HW, et al. Acquired immunodeficiency syndrome, New York City: evaluation of an active surveillance system. JAA4A. 1985;253:383-387. 4. Lafferty WE, Hopkins SG, Hoeny J, Harwell JD, Shoemaker PC, Kobayashi JM. Hospital charges for people with AIDS in Washington State: utilization of statewide hospital discharge data base. Am J Public Health. 1989;78:949-952. 5. Modessit SK, Hulman S, Fleming D. Evaluation of active versus passive AIDS surveillance in Oregon. Am J Public Health.
1990;80:463-464. 6. O'Keith G, Froelicher E, Garza A, Lieb L, Eastman C, Williams A. Using International Classification of Disease-9th revision (ICD-9) coded hospital discharge data to augment AIDS surveillance. In: Abstracts of the VI International Conference on AIDS; June 1990; San Francisco, Calif. Abstract Th.C.735. 7. Jones JL, Rion P, Hermann P, Kettinger L, Gamble WB. Improvement of AIDS case reporting, South Carolina. JAMA4. 1991; 265:356. Letter. 8. Mokotoff ED, Galbraith MA. Underreporting of AIDS. JAMA4. 1990;263:24492550. Letter. 9. Oddleifson S, Heinze-Lacey B, Makarewicz L, Day J, Seage G. Evaluation of
AIDS case reporting in Massachusetts. In: Abstracts of the International Conference on AIDS; June 1989; Montreal, Canada. June 1989. Abstract T.A.P.74. 10. Hindon RH, Thomas P, Nicholas A, Miller N, Schultz S. Evaluating completeness of New York City's case registxy. In: Abstracts of the V International Conference on AIDS; June 1989; Montreal, Canada. Abstract T.A.P.70. 11. Buehler JW, Berkelman RL, Stehr-Green JK. Completeness of AIDS surveillance. J AIDS. In press. 12. Conway GA, Colley-Niemeyer BD, Pursley C, et al. Underreporting of AIDS cases in South Carolina, 1986 and 1987. JAMA 1990;262:2859-2863. 13. Greenberg AE, Evans R, Bryan E, Lopez E, Thomas P. Completeness of AIDS case reporting and the spectrum of HIV-related disease among outpatients in New York City. In: Abstracts of the VI International Conference on AIDS; June 1990; San Francisco, Calif. Abstract Th.C.708. 14. Laumann EO, Gagnon JH, Michaels S, Michaels RT, Coleman JS. Monitoring the AIDS epidemic in the United States: a network approach. Science. 1989;244:11861189. 15. US Department of Health and Human Services. AIDS Public Information Data Set (documentation). Atlanta, Ga: Public Health Service, Centers for Disease Control; 1989. 16. World Health Organization. Intemational Classification of Diseases, Manual of the International Statistical Classiication of Diseases, Injuwes, and Causes of Death. 9th revision. Geneva Switzerland: World Health Organization; 1977. 17. Centers for Disease Control. Human immunodeficiency virus (HIV) infection codes: official authorized addendum 1CD9-CM (revision no. 1). MMWR 1987;36:124. 18. Converse M. ICD-9-CM notes: update on AIDS, 279.19. JAm Rec Assoc. 1983;54: 31. 19. Centers for Disease Control. Revision of the CDC surveillance case definition of ac-
quired immunodeficiency syndrome.
MMWR 1987;36:3-15.
20. Sekar CC, Deming WE. On a method of estimating birth and death rates and the extent of registration.Am StatAssoc J. 1949; 44:101-115. 21. Cox DR, Hinldey DV. 7heoretical Statistics. Cambridge, England: University Press; 1974:217. 22. Draper N, Smith H. Applied Regession Analysis. 2nd ed. New York: Wiley; 1980: 108-109. 23. Brookmeyer R, [Lao Jiangang. The analysis of delays in disease reporting: methods and results for the acquired immunodeficiency syndrome. Am J EpidemioL 1990; 132:355-365. 24. Centers for Disease Control. 1991 classification system for H1V infection and expanded case definition in adolescents and adults. MMWR In press. 25. Farizo K, BuehlerJ, Chamberland M, et al. Spectrum of disease in persons with human immunodeficiency virus infection in the United States. JAAL4. 1992;267:17981805.
American Journal of Public Health 1499
