Opinion
EDITORIAL
Accuracy in Reporting Past Psychiatric Symptoms The Role of Cross-sectional Studies in Psychiatric Research Wilson M. Compton, MD, MPE; Marsha F. Lopez, PhD, MHS
Cross-sectional studies remain a major source of data for research, practice, and policy, despite well-known limitations of this approach in assessing psychopathology. Such studies are especially valuable when the intent is to identify current or recent cases and in repeated Related article page 273 studies where trends in rates can be examined. Yet, how well do they identify psychiatric symptoms and syndromes from the past? This is not a trivial question because the accuracy of such classification is important for association studies and, in particular, for clinical neuroscience and genetic studies where the accuracy of phenotypic classification is essential to determining the relationship of predictors and purported outcomes. Taken at face value, Takayanagi et al1 demonstrate that cross-sectional studies are much worse than longitudinal in identifying a history of psychiatric disorders. The authors find marked underestimates in lifetime prevalence estimates from a single cross-sectional study, and this lack of agreement with psychiatric symptom reporting across the waves of interviews is remarkable. Among those who received a diagnosis in a prior wave, 92% of those with obsessive-compulsive disorder and about two-thirds of those with major depressive disorder, panic disorder, and substance abuse or dependence (both alcohol and other drugs) did not report diagnostic symptoms sufficient for a diagnosis at the fourth interview. Also, somewhat surprisingly, recent receipt of mental health services made little difference in the accuracy of reporting of lifetime symptoms. Undoubtedly, a longitudinal study design has many advantages for inquiry: clear separation of risk and outcome variables, repeated measurement of psychopathology to identify acute/temporary and chronic symptom patterns, and less reliance on long-term memory for reports. Similarly, assessment that includes multiple distinct informants (typical in child psychiatry where parents and teachers are routinely queried) can improve diagnostic coverage.2 The key issue for researchers is to determine when this additional information is worth the extra effort (and time) to acquire. A secondary and intriguing question addressed by Takayanagi and colleagues is whether there is something unique about mental illnesses that makes them particularly prone to poor recall compared with identifying physical ailments. Their results suggest that physical conditions are much more consistently reported than mental illnesses. However, absent from the conclusions by Takayanagi and colleagues regarding the differences between accuracy of psychiatric disorders and that of physical conditions is a discussion about the differences in jamapsychiatry.com
the approach used to classify the physical compared with psychiatric conditions. The measures that provide the building blocks of psychiatric diagnosis in this study are reports of individual symptoms. The DSM diagnoses are constructed post hoc by the researchers based on the endorsement of a grouping of self-reported symptoms without any explicit discussion with the study participant about a diagnostic label. Physical conditions, on the other hand, are measured based on the respondent’s self-report of a diagnosis, presumably because a physician has diagnosed and informed them to that effect. It is clear that long-term recall of psychiatric symptoms has weaknesses, but is it really worse than recall of physical symptoms? Are psychiatric disorders reported as having been given by a physician different from report of physician-diagnosed physical conditions? We just do not know. Nevertheless, the important message about the limitations of recall of earlier symptoms is significant and suggests particular caution in interpreting lifetime diagnoses that may rely on reports of earlier symptoms. Results documented by the authors should be considered in the context of other methodological issues as well, such as participant retention, case severity, and subthreshold diagnoses. The overall 53% follow-up rate at the wave 4 interview suggests the possibility that those lost to follow-up might have been the more severe cases, for whom case agreement between cumulative and retrospective estimates might have been greater. In addition, less severe cases could easily fall to subthreshold if only a single clinical feature is forgotten. Neither the number of clinical features endorsed toward a particular DSM diagnosis nor the frequency of subthreshold and barely threshold cases is reported. Might these or other biases influence the results? Possibly, although it is doubtful that differential attrition or other methodological issues could account for the 3- to 4-fold difference in rates identified by the authors. In addition, thoughtful sensitivity analyses examining the importance of the change in diagnostic systems across the waves of follow-up and the potential influence of cognitive impairment on findings are all essential in ensuring the underlying validity of the findings. Although this is the first study of the issue of recalled lifetime diagnosis compared with prospectively ascertained diagnosis in adults, the results are quite consistent with the differences in cumulative vs cross-sectional lifetime rates in adolescents and young adults in studies by Copeland et al3 and Moffitt et al.4 Thus, we are confident that Takayanagi and colleagues have identified a significant concern, and we are encouraged to reexamine how we use the results from cross-sectional surveys. JAMA Psychiatry March 2014 Volume 71, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpsyc.jamanetwork.com/ by a University of New South Wales User on 08/12/2015
233
Opinion Editorial
Most service delivery planning is based on an estimate of point (or annual) prevalence to help with resource allocation. Trends depend on multiple repeated measures to identify significant changes in prevalence over time. For these purposes, cross-sectional studies still have extraordinary value, and we will not stop conducting them. However, Takayanagi and colleagues remind us quite clearly that a cross-sectional study design should not be applied indiscriminately. Overall, the findings from this study are an important reminder to be quite cautious in using lifetime prevalence rates generated from a single cross-sectional study. At a minimum, we should see the lifetime prevalence estimates generated from cross-sectional studies as the low estimates and as a starting point for inquiry rather than as an accurate depiction of the true burden of disease. ARTICLE INFORMATION Author Affiliations: National Institute on Drug Abuse, Bethesda, Maryland. Corresponding Author: Wilson M. Compton, MD, MPE, National Institute on Drug Abuse, 6001 Executive Blvd, MSC 9589, Bethesda, MD 20892 ([email protected]). Published Online: January 8, 2014. doi:10.1001/jamapsychiatry.2013.4111. Conflict of Interest Disclosures: Dr Compton reports stock holdings in Pfizer, General Electric, and 3M. No other disclosures were reported. Disclaimer: The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the National Institute on Drug
We congratulate Takayanagi and colleagues on identifying an important limitation in survey methods as applied to psychiatric illnesses where stronger methods that provide multiple assessments across time and/or from different informants are suggested. The more we learn about psychiatric conditions, the greater the need is for external validators and endophenotypes.5 Only when we know the underlying markers of disease and enhance clinical appraisal with genetic or neuroscience measures will we be able to completely accurately understand the prevalence of these conditions. Until such a day when these markers and measures are available, we will rely on multiple sources of information that triangulate on results as the next best alternative for confirming our insights and improving our knowledge.
Abuse, the National Institutes of Health, or the US Department of Health and Human Services. REFERENCES 1. Takayanagi Y, Spira AP, Roth KB, Gallo JJ, Eaton WW, Mojtabai R. Accuracy of reports of lifetime mental and physical disorders: results from the Baltimore Epidemiological Catchment Area Study [published online January 8, 2013]. JAMA Psychiatry. doi:10.1001/jamapsychiatry.2013.3579. 2. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry. 2000;39(1):28-38.
3. Copeland W, Shanahan L, Costello EJ, Angold A. Cumulative prevalence of psychiatric disorders by young adulthood: a prospective cohort analysis from the Great Smoky Mountains Study. J Am Acad Child Adolesc Psychiatry. 2011;50(3):252-261. 4. Moffitt TE, Caspi A, Taylor A, et al. How common are common mental disorders? evidence that lifetime prevalence rates are doubled by prospective versus retrospective ascertainment. Psychol Med. 2010;40(6):899-909. 5. Jupp B, Dalley JW. Behavioral endophenotypes of drug addiction: etiological insights from neuroimaging studies [published online June 10, 2013]. Neuropharmacology. doi:10.1016 /j.neuropharm.2013.05.041.
Mortality in Patients With Traumatic Brain Injury Robert G. Robinson, MD
The current issue of JAMA Psychiatry includes an important article on premature mortality among patients with traumatic brain injury (TBI) in Sweden between 1969 and 2009.1 According to the study by Fazel et al,1 among 218 300 patients with a Related article page 326 TBI compared with age- and sex-matched controls without brain injury (10 to 1 match, n = 2 163 190) and unaffected siblings of TBI patients (n = 150 513), there was a 3-fold increased odds of all-cause mortality, adjusted for sociodemographic confounders (adjusted odds ratio [aOR], 3.2; 95% CI, 3.0-3.4), among patients who survived at least 6 months after TBI compared with general population controls or unaffected siblings (aOR, 2.6; 95% CI, 2.3-2.8). The increased rates of mortality were related to injury (aOR, 4.3; 95% CI, 3.8-4.8), assault (aOR, 3.9; 95% CI, 2.75.7), or suicide (aOR, 3.3; 95% CI, 2.9-3.7). A major strength of this study, which used national Swedish patient registry data, was the statistical power derived from the large database. This database allowed the authors to examine, in detail, causes of early mortality when the overall frequency of premature mortality was relatively small. Among the 218 300 patients with TBI, premature death, which occurred after 6 234
months following the TBI and before 56 years of age, constituted only 2378 (1.1%) of the TBI cases. Furthermore, the causes of mortality, which include multiple physical illnesses and external causes, such as motor vehicle collisions, other injuries, suicide, and assault, were compared with controls and examined in detail. Consequently, this study had the power to compare death rates in a small portion of the population. External causes of mortality constituted almost half (48.6%) of the premature deaths compared with physical disorders, such as neoplasms, which caused premature death in 11.1% of the 2378 cases. Traumatic brain injury has become a topic of public interest in the United States within the past 10 years because of the common causes of brain injury, such as combat, sports (particularly football), and unprovoked assaults. These potentially preventable injuries have attracted the attention of the military leadership, the families of injured people, politicians, sports organizations (particularly the National Football League), psychiatrists, psychologists, attorneys, and others. The pathologic features, symptoms, and course of TBI can be widely different depending on the cause, type (eg, contusion, shear, and intracranial bleed), or severity of the injury. Many of the US soldiers from the Iraq and Afghanistan wars experienced TBI from shock waves
JAMA Psychiatry March 2014 Volume 71, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpsyc.jamanetwork.com/ by a University of New South Wales User on 08/12/2015
jamapsychiatry.com
EDITORIAL
Accuracy in Reporting Past Psychiatric Symptoms The Role of Cross-sectional Studies in Psychiatric Research Wilson M. Compton, MD, MPE; Marsha F. Lopez, PhD, MHS
Cross-sectional studies remain a major source of data for research, practice, and policy, despite well-known limitations of this approach in assessing psychopathology. Such studies are especially valuable when the intent is to identify current or recent cases and in repeated Related article page 273 studies where trends in rates can be examined. Yet, how well do they identify psychiatric symptoms and syndromes from the past? This is not a trivial question because the accuracy of such classification is important for association studies and, in particular, for clinical neuroscience and genetic studies where the accuracy of phenotypic classification is essential to determining the relationship of predictors and purported outcomes. Taken at face value, Takayanagi et al1 demonstrate that cross-sectional studies are much worse than longitudinal in identifying a history of psychiatric disorders. The authors find marked underestimates in lifetime prevalence estimates from a single cross-sectional study, and this lack of agreement with psychiatric symptom reporting across the waves of interviews is remarkable. Among those who received a diagnosis in a prior wave, 92% of those with obsessive-compulsive disorder and about two-thirds of those with major depressive disorder, panic disorder, and substance abuse or dependence (both alcohol and other drugs) did not report diagnostic symptoms sufficient for a diagnosis at the fourth interview. Also, somewhat surprisingly, recent receipt of mental health services made little difference in the accuracy of reporting of lifetime symptoms. Undoubtedly, a longitudinal study design has many advantages for inquiry: clear separation of risk and outcome variables, repeated measurement of psychopathology to identify acute/temporary and chronic symptom patterns, and less reliance on long-term memory for reports. Similarly, assessment that includes multiple distinct informants (typical in child psychiatry where parents and teachers are routinely queried) can improve diagnostic coverage.2 The key issue for researchers is to determine when this additional information is worth the extra effort (and time) to acquire. A secondary and intriguing question addressed by Takayanagi and colleagues is whether there is something unique about mental illnesses that makes them particularly prone to poor recall compared with identifying physical ailments. Their results suggest that physical conditions are much more consistently reported than mental illnesses. However, absent from the conclusions by Takayanagi and colleagues regarding the differences between accuracy of psychiatric disorders and that of physical conditions is a discussion about the differences in jamapsychiatry.com
the approach used to classify the physical compared with psychiatric conditions. The measures that provide the building blocks of psychiatric diagnosis in this study are reports of individual symptoms. The DSM diagnoses are constructed post hoc by the researchers based on the endorsement of a grouping of self-reported symptoms without any explicit discussion with the study participant about a diagnostic label. Physical conditions, on the other hand, are measured based on the respondent’s self-report of a diagnosis, presumably because a physician has diagnosed and informed them to that effect. It is clear that long-term recall of psychiatric symptoms has weaknesses, but is it really worse than recall of physical symptoms? Are psychiatric disorders reported as having been given by a physician different from report of physician-diagnosed physical conditions? We just do not know. Nevertheless, the important message about the limitations of recall of earlier symptoms is significant and suggests particular caution in interpreting lifetime diagnoses that may rely on reports of earlier symptoms. Results documented by the authors should be considered in the context of other methodological issues as well, such as participant retention, case severity, and subthreshold diagnoses. The overall 53% follow-up rate at the wave 4 interview suggests the possibility that those lost to follow-up might have been the more severe cases, for whom case agreement between cumulative and retrospective estimates might have been greater. In addition, less severe cases could easily fall to subthreshold if only a single clinical feature is forgotten. Neither the number of clinical features endorsed toward a particular DSM diagnosis nor the frequency of subthreshold and barely threshold cases is reported. Might these or other biases influence the results? Possibly, although it is doubtful that differential attrition or other methodological issues could account for the 3- to 4-fold difference in rates identified by the authors. In addition, thoughtful sensitivity analyses examining the importance of the change in diagnostic systems across the waves of follow-up and the potential influence of cognitive impairment on findings are all essential in ensuring the underlying validity of the findings. Although this is the first study of the issue of recalled lifetime diagnosis compared with prospectively ascertained diagnosis in adults, the results are quite consistent with the differences in cumulative vs cross-sectional lifetime rates in adolescents and young adults in studies by Copeland et al3 and Moffitt et al.4 Thus, we are confident that Takayanagi and colleagues have identified a significant concern, and we are encouraged to reexamine how we use the results from cross-sectional surveys. JAMA Psychiatry March 2014 Volume 71, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpsyc.jamanetwork.com/ by a University of New South Wales User on 08/12/2015
233
Opinion Editorial
Most service delivery planning is based on an estimate of point (or annual) prevalence to help with resource allocation. Trends depend on multiple repeated measures to identify significant changes in prevalence over time. For these purposes, cross-sectional studies still have extraordinary value, and we will not stop conducting them. However, Takayanagi and colleagues remind us quite clearly that a cross-sectional study design should not be applied indiscriminately. Overall, the findings from this study are an important reminder to be quite cautious in using lifetime prevalence rates generated from a single cross-sectional study. At a minimum, we should see the lifetime prevalence estimates generated from cross-sectional studies as the low estimates and as a starting point for inquiry rather than as an accurate depiction of the true burden of disease. ARTICLE INFORMATION Author Affiliations: National Institute on Drug Abuse, Bethesda, Maryland. Corresponding Author: Wilson M. Compton, MD, MPE, National Institute on Drug Abuse, 6001 Executive Blvd, MSC 9589, Bethesda, MD 20892 ([email protected]). Published Online: January 8, 2014. doi:10.1001/jamapsychiatry.2013.4111. Conflict of Interest Disclosures: Dr Compton reports stock holdings in Pfizer, General Electric, and 3M. No other disclosures were reported. Disclaimer: The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the National Institute on Drug
We congratulate Takayanagi and colleagues on identifying an important limitation in survey methods as applied to psychiatric illnesses where stronger methods that provide multiple assessments across time and/or from different informants are suggested. The more we learn about psychiatric conditions, the greater the need is for external validators and endophenotypes.5 Only when we know the underlying markers of disease and enhance clinical appraisal with genetic or neuroscience measures will we be able to completely accurately understand the prevalence of these conditions. Until such a day when these markers and measures are available, we will rely on multiple sources of information that triangulate on results as the next best alternative for confirming our insights and improving our knowledge.
Abuse, the National Institutes of Health, or the US Department of Health and Human Services. REFERENCES 1. Takayanagi Y, Spira AP, Roth KB, Gallo JJ, Eaton WW, Mojtabai R. Accuracy of reports of lifetime mental and physical disorders: results from the Baltimore Epidemiological Catchment Area Study [published online January 8, 2013]. JAMA Psychiatry. doi:10.1001/jamapsychiatry.2013.3579. 2. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry. 2000;39(1):28-38.
3. Copeland W, Shanahan L, Costello EJ, Angold A. Cumulative prevalence of psychiatric disorders by young adulthood: a prospective cohort analysis from the Great Smoky Mountains Study. J Am Acad Child Adolesc Psychiatry. 2011;50(3):252-261. 4. Moffitt TE, Caspi A, Taylor A, et al. How common are common mental disorders? evidence that lifetime prevalence rates are doubled by prospective versus retrospective ascertainment. Psychol Med. 2010;40(6):899-909. 5. Jupp B, Dalley JW. Behavioral endophenotypes of drug addiction: etiological insights from neuroimaging studies [published online June 10, 2013]. Neuropharmacology. doi:10.1016 /j.neuropharm.2013.05.041.
Mortality in Patients With Traumatic Brain Injury Robert G. Robinson, MD
The current issue of JAMA Psychiatry includes an important article on premature mortality among patients with traumatic brain injury (TBI) in Sweden between 1969 and 2009.1 According to the study by Fazel et al,1 among 218 300 patients with a Related article page 326 TBI compared with age- and sex-matched controls without brain injury (10 to 1 match, n = 2 163 190) and unaffected siblings of TBI patients (n = 150 513), there was a 3-fold increased odds of all-cause mortality, adjusted for sociodemographic confounders (adjusted odds ratio [aOR], 3.2; 95% CI, 3.0-3.4), among patients who survived at least 6 months after TBI compared with general population controls or unaffected siblings (aOR, 2.6; 95% CI, 2.3-2.8). The increased rates of mortality were related to injury (aOR, 4.3; 95% CI, 3.8-4.8), assault (aOR, 3.9; 95% CI, 2.75.7), or suicide (aOR, 3.3; 95% CI, 2.9-3.7). A major strength of this study, which used national Swedish patient registry data, was the statistical power derived from the large database. This database allowed the authors to examine, in detail, causes of early mortality when the overall frequency of premature mortality was relatively small. Among the 218 300 patients with TBI, premature death, which occurred after 6 234
months following the TBI and before 56 years of age, constituted only 2378 (1.1%) of the TBI cases. Furthermore, the causes of mortality, which include multiple physical illnesses and external causes, such as motor vehicle collisions, other injuries, suicide, and assault, were compared with controls and examined in detail. Consequently, this study had the power to compare death rates in a small portion of the population. External causes of mortality constituted almost half (48.6%) of the premature deaths compared with physical disorders, such as neoplasms, which caused premature death in 11.1% of the 2378 cases. Traumatic brain injury has become a topic of public interest in the United States within the past 10 years because of the common causes of brain injury, such as combat, sports (particularly football), and unprovoked assaults. These potentially preventable injuries have attracted the attention of the military leadership, the families of injured people, politicians, sports organizations (particularly the National Football League), psychiatrists, psychologists, attorneys, and others. The pathologic features, symptoms, and course of TBI can be widely different depending on the cause, type (eg, contusion, shear, and intracranial bleed), or severity of the injury. Many of the US soldiers from the Iraq and Afghanistan wars experienced TBI from shock waves
JAMA Psychiatry March 2014 Volume 71, Number 3
Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: http://archpsyc.jamanetwork.com/ by a University of New South Wales User on 08/12/2015
jamapsychiatry.com
![[Ethics in psychiatric research].](/assets/img/hold.png)
